I came across a February 19, 2024 research summary published by Gartner titled Gartner Predicts Search Engine Volume Will Drop 25% by 2026, Due to AI Chatbots and Other Virtual Agents. I do not have a strong opinion on whether this will prove to be an accurate market assessment or whether Gartner’s projections are typically sound (I never heard of the organization before seeing this article). On one hand I am skeptical that ordinary internet users will prefer “AI” chatbots to entering brute force search terms into Google. On the other hand, I hear that the big tech players including, but not limited to, Google and Microsoft, are very aggressively pushing their large language models and chatbots at users.
But regardless of whether Gartner’s prediction is accurate, I do find the trend of conflating generative “AI” with traditional search concerning. Back when we were randomly blacklisted by the Bing search engine, I wrote an article expressing my concern that language models being integrated into search engines were borrowing from writing by humans without proper attribution. Moreover, I noted that even in the cases where the search engine-language model duo cites its sources, Google, Microsoft, and the like are incentivized to design things in such a way as to keep users on their portals instead of on sites linked from their portals.
One quote from Mr. Alan Antin, a Vice President Analyst at Gartner (I have a feeling this Gartner company has many Vice Presidents), struck me as particularly interesting:
Generative AI (GenAI) solutions are becoming substitute answer engines, replacing user queries that previously may have been executed in traditional search engines. This will force companies to rethink their marketing channels strategy as GenAI becomes more embedded across all aspects of the enterprise.
Let us key in on Mr. Antin’s use of the term “answer engine.” An answer presupposes the existence of a question. The implication here is that these GenAI “solutions” exist to answer questions. That is generally consistent with my understanding of the main purpose of these GenAI tools insofar as they are integrated with traditional search engines and tools.
Speaking of traditional search engines, I highlight that Mr. Antin appears to distinguish the concept of an answer engine from a traditional search engine. This is the correct approach. One can ask a search engine questions – and search engines have increasingly integrated non-search features such as instant answer functionality to prioritize delivering responses to questions instead of delivering searchers to an external source that may answer their question. But the concept of a search engine extends beyond answering questions. For example, just earlier today (on the day I am writing this draft – not the day it is published), I wanted to find a specific quote on Project Gutenberg. I used a site-specific DuckDuckGo search to search gutenberg.org for the quote (placing the quote in quotation marks) and found one result. One could say I was implicitly asking DuckDuckGo a question – to find all examples of a specific quotation in resources on gutenberg.org. But this was not a question as the term would be understood in ordinary interaction. I simply pointed DuckDuckGo at a domain and gave it an exact term to return results for.
It is important to resist conflating GenAI tools with traditional search engines. From the perspective of someone who runs a small, humane (as in by a human for other human beings) website, I prefer search engines – properly conceived – because they ideally point searchers toward internet writing responsive to their queries instead of trying to answer questions based on third-party writing. From the perspective of someone who enjoys reading thoughtful and informative writing around the web, I prefer a tool that points me toward good writing rather than one that tries to replace it. (Note that for the purpose of the instant discussion – I am only focusing on GenAI as a quasi-replacement for search, as as a tool to do things such as generate photos.)
None of this is to say that traditional search engines are in the best place at the moment. The biggest search engines are run by ad companies, which most likely negatively affects results. The use of GenAI for writing – Gartner euphemistically noted that “GenAI [is] driving down the cost of producing content” – is already making it more difficult to find humane “content” through big tech search. As I learned last year, arbitrary and capricious decisions by one search engine against a legitimate website can have significant downstream search effects because most of the alternatives to the Google-Bing duopoly use Google’s or Bing’s search index. Moreover, search engines have been slowly adding functionality to keep searchers in their portals before the GenAI fad – one such feature being instant answers-type additions. I note this all only to say that I think there are problems in search today for small website owners and people who want to find interesting things on the internet that pre-date and go beyond GenAI.
There is no easy solution to all of this for people who want others to find their humane websites or searchers who want to discover humane websites. But I think there are ways to zag while big tech and big GenAI insists you zig. One should always be an active searcher. If using a big tech search engine or derivative thereof, consider how the search engine interprets search queries and how it can be manipulated to deliver more interesting and/or responsive results. Understand the sort of junk results that it may favor for one query or another. Regarding GenAI or other similar integrations, stick to trying to use a search engine as a traditional search engine – that is as a means to finding an interesting external resource instead of the end in and of itself. As I did in 2021, I recommend using different search engines and tools for different purposes. For example, the free and open source Marginalia Search engine is a terrific tool for finding small web sites and older sites and forums that would never turn up with bigger indexes. Rely on internal and external links within articles on good sites. Sometimes the best solution is either to use a website’s own on-site search functionality or to use a bigger, generalist search engine to conduct a domain-specific search of a particular website. I have over time come to favor more and more searching individual websites I like or trust for information. If you incorporate this last tip into your repertoire, I will note that The New Leaf Journal’s internal search – while not the most sophisticated – should turn up some good results from our 1,000+ articles.