Will Search Generative Experience (SGE) really transform web search?

As soon as ChatGPT took off in late 2022, impressed users were speculating that the chatbot could be the ‘death’ of Google, due to its facility for fluently answering queries of all kinds.

Although at the time ChatGPT appeared hampered by its lack of internet access and up-to-date information (as its training dataset originally did not cover anything beyond the year 2021), many predicted that with these additions, it would quickly be eating Google’s lunch. Econsultancy’s first coverage of ChatGPT (penned by yours truly) examined whether ChatGPT really could develop into a viable search engine competitor.

Microsoft evidently thought so, debuting Bing Chat to great fanfare in early February 2023 and showing off in an ebullient presentation the AI-enhanced search engine’s ability to answer complex, multi-part queries about products, travel itineraries and influential artists. Google responded with an apparently hastily thrown-together presentation of its equivalent product, Bard.

Unfortunately for Google, onlookers were quick to realise that one of the Bard-generated answers in its promotional material was incorrect, something that coincided with a drop in parent company Alphabet’s stock price to create headlines about ‘Google chatbot’s $100bn error’ (in reality, the stock price drop was likely related to Google’s broader presentation rather than the factual mistake).

Microsoft was widely proclaimed to have ‘won’ the battle of chat-enhanced search, but in truth, its mistakes were simply less well-publicised; Dmitri Brereton took to Substack to point out numerous errors in Microsoft’s own demonstration. “Bing AI got some answers completely wrong during their demo. But no one noticed. Instead, everyone jumped on the Bing hype train,” he wrote.

Alphabet CEO Sundar Pichai has admitted that these factual issues are inherent to “all” AI models, adding that, “No one in the … field has yet solved the hallucination problems.”

Hallucinations have serious implications for generative AI in most circumstances, but particularly in a situation where people will be looking for concrete facts – web search – and while it may be possible to check the AI’s generated responses, that rather defeats the object of offering an AI-generated answer in the first place.

The contradictions of generative AI-enhanced search

The appeal of an AI assistant answering questions rather than a search engine is that the assistant can provide a definitive response, even to a complex query – but if that response contains errors, it would seriously undermine public trust in the tool and could also lead to misinformation being perpetuated.

Google identified this risk itself when asked by employees why it had been slow to respond to the threat of ChatGPT, despite years of developing its own large language models; AI lead Jeff Dean stated that, “[ChatGPT] really strikes a need that people seem to have but it’s also important to realize these models have certain type of issues.”

This didn’t prevent Google from releasing Bard, even in a limited capacity, in response to Bing Chat – or from announcing what it has dubbed Search Generative Experience (SGE), which Google boasts “can unlock entirely new types of questions you never thought Search could answer, and transform the way information is organized”. However, despite this fanfare, Google continues to make SGE available only as an experiment in Search Labs, even as it expands the new search experience with additional languages and regions.

Barry Schwartz of Search Engine Roundtable also noticed that Google had quietly removed the original end date of its SGE trial, no longer committing to a time when SGE would be deployed fully.

What of Microsoft, who jumped in feet first with a public deploy of generative AI-enhanced search? Search Engine Roundtable recently observed that close to a year on from Microsoft’s launch of Bing Chat, the company’s market share had grown by less than 1%.

Microsoft would obviously be facing a huge uphill climb to close the gap between itself and Google, with less than 4% market share to Google’s more than 91%, according to GlobalStats. However, one would think that Microsoft would be making more inroads than it has if its new search were fulfilling people’s needs the way the company has promised. It’s also telling that Microsoft has yet to break out Bing Chat data within the Bing Webmaster Tools Performance Report, despite initial promises to do so.

Microsoft and Google will both be aware of two major conundrums with SGE: first, the potential for hallucinations, which threatens to detract from the benefits of having generative AI supply answers to complex queries. And secondly, the possibility of alienating publishers, many of whom are none too pleased at having their website content form the basis of generative AI answers while potentially losing out on clicks through to that content.

In a prominent copyright lawsuit currently underway, the New York Times cited examples from Browse With Bing, a feature available to ChatGPT Plus subscribers that lets them access the internet, in which Browse With Bing allegedly reproduced text from its product review site, Wirecutter, without linking to the original or including affiliate links. The Times wrote that,

“Decreased traffic to Wirecutter articles and, in turn, decreased traffic to affiliate links subsequently lead to a loss of revenue for Wirecutter.”

Few complaints from publishers are likely to veer into the realm of legal action, and some are even exploring reciprocal arrangements with generative AI, such as the Associated Press, which signed a two-year deal in July that would provide OpenAI with access to select news content in exchange for the use of OpenAI’s technology and product expertise. Meanwhile, Apple is reported to be negotiating with major news outlets to access their content for development of its own large language model.

Nevertheless, with generative AI not proving to be a major winner in search for Microsoft – which is more focused in integrating it into other tools – it seems unlikely that Google will rush to roll out SGE unless (or until) it can be sure that the benefits are worth the risks.