When Sundar Pichai, Google’s chief executive, introduced a generative artificial intelligence feature for the company’s search engine last month, he and his colleagues demonstrated the new capability with six text-based queries that the public could try out.
The questions included “how do you clean a fabric sofa” and “what should I use to get a coffee stain out of my carpet.” These were intended to highlight how Google’s new feature, A.I. Overviews, could generate full and useful information summaries above traditional search results.
But by Friday, only one of the six queries still yielded an A.I. Overview, according to tests by The New York Times. Instead, the feature was noticeably less prevalent. The search for “what should I use to get a coffee stain out of my carpet” now resulted in a snippet of text from a website, JDog Carpet Cleaning & Floor Care, while “how do you clean a fabric sofa” was replaced by a link to HGTV’s website with the answer. (The results of the searches may vary depending on the user and location.)
The disappearance of A.I. Overviews for some of the searches appeared to be part of a broader rollback after the new technology produced a litany of untruths and errors — including recommending glue as part of a pizza recipe and suggesting that people ingest rocks for nutrients. Users loudly complained on social media about the mistakes, in many cases outright making fun of Google.
Liz Reid, who was recently promoted to Google’s head of search, wrote in a blog post on Thursday that the company had pared back A.I. Overviews in certain ways, launching “additional triggering refinements” to offer more careful responses about health, disabling misleading advice and limiting the inclusion of satire and user responses from forums like Reddit.
“We’ll keep improving when and how we show AI Overviews and strengthening our protections,” she wrote, adding that Google was working on updates to improve broad sets of search results.
Ashley Thompson, a Google spokeswoman, said in a statement on Friday that the company had made more than a dozen technical updates to its systems.
“A.I. Overviews are helping people on a large number of queries on Search today, serving as a jumping-off point to content across the web.” The company added that while it was making adjustments to improve A.I. Overviews, it was not pulling back from the feature for the long term.
The backtracking was a blow to Google’s efforts to keep up with its rivals Microsoft and OpenAI, the maker of the ChatGPT chatbot, in the frenzied race to lead A.I. It also underscored the difficult strategic choice that Google faces over whether to embrace A.I. technology that may not be dependable, or keep its highly popular search engine the same — and risk falling behind its peers.
Google had chosen to go slower than Microsoft, which placed more conversational A.I. into its Bing search engine early last year. Google, which has significantly more users than Bing, tested A.I. features for its search engine a year before introducing A.I. Overviews. The company said the new feature would be rolled out to users in the United States immediately and to more than a billion people by the end of the year.
But ultimately, Google ”should have rolled this out more slowly,” said Patrick Hall, an assistant professor of decision sciences at the George Washington University School of Business. “Once something like this happens, you really have to retreat. And if nothing else, there’s reputational harm” to the company.
Google, which has led internet search for more than two decades, has scrambled since OpenAI released ChatGPT in 2022. Some tech industry insiders considered the chatbot’s ability to generate answers to be a serious threat to Google’s search engine, which has been the most popular way to get information online.
Since then, Google has aggressively worked to regain its advantage in A.I., releasing a family of technology named Gemini, including new A.I. models for developers. The company also infused the technology into YouTube, Gmail and Docs, helping users create videos, emails and drafts with less effort.
Last month, Ms. Reid said at Google’s developer conference that the search engine would do more of the googling for users with A.I. Overviews. She highlighted increasingly complex requests that Google could answer with the feature, but those capabilities have not yet been launched for users.
Onstage, simpler questions like “how do I get the smell of a campfire out of my clothes” yielded A.I. answers including air it out, add baking soda and spray with lemon juice.
But when users got their hands on the new service, they found that A.I. Overviews sometimes generated wrong — or downright dangerous — answers, including recommending putting nontoxic glue on pizza to make the cheese stick. It also misunderstood some websites it was quoting and got presidential history wrong.
Of the six questions that Google launched for the public, the only one that consistently triggered an A.I. Overview on Friday was “what are interesting science projects I can do with my son who is 12 years old.”
The answer? Growing crystals, extracting DNA from saliva and writing a message with invisible ink.
Discussion about this post