Google Is Playing a Dangerous Game With AI Search

🌈 Abstract

The article discusses the risks and concerns around Google's new AI-powered search feature that generates answers to health-related queries, highlighting the potential for inaccurate or harmful information to be provided.

🙋 Q&A

[01] Google's New AI Search Feature

1. What are the key concerns raised about Google's new AI search feature?

• The article expresses concerns that Google's AI-generated answers on health-related queries could be inaccurate or potentially harmful, as the AI system has been found to provide incorrect information on various topics.
• There are worries that people seeking immediate answers may rely solely on the AI overview without digging deeper into the search results, which could lead to real-world harm if the information is wrong, especially for sensitive health questions.
• The article notes that while Google claims to have higher quality standards for health queries, the AI system has still provided some flagrantly wrong or weird responses on health topics.
• There are questions about Google's ability to properly moderate and control the AI system, given the vast number of unique search queries it faces daily.

2. How does the article describe the implementation and scope of Google's new AI search feature?

• The feature generates AI-written summaries that appear at the top of search results, above links to external websites.
• It is being rolled out to all Google users in the United States as one of the biggest design changes in recent years.
• The AI overviews appear for many health-related queries, including sensitive topics like cancer, heart attacks, and medication like Ozempic.
• Google says it has an "even higher bar for quality" on health queries, but the article found inconsistencies in which queries triggered the AI response.

3. What are some examples of inaccurate or concerning information provided by Google's AI search feature?

• The AI asserted that dogs have played in the NFL and that President Andrew Johnson had 14 degrees from the University of Wisconsin.
• On health topics, the AI provided responses saying rocks are safe to eat, that chicken is safe to eat at 102°F, and that pregnant women can eat sushi as long as it doesn't contain raw fish (which is inaccurate).
• For a query on chemotherapy, the AI highlighted a statistic about one-year survival rates for head and neck cancers, presenting it as applicable to all cancers.

[02] Risks and Limitations of AI-Powered Search

1. What are the key risks and limitations of Google's AI-powered search feature highlighted in the article?

• The article notes that health is a sensitive area for a technology company's experimental AI system, as health questions contain the potential for real-world harm if answered even partially incorrectly.
• There are concerns that AI responses could stoke unnecessary anxiety about illnesses, or miss critical signs of serious conditions like allergic reactions.
• The article points out that Google's content moderation challenges will only be exacerbated by the introduction of generative AI, as there are many ways to phrase any given search query.
• The tool has shown inconsistencies in which queries it will respond to, sometimes refusing to answer a question but then providing a response to a similar version of the same query.

2. How does the article suggest Google's AI search feature may have limitations compared to traditional search results?

• Wading through a list of search results can be tedious, so the AI overview feature may be helpful in summarizing key information.
• However, the article argues the AI tool may never be perfect, as there will always be ways for users to "jailbreak" the system with creative prompts.
• If the AI overviews are inconsistent and problematic even for health queries, which Google claims to prioritize, the article questions what the quality and reliability will be for all other types of searches.