Understanding Google’s AI Mode: Quality, Helpfulness, and SEO

Google’s Vice President of Product for Google Search, Robby Stein, recently sat down for an interview to discuss how Google’s AI Mode handles quality, evaluates helpfulness, and leverages its experience with search to identify helpful content. In this article, we will delve into the key points discussed during the interview, including how Google controls hallucinations, evaluates helpfulness in AI Mode, and the five quality SEO-related factors used for AI Mode.

How Google Controls Hallucinations

During the interview, Stein was asked about hallucinations, where an AI lies in its answers. He explained that the quality systems within AI Mode are based on everything Google has learned about quality from 25 years of experience with classic search. The systems that determine what links to show and whether content is good are encoded within the model and are based on Google’s experience with classic search. Stein stated that while AI and generative AI are new, thinking about quality systems for information is something that’s been happening for 20-25 years. All of these AI systems are built on top of those quality systems, and there’s an incredibly rigorous approach to understanding what good information is and what users value.

Grounding AI Answers in Classic Search Signals

Stein’s explanation makes it clear that AI Mode is encoded with everything learned from Google’s classic search systems rather than a rebuild from scratch or a break from them. The risk of hallucinations is managed by grounding AI answers in the same relevance, trust, and usefulness signals that have underpinned classic search for decades. Those signals continue to determine which sources are considered reliable and which information users have historically found valuable. Accuracy in AI search follows from that continuity, with model reasoning guided by longstanding search quality signals rather than operating independently of them.

How Google Evaluates Helpfulness In AI Mode

The next question Stein answered was about the quality signals that Google uses within AI Mode. He explained that the way AI Mode determines quality is very much the same as with classic search. Google looks at a whole battery of things, including evaluating the content offline with real people, looking at actual responses, and whether people are giving thumbs up or thumbs down. Stein also mentioned that they look at whether people are using the information, coming back to it, and voting with their feet because it’s valuable to them.

Evaluating Success with Multiple Signals

Stein’s answer shows that AI Mode evaluates success using the same core signals used for search quality, even as the interface becomes more dynamic. Usefulness is not inferred from a single engagement signal but from a combination of human evaluation, explicit feedback, and behavioral patterns over time. Importantly, Stein notes that just because people use it a lot, presumably in a single session, that the increased usage alone is not treated as success, since repeated attempts to answer the same query indicate failure rather than satisfaction.

Five Quality Signals For AI Search

Lastly, Stein answered a question about the ranking of AI-generated content and if SEO best practices still help for ranking in AI. Stein’s answer includes five factors that are used for determining if a website meets their quality and helpfulness standards. These factors include:
1. Is your content directly answering the user’s question?
2. Is it high quality?
3. Does it load quickly?
4. Is it original?
5. Does it cite sources?

Applying SEO Best Practices

Stein emphasized that the same things apply to AI search as to classic search. If people click on the content, value it, and come back to it, that content will rank for a given question and it will rank in the AI world as well. This means that SEO best practices, such as creating high-quality, original content that loads quickly and cites sources, are still essential for ranking in AI search.

Conclusion

In conclusion, Google’s AI Mode is built on the foundation of classic search quality systems, and it evaluates helpfulness using the same core signals. The five quality SEO-related factors used for AI Mode are essential for determining if a website meets Google’s quality and helpfulness standards. By understanding how Google controls hallucinations, evaluates helpfulness, and applies SEO best practices, we can better navigate the world of AI search and create content that is valuable and useful to users.