Introduction to the Experiment
Aleyda Solís conducted an experiment to test how fast ChatGPT indexes a web page. She created a brand new page titled "LLMs.txt Generators" on her website, LearningAISearch.com, and immediately tested ChatGPT to see if it could access or locate the page. However, ChatGPT failed to find it and responded with the suggestion that the URL was not publicly indexed or possibly outdated.
The Experiment Unfolds
Aleyda then asked Google Gemini about the web page, which successfully fetched and summarized the live page content. She next submitted the web page for indexing via Google Search Console and Bing Webmaster Tools. Google successfully indexed the web page, but Bing had problems with it. After several hours elapsed, Google started showing results for the page with the site operator and with a direct search for the URL. But Bing continued to have trouble indexing the web page.
ChatGPT’s Reliance on Google Search Results
Aleyda went back to ChatGPT and, after several tries, it gave her an incomplete summary of the page content, mentioning just one tool that was listed on it. When she asked ChatGPT for the origin of that incomplete snippet, it responded that it was using a "cached snippet via web search," likely from "search engine indexing." She confirmed that the snippet shown by ChatGPT matched Google’s search result snippet, not Bing’s, which still hadn’t indexed it.
Analysis of the Findings
Aleyda explained that when she followed up asking where the snippet came from, ChatGPT answered that it had "located a cached snippet via web search that previews the page content – likely from search engine indexing." However, she knew the page wasn’t indexed yet in Bing, so it had to be Google search results. She went to check and compared the text snippet provided by ChatGPT vs the one shown in Google Search Results for the specific page, confirming it was the same information.
Not an Isolated Incident
Aleyda’s article on her finding links to someone else’s web page that summarizes a similar experience where ChatGPT used a Google snippet. So, she’s not the only one to experience this. Others have also observed similar behavior, with ChatGPT piggybacking off Google snippets to generate answers.
Implications for Traditional SEO
The experiment shows that traditional SEO remains relevant for AI search. As Kyle Atwater Morley shared his observation, "So ChatGPT is basically piggybacking off Google snippets to generate answers? What a wake-up call for anyone thinking traditional SEO is dead." Stéphane Bureau shared his opinion on what’s going on, "If Bing’s results are insufficient, it appears to fall back to scraping Google SERP snippets."
Theory on ChatGPT’s Behavior
Based on current evidence, the theory is that when browsing is enabled, ChatGPT sends search requests via Bing first. However, if Bing’s results are insufficient or outdated, it appears to fall back to scraping Google SERP snippets—likely via an undocumented proxy or secondary API. This explains why some replies contain verbatim Google snippets that never appear in Bing API responses.
Conclusion
The experiment reveals that ChatGPT relies on Google’s search results as a fallback for web pages that it cannot access or that are not yet indexed on Bing. This is evident from the fact that ChatGPT was initially unable to access the page directly, and it was only after the page began to appear in Google’s search results that it was able to respond to questions about the page. The findings suggest that standard SEO is still applicable for AI-powered search, including for ChatGPT Search, and that there is no need for specialized GEO or AEO to rank well in Google AI Overviews and AI Mode.