Introduction to ChatGPT’s Recommendation System
An investigation by SEO professional James Brockbank reveals a concerning issue with ChatGPT’s recommendation system. It appears that the AI model may be suggesting businesses based on content from hacked websites and expired domains. This discovery was made through personal testing and observations, highlighting a potential flaw in how ChatGPT generates its responses.
How Manipulated Content Appears in ChatGPT Responses
Brockbank identified two main tactics that seem to influence ChatGPT’s business recommendations. These include the use of hacked websites and expired domains to publish manipulated content. In several cases, ChatGPT surfaced gambling recommendations that traced back to legitimate websites that had been compromised. For example, a California-based domestic violence attorney’s site was found hosting a listicle about online slots, and a United Nations youth coalition website was seemingly hijacked to host gambling-related content.
Hacked Websites
Hacked websites are being used to host irrelevant or deceptive content, which can then appear in ChatGPT-generated business recommendations. This is a significant concern, as it allows malicious actors to manipulate the system and promote their content. Brockbank found several examples of legitimate websites being compromised and used to host gambling-related content, including a U.S. summer camp site.
Expired Domains
The second tactic involves acquiring expired domains with strong backlink profiles and rebuilding them to promote unrelated content. In one case, Brockbank discovered a site with over 9,000 referring domains from sources like BBC, CNN, and Bloomberg. The domain, once owned by a UK arts charity, had been repurposed to promote gambling. This highlights the potential for malicious actors to exploit expired domains and use them to manipulate ChatGPT’s recommendation system.
Why This Content is Surfacing
Brockbank suggests that ChatGPT favors domains with perceived authority and recent publication dates. Additionally, the AI model may not sufficiently evaluate whether the content aligns with the original site’s purpose. This can lead to manipulated content appearing in ChatGPT-generated business recommendations. As Brockbank observed, "ChatGPT prefers recent sources, and the fact that these listicles aren’t topically relevant to what the domain is (or should be) about doesn’t seem to matter."
Implications and Concerns
The discovery of manipulated content in ChatGPT’s recommendations raises significant concerns about the credibility of the platform. Brockbank acknowledges that being featured in authentic "best of" listicles or media placements can help businesses gain visibility in AI-generated results. However, leveraging hacked or expired domains to manipulate source credibility crosses an ethical line. As Brockbank writes, "Injecting your brand or content into a hacked site or rebuilding an expired domain solely to fool a language model into citing it? That’s manipulation, and it undermines the credibility of the platform."
Conclusion
While Brockbank’s findings are based on individual testing rather than a formal study, they highlight a real concern about the potential for manipulated content to appear in ChatGPT’s recommendations. The investigation emphasizes the need for companies to improve how these systems detect and filter deceptive content. Until that happens, both users and businesses should approach AI-generated recommendations with a dose of skepticism. As Brockbank concluded, "We’re not yet at the stage where we can trust ChatGPT recommendations without considering where it’s sourced these from." Ultimately, building legitimate authority through trustworthy content and earned media will be crucial in maintaining the credibility of AI-generated recommendations.