Introduction to LLMs.txt
With the growing popularity of AI search, many tools and plugins are offering LLMs.txt to improve AI search visibility. However, the fact is that LLMs.txt is just a proposal, and no AI platform has signed on to use it. This raises the question of why companies are rushing to support a standard that no one actually uses.
The Confusion Around LLMs.txt
A recent Reddit discussion exemplifies this phenomenon. A user was concerned because an SEO tool flagged LLMs.txt as missing, implying that it was necessary. The user asked, "Why is SEMRush showing that the /llm.txt is a 404? Yes, I know I don’t have one for the website, but I’ve heard it’s useless and not needed. Is that true? If I need it, how do I build it?"
Google’s Stance on LLMs.txt
Google’s John Mueller addressed the confusion in the Reddit discussion. He confirmed that LLMs.txt is unnecessary and explained that it’s essential to catch misleading information early on. Mueller stated, "Good catch! Especially in SEO, it’s important to catch misleading & bad information early, before you invest time into doing something unnecessary. Question everything."
Why AI Platforms Don’t Use LLMs.txt
There are several reasons why AI platforms may choose not to use LLMs.txt. One of the main reasons is that it’s inherently untrustworthy. On-page content is relatively trustworthy because it’s the same for users as it is for AI bots. However, a separate file like LLMs.txt can be altered exclusively for AI, making it a potential target for spamming.
The Risk of Manipulation
Research has shown that manipulation of LLMs is possible using techniques like Preference Manipulation Attacks. This allows attackers to trick AI platforms into promoting their content over competitors. For example, a study demonstrated that an attacker can make a targeted camera 2.5× more likely to be recommended by an LLM. This highlights the risk of creating a separate file for AI chatbots, as it can be exploited by spammers.
What SEO Plugins Say About LLMs.txt
Some SEO plugins, like Squirrly WordPress SEO, have acknowledged that they provided the LLMs.txt feature only because users asked for it. They clarify that it has no influence on AI search visibility. On the other hand, plugins like Rank Math overstate the usefulness of LLMs.txt, suggesting that AI chatbots use the curated version of the content presented in the markdown files. Yoast SEO takes a more conservative approach, explaining the purpose of LLMs.txt without overstating its benefits.
The LLMs.txt Misinformation Loop
The conversation around LLMs.txt has become a self-reinforcing loop. Business owners and SEOs feel anxious about AI visibility and believe they must do something, viewing LLMs.txt as the solution. SEO tool providers are compelled to provide the LLMs.txt option, reinforcing the belief that it’s necessary. This perpetuates the cycle of misunderstanding, leading to the adoption of LLMs.txt as a proposed standard that no AI platform currently uses.
Conclusion
In conclusion, LLMs.txt is not a necessary file for improving AI search visibility. Despite its popularity among SEO tools and plugins, no AI platform has signed on to use it. The risks of manipulation and the lack of trustworthiness make it a potential target for spammers. It’s essential to be aware of the misinformation loop surrounding LLMs.txt and to focus on creating high-quality, trustworthy content that benefits both users and AI platforms. By doing so, we can avoid perpetuating the cycle of misunderstanding and instead, focus on creating a better online experience for everyone.