Google News Update: Clarification on Crawler Preferences

Google recently updated their Google News crawler documentation to correct an error that implied publisher crawler preferences addressed to Googlebot-News influenced the News tab in Google search. This change aims to provide clarity on how crawling preferences affect the Google News product.

Understanding the Google News Tab

The Google News tab is a category of search that is displayed near the top of the search results pages (SERPs). It filters search results to show content from news publishers that meet Google’s news content policies. To be eligible for the News tab, websites must adhere to these policies and be considered a legitimate news source.

What’s Changed in the Documentation

The update corrects an error in the user agent description, which previously suggested that publisher preferences for Googlebot-News affected what’s shown in the Google News tab. Google’s changelog explains that this was a mistake and that crawling preferences addressed to Googlebot-News do not influence the News tab in Google search.

Correction in the Documentation

The original description stated: “Crawling preferences addressed to the Googlebot-News user agent affect all surfaces of Google News (for example, the News tab in Google Search and the Google News app).” However, this was incorrect. The corrected version now reads: “Crawling preferences addressed to the Googlebot-News user agent affect the Google News product, including news.google.com and the Google News app.” This change removes any reference to the News tab in Google search, providing clarity on the role of crawling preferences.

Accessing the Updated Documentation

The updated Googlebot-News user agent documentation can be found on the Google Developers website. This resource provides valuable information for publishers and developers looking to understand how Google’s crawling preferences work and how they can optimize their content for the Google News product.

Conclusion

In conclusion, Google’s update to their Google News crawler documentation provides important clarification on the role of crawling preferences in the Google News product. By correcting the error regarding the News tab, Google aims to help publishers and developers better understand how to optimize their content for the Google News platform. This change promotes transparency and accuracy in the documentation, ensuring that users can make informed decisions about their crawling preferences and content strategy.