Google has introduced two new web crawlers to its Googlebot family:

• GoogleOther-Image
• GoogleOther-Video

According to Google, these crawlers are specialized versions of GoogleOther designed specifically for retrieving image and video content. The GoogleOther crawler was first introduced in April 2023 for internal use by Google teams to access public web content, helping reduce the load on the primary Googlebot crawlers.

GoogleOther-Image
As per the documentation, GoogleOther-Image is a version of GoogleOther optimized for fetching publicly available image URLs. It will identify itself using the user agent tokens GoogleOther-Image and GoogleOther. The full user agent string will be: GoogleOther-Image/1.0.

GoogleOther-Video
Similarly, GoogleOther-Video is the version of GoogleOther tailored for fetching publicly accessible video URLs. It will use the user agent tokens GoogleOther-Video and GoogleOther, with the full user agent string being: GoogleOther-Video/1.0.

Purpose of These New Crawlers
Google stated that these new crawlers were launched to better support the retrieval of binary data for research and development purposes.

More About Google Crawlers
The different types of Googlebot crawlers currently include:

• Googlebot: This is the main crawler used for Google’s search products. Google emphasizes that this crawler always respects robots.txt rules.
• Special-case crawlers: These are crawlers designed for specific tasks (like AdsBot), and may or may not respect robots.txt directives.
• User-triggered fetchers: Tools or product features where a user action initiates a crawl request. For example, Google Site Verifier acts based on user requests, or certain Google Search Console tools can trigger Google to fetch a page after a user interaction.

Google also provides IP address ranges and reverse DNS masks for each crawler type:

• Googlebot: googlebot.json (crawl-**–**–**–**.googlebot.com or geo-crawl-**–**–**–**.geo.googlebot.com)
• Special-case crawlers: special-crawlers.json (rate-limited-proxy-**–**–**–**.google.com)
• User-triggered fetchers: user-triggered-fetchers.json (**–**–**–**.gae.googleusercontent.com)

Why This Matters
Many website owners monitor bot activity and crawling behavior through log files. If you notice these new GoogleOther crawlers in your logs, there's no need for concern—they are legitimate Google crawlers.

The above is the detailed content of Google adds two new Googlebot crawlers: GoogleOther-Image?and?GoogleOther-Video. For more information, please follow other related articles on the PHP Chinese website!