Google clarifies how Google's crawlers handle cache control headers
May 13, 2025 am 09:12 AM
Google has added a new section to its crawler and fetcher documentation for HTTP caching, which clarifies how Google’s crawlers handle cache control headers. With that, Gary Illyes from Google also wrote a blog post named HTTP caching asking site owners to allow Google to cache its pages.
What is new. The new documentation now says:
“Google’s crawling infrastructure supports heuristic HTTP caching as defined by the HTTP caching standard, specifically through the ETag response- and If-None-Match request header, and the Last-Modified response- and If-Modified-Since request header.”
Google does not support other HTTP caching directives and if both ETag and Last-Modified response header fields are present in the HTTP response, Google’s crawlers use the ETag value.
Caching decreased. Gary Illyes from Google said in the blog post that the “number of requests that can be returned from local caches has decreased.” He said, “10 years ago about 0.026% of the total fetches were cacheable, which is already not that impressive; today that number is 0.017%.”
“If you’re in the business of making your users happy and perhaps also want to potentially save a few bucks on your hosting bill, talk to your hosting or CMS provider, or your developers about how to enable HTTP caching for your site. If nothing else, your users will like you a bit more,” Gary Illyes added.
Why we care. Caching may help Google crawl your site more efficiently, which may make for a happier Googlebot. There is no mention of any SEO or ranking benefit to using caching, there is also no mention of crawl budget benefit.
The above is the detailed content of Google clarifies how Google's crawlers handle cache control headers. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
Are we becoming less curious about SEO?
Jul 07, 2025 am 09:12 AM
I noticed that a strong comment from Google’s VP of Search, Hyung-Jin Kim, at SMX Next in November 2022 has largely gone unnoticed by the SEO community up to now.He stated (my emphasis):“E-A-T is a template for how we rate an individual site. We do i
What we're seeing a week into the Google March 2024 core and spam updates
Jul 07, 2025 am 09:16 AM
We are now just about a week into the Google March 2024 core and spam updates, and boy, has it been busy. In that time, we have seen search ranking volatility, some related to the algorithmic updates and some related to Google issuing manual actions
Bing Deep Search is officially live for all users
Jul 05, 2025 am 09:32 AM
Bing Deep Search, an optional generative AI feature designed to assist users with complex questions that lack straightforward answers, is now fully available to all users. Microsoft has announced that the Deep Search function within Bing Search can n
Google starts testing AI overviews from SGE in main Google search interface
Jul 05, 2025 am 09:33 AM
Google is currently trialing AI overviews directly within the standard Google Search results, even for users who haven't signed up for the Google Search Generative Experience (SGE) Labs feature. According to a Google spokesperson speaking to Search E
Is ChatGPT the Google Search killer we've been expecting?
Jul 05, 2025 am 09:14 AM
AltaVista. Lycos. Yahoo. Once upon a time, these were the most popular search engines in the world. Then along came Google. It did Search better. Since around 2002, Google has been the search engine – and its dominance has only grown year after
Mikhail Parakhin, head of Microsoft Bing Search and Microsoft Advertising steps down
Jul 05, 2025 am 09:15 AM
Mikhail Parakhin is leaving his position as the head of Bing Search and Microsoft Advertising, potentially moving into a different role within the company. “Mikhail Parakhin, who leads the company’s Bing search engine and advertising divisions, will
4 keys to SEO and PPC collaboration in 2024
Jul 06, 2025 am 09:27 AM
Every year brings a ton of change in digital marketing. In each of my 10 years in the industry, I’ve noticed that the beginning of the year can mark a surge in calls for SEO and PPC to work together.The difference in 2024? There’s an elephant in the
8 steps to maximize success during global site migrations
Jul 04, 2025 am 09:13 AM
Migrating a website is a complex undertaking, but when it involves transitioning global sites across multiple markets, the challenges are exponentially greater. This article provides a comprehensive guide to maximizing success with global site mi
