Caching explained: browser, CDN, page and object cache

The idea

Building a web page can be a lot of work. A WordPress page, for example, runs code, asks the database dozens of questions, assembles the answers into HTML and sends it. If a thousand visitors ask for the same page this morning, doing that thousand times is a waste. A cache does it once, keeps the result, and hands the saved copy to everyone who asks until it goes out of date.

The saving shows up in every speed number that depends on the first wait. A cached page is answered in a fraction of the time, which shortens the time to first byte and gives everything after it, including LCP, a head start. Cached images, scripts and styles do not have to be downloaded again on the next visit, which is one more reason to follow the advice in image optimization about file size and format.

The five layers

1. The browser cache. Every browser keeps copies of the files it has downloaded: images, stylesheets, scripts, fonts. When the visitor returns, or goes to another page that uses the same logo and stylesheet, the browser reuses its copy instead of downloading it. It is private to that one visitor, it costs nothing to serve, and you control it with the Cache-Control header your server sends with each file.

2. The CDN cache. A content delivery network keeps copies of your files on servers around the world. A visitor in Madrid gets the copy from a server in Spain instead of from your server in Florida. Most CDNs cache images, scripts and stylesheets. Many can also cache the finished HTML of your pages, which is the biggest saving for a dynamic site, because the page is served without touching your server at all. Whether you need one depends on where your visitors are and how dynamic the site is; a site with visitors on several continents gains the most. The guide to what a CDN is helps you decide.

3. The page cache. Your own server can save the finished HTML of each page after building it once. On WordPress this is what a caching plugin or your host's cache does. Visitors who miss the CDN still get a saved copy, and the server skips the code and the database.

4. The object cache. Parts of the work are cached in memory: the results of database queries, calculated values, menus. A tool such as Redis or Memcached holds them so that the same question is not asked twice. It helps pages that cannot be cached whole, such as a logged-in dashboard or a shopping cart, because it speeds up the work that has to be done.

5. The database. The original source. It is the slowest layer and the one that every other layer exists to protect.

A request stops at the first layer that has a fresh copy. A returning visitor's request often never leaves the browser. A new visitor's request might be answered by the CDN. Only a cache miss at every layer reaches the database.

Fresh, stale and validated

A cached copy has a life. While it is fresh, the cache gives it out without asking anyone. When it becomes stale, the cache has two choices: fetch a new copy, or ask the server whether the old one is still good. That question is called revalidation. The server answers "yes, unchanged" with a very small response, and the cache keeps using its copy. This saves the download, though not the round trip.

You decide how long a copy is fresh, per kind of file, with the Cache-Control header on the response:

• max-age=31536000 means fresh for a year (the number is in seconds).
• no-cache means "you may keep it, but check with me before using it". It does not mean "do not cache".
• no-store means do not keep it at all.
• private means only the visitor's own browser may keep it, never a shared cache.
• public marks a response as storable by any cache, even one that came with an Authorization header. It is rarely needed otherwise.

Every header, with examples for Apache and Nginx, is in HTTP caching headers.

What to cache, and for how long

The rule of thumb for each kind of file:

The best files for caching never change. For files that do, the standard method is to put a version or a hash of the content in the file name, and change the name when the content changes. Then the old copy can stay in caches for a year without ever being served by mistake, because nobody asks for that name again. This is called cache busting. A version in a query string (style.css?v=12) also works, but some caches and CDNs treat addresses with and without a query string alike depending on their settings, so a change of file name is the safer choice.

HTML cannot use the same trick because its address must stay the same. That is the hard part of caching. A CDN or a service worker that you can clear by command solves it: the page is cached for a long time and cleared the moment you publish a change.

Where caching goes wrong

• No headers at all. A response without Cache-Control is not left uncached. Browsers store it and guess a lifetime, typically a tenth of the time since the file last changed, which is unpredictable. Set a policy on every response.
• Caching personal pages. A cart or account page that lands in a shared cache can show one visitor's data to another. Pages that depend on a cookie need private, and the CDN must be told to skip them.
• Stale content after an update. You publish a change and nobody sees it. Version file names, validate the HTML, and clear the CDN and page cache when you publish.
• A cache that never fills. Pages with a different query string on every visit (a tracking parameter, a session id in the address) are treated as different pages, so the cache is never hit. Configure the cache to ignore parameters that do not change the page.
• Too much cached, too long. A "year" on a file whose name never changes is a way to ship bugs that last a year.
• Several layers, one forgotten. After a change, clear every layer that holds the old copy: the page cache, the CDN and your own object cache.

How to see what is cached

• Chrome DevTools. In the Network panel, a file served from the browser's cache shows "(disk cache)" or "(memory cache)" in the size column. Click a request and read its response headers for Cache-Control, ETag and Age.
• A header check from the command line. curl -I https://your-site.com/ shows the headers. Many CDNs add a header that says whether the answer was a hit or a miss.
• The report. In Lighthouse 13, which PageSpeed Insights runs, the insight called Use efficient cache lifetimes lists files that are re-downloaded because their cache lifetime is short. It looks at static files such as images, scripts, styles and fonts, skips responses that deliberately turn caching off, and leaves alone any file whose lifetime is 30 days or more. A shorter lifetime is listed, with an estimate of the bytes that repeat visitors download again.

A sensible setup

1. Fingerprint your static files, and serve them with a one-year lifetime and immutable.
2. Put a page cache in front of your dynamic pages, on the server or in the CDN.
3. Put a CDN in front of the whole site, with HTML caching switched on where pages are not personal.
4. On sites with a database, add an object cache.
5. Clear what must be cleared on publish, and nothing else.
6. Test as a new visitor and as a returning one.

Check your own page

Run the free audit and look at the first wait for the page and at the cache lifetime insight. A slow first response for a page that is the same for everyone usually means there is no page cache, and a list of files to re-download points at missing cache headers on static files.

Sources

• HTTP caching on MDN: private and shared caches, freshness, validation, cache busting, and the patterns for static and main resources.
• Cache-Control on MDN: every directive.
• Optimize Time to First Byte on web.dev: CDNs, caching and the other ways to shorten the first wait.