Part 2: Output caching in Umbraco 17, and whether we needed it at all

This is the second of three posts about rebuilding the Nevitech website. Part 1 covered the twelve month story. This one is about page caching.

Let me start with the honest bit.

Umbraco 17 on .NET 10 is quick. The content cache in modern Umbraco is fast, Razor is fast, and a marketing site of this size with a few dozen pages was never going to struggle. Add Cloudflare in front of it for static assets and edge caching and the case for building a custom output cache policy is thin.

We did it anyway, because I wanted to know what was possible. A website with no client, no deadline and no risk is exactly the right place to find out how far something goes before you try it on somebody else's project. Some of what follows is useful. Some of it is complexity I would think twice about before repeating.

That is the honest version. Here is the implementation.

Output caching in ASP.NET Core stores the rendered response and serves it again without going near your controller, your views or Umbraco. It has been in the framework since .NET 7 and it is quite happy to sit inside an Umbraco pipeline, as long as you put it in the right place.

The policy is registered with a duration that comes from configuration, a couple of query strings to vary on, and a tag for invalidation:

services.AddOutputCache(options =>
{
    options.AddPolicy("Umbraco", policy => policy
        .AddPolicy<UmbracoOutputCachePolicy>()
        .Expire(cacheDuration)
        .SetVaryByQuery("page", "tags")
        .Tag("umbraco-pages"));
});

The middleware goes inside the Umbraco pipeline, after routing has been set up:

app.UseUmbraco()
    .WithMiddleware(u =>
    {
        u.UseBackOffice();
        u.UseWebsite();
 
        u.AppBuilder.UseNonceReplacement();
        u.AppBuilder.UseOutputCache();
    })

That ordering matters twice over. Output caching has to come after Umbraco has worked out what it is serving, and nonce replacement has to come before the output cache, which is the subject of Part 3.

The important design decision is that caching is opt in rather than opt out. It is applied with an attribute on the default render controller only:

public class DefaultRenderController : RenderController
{
    [OutputCache(PolicyName = "Umbraco")]
    public override IActionResult Index()
    {
        return base.Index();
    }
}

The error page controller and the contact page controller are separate classes without the attribute, so they can never be cached by accident. If you take one thing from this post, take that. Caching should be something you turn on for a page, not something you have to remember to turn off.

Most of the work in a cache policy is deciding when to leave well alone. Ours bypasses caching entirely for preview mode, detected by the UMB_PREVIEW cookie, and for any page containing a contact form block.

It also refuses to store anything that is not a 200, is not text/html, or has a Set-Cookie header on it, which is the sort of check that stops you caching somebody else's session cookie and handing it to the next visitor. That would be a very bad day.

The contact form exclusion is worth explaining, because it catches people out. Antiforgery tokens are per request. Cache the HTML and everybody gets the same stale token, which either fails validation or, worse, appears to work until it does not. You can work around it with a token refresh call, but for a website with one form on it, not caching those pages is the sane answer.

The cookie banner is the interesting one, and it is where I got it wrong first time round.

The website only loads analytics if consent has been given, so the accepted version of a page and the declined version are genuinely different HTML. Serve one to the other group and you have either broken your own analytics or, far worse, loaded tracking for somebody who explicitly declined it. That is not a performance bug, that is a compliance problem.

My first version dealt with it by refusing to cache anything at all until the banner had been answered. Safe, and fairly useless, because the visitor who has not answered the banner yet is every first time visitor. That is precisely the audience you want the fast page for.

The reason it would not cache them was subtler than I expected. The banner is a form, and the form helper emits an antiforgery token by default. Issuing a token sets a cookie, and the policy refuses to store any response carrying a Set-Cookie header. So every pre-consent page was quietly disqualifying itself.

The fix was to take the token off the banner form, and that is a decision to make deliberately rather than because it is convenient. The only thing that endpoint does is record a cookie preference. There is no privileged action behind it and no data access, and there is nothing for an attacker to gain by forging a request that sets somebody's own cookie preference. On a form that does something that matters, the token stays. Ours still does on the contact form.

Two gotchas if you go the same way. BeginUmbracoForm emits the token unless you tell it not to:

@using (Html.BeginUmbracoForm<CookieConsentController>("SubmitCookieConsent", FormMethod.Post, antiforgery: false))

And SurfaceController is decorated with [AutoValidateAntiforgeryToken], so dropping the token from the form is not enough on its own. The action has to opt out explicitly:

[HttpPost]
[IgnoreAntiforgeryToken]
public IActionResult SubmitCookieConsent(bool cookieConsent, Guid? contentKey = null)

With the token gone, consent status is simply part of the cache key:

context.CacheVaryByRules.VaryByValues["CookieConsent"] =
    cookieConsentService.CurrentStatus.ToString();

That gives three cached copies of each URL, one each for not answered, accepted and declined. Everybody gets a cached page including first time visitors, and nobody gets served somebody else's consent state.

That is the cost of caching in one line. Every genuine variation in your output has to become a vary rule, and every vary rule multiplies the number of copies you are storing. Get it wrong and you do not get an error, you get the wrong page served to the wrong person.

A cache nobody can clear is a liability. Editors publish something, do not see it, and start getting frustrated.

Every cached page is tagged, both individually and with a shared umbraco-pages tag, and Umbraco's own notifications clear it. Publish, unpublish and move to recycle bin all run the same helper:

await outputCacheStore.EvictByTagAsync("umbraco-pages", cancellationToken);
 
appCaches.ClearPartialViewCache();

It is a blunt instrument. Publishing any page clears the lot rather than just the affected page and its parents. For a website this size that is the right trade, because the cost of rebuilding a few dozen pages is nothing. On a website with thousands of pages I would use the per content tags properly and evict more precisely.

Note the second line. Output caching and partial view caching are separate systems and clearing one does not clear the other. Miss that and you will be very confused for about half a day.

Underneath the page cache there is a partial cache, and honestly this is the part I would keep if I could only keep one.

Umbraco has had CachedPartialAsync for years. Ours is a thin wrapper around it that reads the duration and an on off switch from configuration, and can key the cache by page and by named query strings:

@await Html.CachedPartialAsync("_MetaData", Model, cacheByPage: true, cacheByQueryString: ["tags"])

The main menu, the footer, the metadata block and the schema data all go through it. Those are the bits that are identical for thousands of requests and get rebuilt every time otherwise. It works on pages that cannot be output cached, it survives everything the page cache has to bypass, and the global switch means you can turn the whole thing off in development without touching code.

If you want 80% of the benefit for 10% of the complexity, cache your partials and stop there.

Being fair about the downsides:

Debugging gets harder. You change something, it does not appear, and you lose ten minutes before remembering why. We added an X-Cache-Status response header in development so you can see at a glance whether you are looking at a cached response.

Everything personalised becomes a special case. Forms, consent, anything that sets a cookie. Each one is either a bypass or a vary rule, and each one is a place to get it wrong. The antiforgery discovery above is a good example: the thing stopping the cache working was two layers away from the cache itself.

It interacts with your security headers. If you use CSP nonces, output caching will break them, and vice versa. That took real work to solve and it is the whole of Part 3.

Memory. The default store is in memory on the server. Fine for one instance, less fine if you scale out, at which point you want a distributed store and another set of decisions.

The gain was smaller than the effort. Umbraco was already fast, and Cloudflare was already handling the static assets. Output caching helps most when your pages are expensive to build, and ours are not.

Sometimes. If a website has expensive pages, heavy listing and filtering, external API calls during rendering, or a traffic profile with sharp spikes, output caching earns its keep quickly and I would put it in.

For a normal Umbraco marketing website on a current version, I would cache the partials, put Cloudflare in front and leave the page cache alone until something proves it is needed. Most performance problems on Umbraco websites are not solved by caching. They are images that were never resized, a bundle nobody trimmed and a database call inside a loop.

The thing I would take to every project is not the cache itself. It is the opt in attribute, the explicit list of situations where caching must not happen, and the invalidation being wired to Umbraco's own notifications so editors never have to think about it.

Part 3 is the awkward one: making Content Security Policy nonces work with a cache that is serving the same HTML to everybody.