Tweaking the Varnish Cache

Varnish

From the docs, Varnish is a web application accelerator, or a caching HTTP reverse proxy. At its core, Varnish is designed to reduce response times by serving up content that it's previously stored without having to bother your application or web server. With its out-of-the-box configurations alone Varnish is a powerful ally in the war against latency, but sometimes it pays to be a bit more clever with your implementations.

Today, we'll be walking through the Varnish caching strategy for Happy Fun Town, a hypothetical backend component of a much larger distributed system. For simplicity's sake, let's say that the only endpoint exposed by the service is "/available-times", which takes many different parameters and returns a JSON response.

Given that Happy Fun Town is such an important piece of infrastructure, it has a lot of consumers and an SLA that it has to meet. With that in mind, a great way to improve response times is to throw your service behind a cache, and I've found Varnish to be really useful in that regard.

Problem 1

Happy Fun Town's "/available-times" endpoint happens to accept list-style parameters delimited by commas. So, an example request to this endpoint could look like:

{
		"availableTimes": {
				"Mon": ["1:30 PM", "2:30 PM"],
				"Tues": [],
				"Wed": ["1:30 PM", "2:30 PM"]
		}
}

Well, Varnish has no problem caching this content; subsequent requests to "/available-times?days=Mon,Tues,Wed" within your configured ttl will return the response above. But, what about "/available-times?days=Tues,Mon,Wed" or "/available-times?days=Wed,Mon,Tues"? We wouldn't want to unnecessarily waste cache memory by storing the exact same response for every permutation of our list parameters.

The most common solution is to use a popular VMOD (Varnish module) called boltsort. Using this VMOD lets us sort the query strings of incoming requests so we don't waste time with permutated duplicates. But first...

VCL

At this point in our exploration of Happy Fun Town and the caching problems that plague it, it's worth breaking off into a tangent about VCL: Varnish Configuration Language.

While Varnish itself is written in C, and you can write C inline if you're crazy enough, you'll typically find yourself writing custom logic in Varnish's small, C-like DSL. Using VCL, you can inject code into various states in the request-response lifecycle in a middlewarish fashion. This can be extremely handy, as we're about to see..

Unfortunately, boltsort won't work for us out of the box because of its implementation. If you poke through the code you'll notice that boltsort uses & as its determination of where to sort (i.e. by query-string key). This is a problem for us since we actually want to sort by the values of the days parameter. As a workaround, we can perform some magic using Varnish's built-in regex support along with the boltsort vmod. In the vcl_hash function, which defines our hash key, we can write:

import boltsort;

sub vcl_hash {
				hash_data(boltsort.sort(regsuball(regsub(req, "days=", ""), ",", "&")));
				return (hash);
}

We completely remove the days parameter and substitute every , with &; this yields "?Mon&Wed&Tues". It's not pretty but it'll serve our purposes. As a minor clarification, our call to hash_data is just computing our hash key, not actually mutating the request. After the appropriate substitutions, we go ahead and sort our data, consistently giving us a query string that looks like "?Mon&Tues&Wed".

As a bonus, since we're alphabetizing the query string, we also removed the problem of making sure these requests share a cache key:

curl "www.happy-fun-town/available-times?days=Mon,Tues&location=Chicago"
curl "www.happy-fun-town/available-times?location=Chicago&days=Mon,Tues"

Problem 2

As I mentioned above, Happy Fun Town is just another cog in the SOA machine, so we like to know who our service's consumers are. A common way to do this is to have clients tack on some extra information in a query string to identify themselves.

curl "www.happy-fun-town.com/available-times?days=Mon,Tues&myName=Marlon"

If you haven't already surmised from the context above, Varnish does a sort of key-value lookup based on URL. So, if Marlon keeps bugging us for our available times we'll keep serving him from the cache, but we'll end up with a cache miss for any other consumers even if they're checking the same days.

curl "www.happy-fun-town.com/available-times?days=Mon,Tues&myName=Josie"

It seems like what we'd want to do is somehow remove the myName parameter from the request for caching purposes, but still forward the original request to our backend.

So, how do we make sure Josie and Marlon get served from the same cache? We can use regexes again!

With the strategy we mentioned above, our hashing implementation would look something like this:


import boltsort;
sub vcl_hash {
				hash_data(
								regsuball(
												regsuball(req.url, "(?:(\?)|&)myName=[^&]*", "\1"),
												"(\?)&|\?\Z", "\1"
								)
				);
				return(hash);
}

What the first part of this regex attempts is safe removal of the myName parameter. If it's the first query parameter, then keep the question mark but remove the myName=Josie. Otherwise, completely strip the key-value query pair. With me so far?

The second part just does some additional sanitization. After removing myName=Josie from "?myName=Josie&days=Mon", we end up with "?&days=Mon", which we would want to turn into just "?days=Mon". Alternatively, if there's a dangling question mark because "myName=Josie" was the only parameter and we've removed it, then we go ahead and strip that off as well.

These regexes are really paying off!

Just Getting Started...

This was a really quick sampling of some of the capabilities of Varnish and how it can be tweaked to fit various use cases. One of the software's greatest strengths is its configurability. That being said, there's still a ton to talk about: probes, the Evil Backend Hack, Grace and Saint mode, cache misses for services that need real-time data, etc. I encourage you to take a deeper dive to get the most out of the technology; the Varnish reference book is a great place to start.

Shout-Outs

The team I worked on these configurations with (I can't fathom how long it would've taken me to figure this out on my own)
The folks that proofed this for me