Vcl on Varnish Cache

VSL cheatsheet

Wed, 29 May 2024 00:00:00 +0000

This page explains a few options common to VSL tools and provides ready-to-use VSL queries to simplify the first steps around the Varnish Shared Log.

Concepts

For performance reasons, Varnish doesn’t write log files. Instead, it outputs them in an in-memory circular buffer. Other processes can then read this buffer and decide what to do with the information.

This translates into a few points:

logging is extremely fast and doesn’t slow down the main process
this allows you to log a great deal of data for each transaction
another tool is needed to collect/process the logs

varnishlog

varnishlog is not the tool you want to collect logs to store. However, it is the perfect choice to obtain debugging data thanks to the vast amount of data it produces.

Mixed content and ERR_TOO_MANY_REDIRECTS errors in WordPress when using Varnish

Tue, 05 Dec 2023 00:00:00 +0000

If you’re running a WordPress site and use Varnish without TLS support, there are 2 typical issues you may encounter when it comes to offloading TLS: mixed content and the dreaded ERR_TOO_MANY_REDIRECTS error. Both issues are related to a lack of TLS awareness in the stack.

This tutorial will show you how to tackle both issues and how to create TLS awareness in a situation where both WordPress and Varnish might be lacking that awareness.

Bypassing the cache for specific URL patterns

Tue, 21 Sep 2021 00:00:00 +0000

Bypassing the cache in Varnish is done by calling return (pass) in the vcl_recv subroutine of your VCL file. This return statement will send you to the vcl_pass subroutine where a backend fetch will be triggered instead of performing a cache lookup.

This is something you would typically do when you can determine, based on the request information, that the response is not cacheable.

A typical example is using the URL to identify non-cacheable resources.

Performing HTTP redirections in Varnish

Mon, 13 Sep 2021 00:00:00 +0000

Varnish can perform HTTP redirections by returning a synthetic HTTP 301 response containing a Location header with the redirection endpoint.

Synthetic responses are triggered by a return (synth()) statement called from a client-side VCL subroutine. This also requires extending the vcl_synth logic and adding the necessary redirection logic.

HTTP to HTTPS redirections

One of the most common types of HTTP redirection is HTTP to HTTPS.

Here’s an example where an HTTP request is redirected to its HTTPS equivalent:

Using multiple backends in Varnish

Thu, 02 Sep 2021 00:00:00 +0000

The most common Varnish setups will feature a single backend, as illustrated in the VCL snippet below:

vcl 4.1;

backend default {
 .host = "127.0.0.1";
 .port = "8080";
}

However it is also possible to define multiple backends and to use VCL logic to decide which backend to use.

VCL example

The example below has 2 backends:

default that connects to backend.example.com on port 80
image that connects to image.example.com on port 80

The image backend in the example is only used when the URL ends with .png, .jpg or .svg.

Banning content from Varnish

Tue, 31 Aug 2021 00:00:00 +0000

Banning is a concept in Varnish that allows expression-based cache invalidation. This means that you can invalidate multiple objects from the cache without the need for individual purge calls.

A ban is created by adding a ban expression to the ban list. All objects in the cache will be evaluated against the expressions in the ban list before being served. If the object is banned Varnish will mark it as expired and fetch new content from the backend.

Purging content from Varnish

Mon, 30 Aug 2021 00:00:00 +0000

The most basic and easy-to-use cache invalidation mechanism in Varnish is purging.

The idea is that you can perform a return (purge) in vcl_recv, and Varnish will remove the object. This would free up space in the cache after an object lookup.

This means that the hash of the object is used to identify it, but return (purge) would remove it along with its variants. This isn’t out-of-the-box behavior; you won’t find it in the built-in VCL. You need to write some VCL code for it.

Configuring Varnish for Drupal

Mon, 09 Aug 2021 00:00:00 +0000

Drupal is an open-source content management framework that is popular for mid-market and enterprise solutions that require more complex content types and workflows.

This tutorial is a step-by-step guide on how to configure Varnish for Drupal.

This tutorial features Drupal 9 and Varnish 9.

1. Install and configure Varnish

If you are already running a Drupal CMS and you want to use Varnish to accelerate it, you’ll have to decide where to install Varnish:

Configuring Varnish for Magento

Mon, 09 Aug 2021 00:00:00 +0000

Magento is one of the most popular e-commerce platforms out there and has both a Community Edition and an Enterprise Edition.

Magento is very flexible and versatile and that comes at a cost: the performance of a Magento store with lots of products and lots of visitors is often quite poor without a decent caching layer.

Luckily Magento has built-in caching mechanisms and the full page cache system has native support for Varnish.

Configuring Varnish for WordPress

Mon, 09 Aug 2021 00:00:00 +0000

WordPress is by far the most popular content management system on the internet. It is no longer exclusively used for simple blogs, but serves as the foundation of many popular websites.

In order to maintain the desired performance under heavy load, a solid caching strategy is a must. And that’s where Varnish truly shines.

In this tutorial we’ll explain how to install and configure Varnish on your WordPress site.

1. Install and configure Varnish

If you are already running WordPress in production, and you’re looking to accelerate the website with Varnish, you’ll have to decide where to install Varnish:

Example VCL template

Mon, 09 Aug 2021 00:00:00 +0000

Varnish’s built-in VCL is very conservative and focuses on not caching stateful and personalized content. By following the HTTP standard for caching rules, Varnish is safe by default. Unfortunately, in many real-life situations where backend servers do not send good caching headers, this will result in a low hit rate.

In this tutorial we’ll present a collection of customizable VCL examples for Varnish. We’ll focus on the individual VCL template features and in the end we’ll bring it all together into a single VCL file.

Caching POST requests in Varnish

Fri, 17 May 2024 00:00:00 +0000

Introduction

When a client (web browser) wants a regular resource (e.g., an HTML document or an image) from a web server, it will make a GET request to the server. In a default installation, Varnish will cache such requests, and this will lessen the load on the conventional web server.

When a user logs in to a site, or provides some unique or private information, this will be done through a POST request. By default, Varnish will not cache POST requests, but pass them directly to the backend server, unmodified. This is typically a good idea, but sometimes it makes sense to also cache POST requests.

Logging cache hits and misses

Thu, 28 Mar 2024 00:00:00 +0000

One of the first things you’ll want to know about your Varnish setup is whether or not your content is cached. In this tutorial, we’ll provide a VCL snippet and explain how to leverage it via varnishlog and varnishncsa.

VCL

sub vcl_recv {
	unset req.http.x-cache;
}

sub vcl_hit {
	set req.http.x-cache = "hit";
	if (obj.ttl <= 0s && obj.grace > 0s) {
		set req.http.x-cache = "hit graced";
	}
}

sub vcl_miss {
	set req.http.x-cache = "miss";
}

sub vcl_pass {
	set req.http.x-cache = "pass";
}

sub vcl_pipe {
	set req.http.x-cache = "pipe uncacheable";
}

sub vcl_synth {
	set req.http.x-cache = "synth synth";
	# uncomment the following line to show the information in the response
	# set resp.http.x-cache = req.http.x-cache;
}

sub vcl_deliver {
	if (obj.uncacheable) {
		set req.http.x-cache = req.http.x-cache + " uncacheable" ;
	} else {
		set req.http.x-cache = req.http.x-cache + " cached" ;
	}
	# uncomment the following line to show the information in the response
	# set resp.http.x-cache = req.http.x-cache;
}

The x-cache header is where we store the information using two terms that cover slightly different aspects of the content.

Avoiding HTTP to HTTPS redirect loops in Varnish

Tue, 22 Mar 2022 00:00:00 +0000

When your force HTTP to HTTPS redirection in your web server or web application and cache the output, you might get stuck in a redirect loop. Your browser may present the following error message as a result:

Behind the scenes, your browser will receive a 301 Moved Permanently status code and your browser will follow the URL from the Location header:

HTTP/1.1 301 Moved Permanently
Location: https://example.com/
Content-Length: 226
Content-Type: text/html; charset=iso-8859-1
X-Varnish: 2
Age: 0
Via: 1.1 varnish

Despite using an https:// URI scheme, the origin server will continue to issue redirects and you’re in fact stuck in a redirect loop. This loop continues until your browser gives up, at which point the error message will appear.

Varnish Configuration Language (VCL)

Thu, 16 Dec 2021 00:00:00 +0000

The Varnish Configuration Language (VCL) is a domain-specific programming language used by Varnish to control request handling, routing, caching, and several other aspects.

At first glance, VCL looks a lot like a normal, top-down programming language, with subroutines, if-statements and function calls. However, it is impossible to execute VCL outside of Varnish. Instead, you write code that is run inside Varnish at specific stages of the request-handling process. This lets you define advanced logic that extends the default behavior of Varnish.

Load a synthetic output template from a file

Tue, 05 Oct 2021 00:00:00 +0000

In this tutorial we will override the VCL output template for vcl_synth and vcl_backend_error by loading it from a file.

These subroutines are used to compose the output when synthetic responses are returned. These are responses that do not originate from the origin server. When you trigger a synthetic response yourself, you end up in the vcl_synth subroutine. When the synthetic response is the result of a failed backend response, you end up in vcl_backend_error.

Varnish built-in VCL

Tue, 05 Oct 2021 00:00:00 +0000

The built-in VCL contains a set of rules that will be executed by default, even if they are not specified in your own VCL file. It is possible to bypass the built-in VCL by issuing a return statement in your VCL code.

The built-in VCL provides much of the safe-by-default behavior of Varnish, so be very careful if you decide to skip it.

The built-in VCL defines the behavior for various states of the Varnish Finite State Machine, each of which comes with its own subroutine. This tutorial will explain what the behavior in each of these subroutines means.

Using basic authentication in VCL backend probes

Fri, 01 Oct 2021 00:00:00 +0000

When your Varnish server proxies requests to a backend that requires authorized access via basic authentication, you need to ensure that the right Authorization header is sent for each probe request.

1. Initial situation

Your initial backend setup will have a backend that may contain a .probe {} definition.

This health-checking probe will send HTTP requests to your backend server at regular intervals. As long as a 200 status code is returned, the backend is considered healthy.

Cache invalidation

Wed, 20 May 2020 00:00:00 +0000

Introduction

A good caching strategy not only defines how the content should be cached, but most importantly, how it should be invalidated and evicted from cache. An object inserted in cache can be served to other clients until it expires, is evicted to make room for other objects, or is invalidated.

The TTL (Time to Live) of an object define how long an object can be cached. An object’s TTL is set when the content is generated (by the backend) or when it’s inserted (in Varnish). The TTL can be set via HTTP caching headers (i.e. Expires or Cache-Control) or via VCL. Either way, Varnish will respect the defined TTLs and evict the object when its Time to Live has expired, making room for fresher content to be inserted in cache.

Object lifetimes: TTL, Grace, Keep

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

If you landed on this page it is very likely that you are familiar with Varnish and you know it improves content delivery performance by storing a copy of your content in cache, and every request thereafter is fulfilled by cached content. Every copy of the content (aka object) stored in cache has a lifetime that defines how long an object can be considered fresh, or live, within the cache.