Conditional GET

This is not a very wellknow technique, but for sites with many returning visitors it can have a very big impact on the performance.

How it works

Let's start by looking at an example scenario.

You are serving an RSS feed from your website. Even though the content in the RSS feed is updated only twice a week, RSS readers are requesting the feed once every hour to check for updates. What conditional GET does is to give us a method for telling the RSS readers that nothing is changed and we therefore won't transfer the feed again before an update occurs. All modern browsers and and RSS readers support the concept of conditional GET.

The scenario also applies to websites, blogs etc. that don't change between every visit.

There are two ways to utilize conditional GETs:

  1. The ETag HTTP header
  2. The Last-Modified HTTP header

The ETag HTTP header

From Wikipedia:

An ETag (entity tag) is an HTTP response header returned by an HTTP/1.1 compliant web server used to determine change in content at a given URL. When a new HTTP response contains the same ETag as an older HTTP response, the contents are considered to be the same without further downloading. The header is useful for intermediary devices that perform caching, as well as for client web browsers that cache results. One method of generating the ETag is based on the last modified time of the file and the size of the file, another is using a checksum.

If the ETag is generated incorrectly, it can lead to updated files not being redownloaded by the user agent, or files that are already in the cache being downloaded again.

When specifying an ETag header in the response, the browser will store that ETag in its internal cache. The next time it visits the same page it will send an HTTP header called If-None-Match to the web server. That header contains the ETag from its previous visit. By reading the If-None-Match request header we can then determine if the browser already have the latest version of the page. If it does, then we can clear the content by using Response.Clear() or Response.SurpressContent = true and add a status 304 to the response. Then we only send a header back to the browser telling it to use the page in its internal cache. We then safe potetially a lot of bandwidth and the page speeds up since it is loaded from the browser cache.

You can generate an ETag string from a date or the GetHashCode() method of any object. As longs as it is unique to the version of the page. There are no rule of the format of an ETag, as long as it is a string and it doesn't contain quote characters inside the ETag.

To utilize conditional GETs using the ETag header, you can use the following method:

public static void SetConditionalGetHeaders(string etag, HttpContext context)

{

  string ifNoneMatch = context.Request.Headers["If-None-Match"];

  etag = "\"" + etag + "\"";

 

  if (ifNoneMatch != null && ifNoneMatch.Contains(","))

  {

    ifNoneMatch = ifNoneMatch.Substring(0, ifNoneMatch.IndexOf(",", StringComparison.Ordinal));

  }

 

  context.Response.AppendHeader("Etag", etag);

  context.Response.Cache.VaryByHeaders["If-None-Match"] = true;

 

  if (etag == ifNoneMatch)

  {

    context.Response.ClearContent();

    context.Response.StatusCode = (int)HttpStatusCode.NotModified;

    context.Response.SuppressContent = true;

  }

}

The Last-Modified HTTP header

The Last-Modified header contains a date of the last time the page was updated. You can set it by calling Response.Cache.SetLastModified(DateTime). Just as the ETag header, this will make the browser save the date of the last modification to its internal cache. Next time the browser requests the same page it will send a request header called If-Modified-Since, which contains the date from the Last-Modified header from its last visit. By comparing the two dates we are able to determine whether or not to send a full response or nothing but an empty 304 status code.

public static void SetConditionalGetHeaders(DateTime lastModified, HttpContext context)

{

  HttpResponse response = context.Response;

  HttpRequest request = context.Request;

  lastModified = new DateTime(lastModified.Year, lastModified.Month, lastModified.Day, lastModified.Hour, lastModified.Minute, lastModified.Second);

 

  string incomingDate = request.Headers["If-Modified-Since"];

 

  response.Cache.SetLastModified(lastModified);

 

  DateTime testDate = DateTime.MinValue;

 

  if (DateTime.TryParse(incomingDate, out testDate) && testDate == lastModified)

  {

    response.ClearContent();

    response.StatusCode = (int)System.Net.HttpStatusCode.NotModified;

    response.SuppressContent = true;

  }

}

The WebOptimizer

The two methods above is included in the WebOptimizer so you can call them from anywhere in your ASP.NET application. It also comes with the ConditionalGetModule that automates this process for every page in your application. It looks at the response before sending it to the browser and then it creates an ETag by doing an MD5 hash of the entire response. It also looks for matching If-None-Match headers sent by the browser, makes the comparison and then decides whether or not to send the full response.

The MD5 hasing of the response stream takes less than 1 millisecond. Since it is so cheap on the CPU to run this module, it is recommeded for all ASP.NET applications - both MVC and WebForms.