HTTP Conditional Get

• Download Demo.zip-5.59KB
• Download Source.zip-11.58KB

Introduction

When the programmer is implementing a download cache library for his/her application to download files from the HTTP 1.1 server, the programmer usually has to write his/her own implementation of determining whether the file is not modified on the server since the last download, hence no need to re-download the file and just read or load from the previously downloaded file on the local folder. HTTP 1.1 specification supports a special type of HTTP Get, called HTTP Conditional Get.

HTTP Conditional Get does not download the file if the file is not modified since the last download. HTTP Get method changes to a HTTP Conditional GET if the request message includes an If-Modified-Since, If-Unmodified-Since, If-Match, If-None-Match, or If-Range header fields. In this article, I am going to show you how to use the If-Modified-Since and If-None-Match header field in HTTP Get request to turn your HTTP GET into HTTP Conditional GET. I will be using .Net 2.0 HTTP classes to demonstrate this method.

For the first download, it is always a HTTP Get request because we do not have the If-Modified-Since date and If-None-Match message digest. So for the first downloads, we must store the ETag, which is the file's message digest, and the last modified date from the server, to be used in the subsequent HTTP Conditional Get requests.

Note: For the ETag, you have to store the ETag: you cannot calculate the ETag message digest yourself, because you do not know what message digest algorithm which the server is using to calculate the message digest.

Note: For last modified date, you have to store the original string as well, if you are not using .Net HTTP classes but some other HTTP libraries; If you store the same date in another string format, some servers may misinterpret it as a different date, thus a 'different' file which needs to be downloaded.

Let us look at the HTTP Get code example which stores the ETag and last modified date from the HttpWebResponse object returned by HttpWebRequest's GetResponse method. To keep the code snippets simple and easy to understand, I have omitted the proper exception handling code.

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Uri); request.Method = "GET"; request.Credentials = new NetworkCredential(userName, password); HttpWebResponse response = (HttpWebResponse)request.GetResponse(); // store the ETag and last modified date of the file to be used in the http conditional get string eTag = response.Headers[HttpResponseHeader.ETag]; string ifModifiedSince = response.Headers[HttpResponseHeader.LastModified]; // Download code //================== Stream strm = response.GetResponseStream(); // Input stream //Output stream FileStream fs = new FileStream(outputFile, FileMode.Create, FileAccess.Write, FileShare.None); const int ArrSize = 10000; Byte[] barr = new Byte[ArrSize]; while(true) { int Result = strm.Read(barr, 0, ArrSize); if (Result == -1 || Result == 0) break; fs.Write(barr, 0, Result); } fs.Flush(); fs.Close(); strm.Close(); response.Close(); // End of Download code //======================= 

Let us look at the HTTP Conditional Get code example which uses the ETag to set the If-None-Match header field and uses last modified date to set the If-Modified-Since header field before getting the HTTP response. HttpWebReponse class will throw a System.Net.WebException with the StatusCode set to NotModified if the file is not modified; we have to catch this exception and check the StatusCode.

try { HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Uri); request.Method = "GET"; // Set the IfNoneMatch with the ETag we have just stored request.Headers[HttpRequestHeader.IfNoneMatch] = eTag; // Set the IfModifiedSince with the last modified date we have just stored DateTime dt = DateTime.Parse(ifModifiedSince); request.IfModifiedSince = dt; request.Credentials = new NetworkCredential(userName, password); HttpWebResponse response = (HttpWebResponse)request.GetResponse(); // Download code //================== Stream strm = response.GetResponseStream(); // Input stream //Output stream FileStream fs = new FileStream(outputFile, FileMode.Create, FileAccess.Write, FileShare.None); const int ArrSize = 10000; Byte[] barr = new Byte[ArrSize]; while(true) { int Result = strm.Read(barr, 0, ArrSize); if (Result == -1 || Result == 0) break; fs.Write(barr, 0, Result); } fs.Flush(); fs.Close(); strm.Close(); response.Close(); // End of Download code //======================= } catch(System.Net.WebException ex) { if (ex.Response != null) { using (HttpWebResponse response = ex.Response as HttpWebResponse) { if (response.StatusCode == HttpStatusCode.NotModified) { MessageBox.Show("File has not been updated since your last request"); return; } else MessageBox.Show(string.Format("Unexpected status code returned: {0}", response.StatusCode)); } } } 

I have made a C# demo which you can download publicly available files from any HTTP 1.1 servers. The first download is always a download. The subsequent downloads would take place, if the file has changed since the first download.

Note: The HTTP Conditional Get could only take place if the server has provided the ETag and last modified date fields in the first download.

Screenshot of the demo application

First download

Subsequent download

License

This article, along with any associated source code and files, is licensed under The Intelliproject Open License (IPOL)

About the author