Now ETagging Right!

17Apr2007 [webprog]

Am now handling etags right! Django already takes care of the messy business - caching generated views and sending the appropriate etags, so if a view doesn't change, the browser shouldn't have to download it again.

macosxpeter:~ peter$ wget -S http://wintermute.com.au/
--19:44:57--  http://wintermute.com.au/
           => `index.html'
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Content-Length: 21112
  ETag: 3290c8e891866e8b1b0884e3a3820b7e
  Content-Type: text/html; charset=utf-8
Length: 21,112 (21K) [text/html]

macosxpeter:~ peter$ wget -S --header='If-None-Match: 3290c8e891866e8b1b0884e3a3820b7e' http://wintermute.com.au/
--19:45:53--  http://wintermute.com.au/
           => `index.html.1'
HTTP request sent, awaiting response... 
  HTTP/1.1 304 NOT MODIFIED
  Content-Type: text/html; charset=utf-8
19:45:54 ERROR 304: NOT MODIFIED.

But the etags weren't doing anything before, as I was generating random references to the banner image and to album photos on every load, so the view would change. I've now written a handy little 'randoms' app instead. Now the html for each page doesn't change - it just points to /random/banner.jpg and to, eg, /random/photo.jpg?1 through ?4. Those point to ordinary django views that deliver the contents of a random banner or photo. So the etag on those image references will change, but not on all the html pages, and that should make a real difference, as there's rarely much real change on most of the pages round this site...

The Django code, for anyone interested

Create a django app in the usual manner; this one is called randoms (I tried random until I realised it kept conflicting with the standard Python library random); you can lose the models.py file as this just pulls in other models...

Note that my setup differs from the usual advised by Django's docs, in that I have the folder where all my apps sit on the Python path. What this basically means is that I don't have to keep putting my_project. before all my app calls, as hey, that violates DRY (though, true, it would've let me call the app random, so I'm not going to argue this point). So if you're using the default setup, remember to drop a my_project. into a bunch of the imports.

And of course, after creating the randoms app, you'll want to include it in your INSTALLED_APPS, and include this line in your project's urls module:

(r'^random/', include('randoms.urls')),

urls.py:

from django.conf.urls.defaults import *
urlpatterns = patterns('randoms.views',
    (r'^banner.jpg$', 'banner'),
    (r'^album-(?P<album_id>\d+).jpg$', 'album'),
    (r'^photo.jpg$', 'photo'),
)

views.py:

from django.conf import settings
from django.http import HttpResponse
import os, random
from photos.models import Photo,Album

def banner(request):
    #return a random from the banners folder
    banner_path = settings.MEDIA_ROOT + 'banners'
    dir_list = os.listdir(banner_path)
    file_list = []
    for file in dir_list:
        index = file.rindex(".")
        if(file[index:] == '.jpg'):
            file_list.extend([file])
    rand = random.choice(file_list)
    return HttpResponse(open(banner_path + '/' + rand,'rb',0).read(),'image/jpeg')

def album(request,album_id):
    #return a random thumb from the given album
    photos = Album(album_id).photos.values('filename')
    photo = random.choice(photos)['filename']
    photo_path = settings.PHOTO_ROOT + 'thumbs/' + photo
    return HttpResponse(open(photo_path,'rb',0).read(), 'image/jpeg')

def photo(request):
    #return a random thumb
    photos = Photo.objects.values('filename')
    photo = random.choice(photos)['filename']
    photo_path = settings.PHOTO_ROOT + 'thumbs/' + photo
    return HttpResponse(open(photo_path,'rb',0).read(), 'image/jpeg')

I've got a lot in here; obviously you can pick and choose between the various views, but there are probably a few useful bits in here.

def banner:

This view, called with /random/banner.jpg, returns the contents of a random image stored in a path of your choosing; I just created a banners folder under the MEDIA_ROOT, though it could easily live outside the webroot. It loops through the folder, looking for .jpgs, picks one at random, reads it in (open(path).read()) and outputs it... Django's HttpResponse is usefully flexible, passing 'image/jpeg' to it means it'll serve the response up with the appropriate mimetype (so the browser has no way of knowing the image was generated).

def album:

This one hooks into my Album model. The idea is that it returns a random image from the given album, so I call this with /random/album-N.jpg, where N is an album's id. (I haven't put the necessary checks in to make sure that N exists, but there may be circumstances where you'd want to do that.) This view grabs a list of files associated with the given album (photos is a ManyToManyField), and returns one at random. The only weird thing here is settings.PHOTO_ROOT, which is a setting I use on this site for my photos app - I use it because I keep a number of photo files outside of the webroot, so the MEDIA_ROOT setting wasn't enough.

Update

Made a little tweak to make the code work on my Windows box; the open call needs a switch to tell it not to buffer ('rb',0); cf random on windows