I've been fooling around with caching http responses lately in Go and I'm trying to figure out the most efficient way possible to generate proper ETags.
Right now I have the following data available:
After some thinking I came to the conclusion that if I combine the name of the template and the dynamic data being produced this should in theory create a legit unique ETag with the least amount of overhead but I don't know how nasty this will get if I start wanting to return like 30kb of html worth of database results.
I'm using a crc32 routine from Go's stdlib to generate the ETag from the data I pass into it.
Is there a better way to generate ETags, or even cache dynamic data? I can't just monitor the last-modified time of a file because the the data can change without the file changing.
In general, you want to use something that is cheap to calculate as an ETag. The reason for this is that if the client sends a conditional request (e.g. via the If-None-Match HTTP request header), you can decide whether it is appropriate to send a 304 Not Modified response without having to do all the processing for the page.
For example, if you have some kind of revision identifier for the content of a page, then that might make a good ETag.
If you will need to do all the work necessary to render the page just to generate an ETag, then you may as well just use a hash of the rendered page content, or no ETag at all.
If you're looking for an efficient ETag mechanism and don't need something cryptographically strong, I would suggest using CRC-32. To help prevent collisions, you could combine a few things like the template name, length of the data, and the crc:
func etag(name string, data []byte) string {
crc := crc32.ChecksumIEEE(data)
return fmt.Sprintf(`W/"%s-%d-%08X"`, name, len(data), crc)
}
This will produce an etag like W/"tpl-17-3074C885".