O'Reilly European Open Source Convention - 17-20 October, 2005

Session
Caching Without Compromising Freshness: A Managed Cache for Apache Web Serving
Ross Heritage, BBC News Interactive

Track: Linux
Date: Tuesday, 18 October 2005
Time: 13:30 - 14:15
Location: Grand Ballroom
bbc.co.uk is one of the largest and most popular web sites in Europe, comprising several million web pages, and viewed by a daily audience of around 3.5 million unique users. With both our audience and content size growing at significant rates, our serving architecture (essentially, large web servers each with the full content) was reaching the limits of its scalability, with disk i/o performance starting to degrade. Our challenge was to keep the full volume of data constantly available, whilst improving the web serving performance and scalability. Additionally, editorial constraints of always serving the very latest version of our content ruled out the use of a traditional 2-tier web-caching model, relying on imprecise "Expires" headers. Our solution was to implement a rich cache that, instead of relying on file stats and heuristics, provides a rich control layer, exposed as a REST API, for the retrieval, deletion, and publishing of content. This solution builds upon Apache 2's ability to chain filters, and works in conjunction with mod_proxy and mod_rewrite. In the session, Ross Heritage will cover the design and implementation of this module, and describe its use in the day-to-day serving of the bbc.co.uk site. Additionally, he will discuss its application in other areas, specifically as a component in his RSS site search.

Download presentation file