Obscure Data Formats, Workflow, and Remote Synchronization

Chad Thatcher, Drakensberg

Date: Wednesday 19 September 2007
Time: 17:05 - 17:50
Location: Saal Maritim A

This is a case study of the migration of the RISM web application from PHP to Rails. RISM is a British Library effort to catalogue all music manuscripts from the 17th and 18th centuries (old site:

On top of sweeping changes to its user interface to take full advantage of Ajax, the underlying data structure was completely rethought to deal with the application's natural data storage: MARC. MARC was developed in the 60s for the digital storage of library catalogues and index cards. It has a fairly complex hierarchical structure and is still used by most libraries in Europe, North America, and Australia.

In the original PHP-based application the MARC records for each musical manuscript were mapped out to a fairly typical RDMS solution for hierarchies. Once the collection passed 45,000 manuscripts and the node table 2.5 million rows, however, this became unwieldy. Editing these MARC records was also tedious and cumbersome.

Drag and drop and auto-completion of Ajax proved to be extremely effective for editing the complex MARC records, and each MARC record could now be stored as a whole in a text field and mapped in real-time into a tree structure (using the composed_of property of ActiveRecord). For searching and navigation within the MARC data, Ferret proved a very powerful tool.

News and Coverage