Dust and Sparkle: Digital Restoration Research

Mara Sedlins:

Although Cerebus stands out as a unique work of comic art, its preservation and digital restoration is not a unique problem. Many of the challenges involved in this undertaking are also faced by those using digital techniques to restore other types of artwork, like motion pictures or daguerreotypes. These other media also feature fragile original materials, large amounts of “data,” and characteristic types of “noise” whose identification requires the careful attention of an expert. I’ve spent a little time researching existing digital restoration approaches with an eye to finding techniques that might be adaptable to the work that Sean and I are doing with Cerebus.

The digital preservation of motion pictures is a controversial topic - some argue that true cinema can only exist in analogue form. But the digitization of film allows for more possibilities for restoration - such as the use of computer algorithms to track the motion of objects across frames, the automatic detection of scratches and dust (or “dirt and sparkle,” as it’s referred to in the industry), and the interpolation of missing pixels with appropriate values from neighboring areas.

Research in computer vision and image processing has also addressed the unique needs of daguerreotype restoration. Daguerreotypes were able to capture an incredible amount of detail and have great historical value - but they’re susceptible to deterioration over time and are delicate enough that attempts at manual restoration are likely to cause irreversible damage. This makes them ideal candidates for digital preservation and restoration. An especially compelling example is the 1848 Cincinnati Waterfront Panorama restoration project. You can read about it here (or for the academically minded, here), but briefly: this series of eight 6.5- by 8.5-inch images captures about 2 miles of the Cincinnati waterfront, and the resolution is so great that you need a microscope to see the finest visible details (wheel spokes, curtains, sign lettering).

The Cincinnati Public Library has made annotated, zoom-able digital images available to the public here.

But the images are marred by dust motes, corrosion, and scratch marks created by the initial polishing of the plates. At such a high resolution, these sources of noise obscure a lot of detail. Also, as you can see in the sky of the second image above, despite carefully controlled lighting the individually photographed sections are distorted at their edges (creating the mosaic effect in the composite image).

As you can imagine, restoring the entire panorama by hand at the pixel level would be a tremendous amount of work. Archivists at the George Eastman House International Museum of Photography (where the plates are housed) partnered with computer scientists at the University of Rochester to develop automated digital restoration techniques uniquely suited to the challenges presented by this series of daguerreotypes. For example, to correct for the tiling effect in the sky (or other “featureless areas” of the image), they created an average image (below) that was then used to correct for “brightness inaccuracies” created by the reflectivity of the plate.

To detect dust and other noise, the researchers used machine learning techniques. A support vector machine was provided with several filters, as well as a sample image that had been annotated by experts to test filter performance against (you can find more details in the academic paper linked above). I read enough to know that this type of approach is beyond my personal expertise - but it would be interesting to partner with a computer scientist and develop Cerebus-specific algorithms to detect and repair the various types of damage described in my taxonomy.

On the other hand, there are aspects of our restoration process that go beyond purely archival preservation goals - the images are specifically being readied for the printed page, and we’re making judgment calls to alter the original artwork in subtle ways to improve readability and honor the original intentions of the artists. I don’t see this process being fully automated anytime soon.