Innovative solutions for dealing with born-digital content in obsolete forms – Part 3

0 trioThis is the third and final post about an SAA lightning talk session. Part 1 began with descriptions of the array of media an archives might confront and an effort to test how much can be done in house. In Part 2, we heard from archivists who had dealt with particularly challenging formats. Part 3 includes the service provider perspective, reaching out to the retrocomputing community, and concludes with some words about agreements between repositories and service providers.

Matthew McKinley, Digital Project Specialist at University of California, Irvine Libraries, gave an archives-to-archives service provider perspective. UCLA Special Collections has 15 5 ¼” floppy disks from the Southern California Women for Understanding collection. The collection inventory says the disks contain mailing lists and research data. He imagines the contents might be interesting research material, but he hasn’t been able to begin the project, because the legal departments and other administrators from both institutions had to sign off on the service agreement.
13 agreement
After finally getting the agreement through the UCI process, UCLA is on its third pass of the document.

His advice is to start with a vendor agreement and workflow for converting other media, such as audiovisual media. Learn from colleagues what roadblocks they ran into. Remember, this is an “innovative service” that administration may not understand or agree with, but you can use that to your advantage: make it known that data could be dying and time is of the essence.

The knowledge level of those involved affects the level of collaboration, oversight, and communication needed. In this situation, both project managers were familiar with archival terms and concepts. If your service provider has more technical experience, but no preservation or library & archive focus, you’ll need to make it very clear exactly what you need in both agreement & ongoing communication: metadata requirements, file format specifications, privacy and security concerns… Explain it and be sure they understand it. It could save you a lot of time and headaches later.

Matthew offered another insight that will make a transfer project go more smoothly: learn as much as you can about when and how the media was created. This is important for provenance metadata, but even more important for accessing the media and converting the content. What you get will depend on the creator’s knowledge or interest level — some may give you detailed version numbers and others may say “I used whatever came on my Mac in 1985.” But any information helps, whether about the donor’s hardware and software or about their computing behavior.

Margo Padilla, Strategic Programs Manager at the Metropolitan New York Library Council (METRO), talked about a born-digital migration service pilot project being coordinated by METRO and the Center for Jewish History, with support from the Delmas Foundation. She harkened back to the 2012 OCLC report, “Swatting the Long Tail of Digital Media: A Call for Collaboration,” which proposed a community-based approach for transferring content off of legacy media to more stable media.
14 swatting
The report suggested that a local institution become a hub, developing expertise and acquiring and maintaining necessary equipment to provide transfer services for institutions without the staff or resources to undertake this sort of work themselves. METRO is coordinating a working group to test this approach. The group is focused on the logistical issues and practical outcomes involved with this process, in order to test and refine real-world workflows, contractual agreements, and deliverables, and to document it for use by others. Partners in the working group include: The American Jewish Historical Society, The Guggenheim Museum Archives, The Leo Baeck Institute, The New-York Historical Society, and Queens Library.

Each organization will explore their own internal processes and requirements for inventorying legacy media, appraising and selecting them for transfer, and determining how they will be reintegrated into collections. So far, each participant has conducted a survey of their materials on digital storage media and a small sample of test materials (floppies, Zip disks, and optical media) have been delivered to METRO.

As a non-profit consortium currently providing a range of other digital services, METRO is well situated to offer digital migration services on a free or cost-recovery basis to local repositories and is focusing on scoping and developing their service model. They will track cost and labor estimates for developing a collaborative forensics, migration, and media archaeology lab and providing transfer-related services such as metadata, content analysis, normalization, and redundant storage. They will also identify and refine other potential requirements such as insurance coverage, secure storage areas, delivery methods, and protocols for dealing with confidential data.

The group has developed a draft service agreement, taking into consideration questions of potential deliverables, security, turnaround times, and other standard agreement language. METRO is in the process of building a dedicated workstation and is about to begin transferring content, initially providing disk images with file-system analytics and a metadata export. As part of an iterative process to develop a tiered service model, each organization will analyze this initial information and let METRO know what sort of additional forensic processing or analysis they would like to receive. Discussions about expectations and scope of work, contract agreements, workflows, and deliverables are ongoing.

Stephen Torrence, Vice President and Co-Founder of the Museum of Computer Culture, gave his perspective as a data migration service provider. MCC, part of a network of collectors, focuses on hardware restoration and stepped forward as a service provider to assist archives with obsolete media.
15 mcc
He did a successful proof-of-concept project with The Ohio State University Libraries, working with three low-risk diskettes. This helped both parties to work through the process and establish a reasonable base price per disk (for disk image, files, and metadata) and a fee for file format conversion.

He’s also learned from less successful projects. He tried to help out the Texas Department of State Health Services when they needed to recover a medical image from a magneto-optical disk with a proprietary operating system. After much time and many attempts, he was finally able to read the disk, but couldn’t decode the bitstream. From this he learned to establish a fallback base cost to recover some of his investment. When he worked with PSU on the AMSTRAD disks, he found that getting non-US hardware and software was cost-prohibitive. Both the service provider and the client need to acknowledge practical limitations.

Stephen then gave advice about “how to be an awesome client:” Know the origin environment before beginning – it will keep costs down and allow more accurate time estimates. Accept exceptions – be prepared to iterate based on feedback and redefine success, if necessary. Share the impact – involving your provider in the import of the context or content can give them a better sense of purpose.

Then we heard from an archivist who reached out to another community.

Dorothy Waugh, Digital Archives Project Archivist at Emory University, talked about the problem of knowing where to turn for help in working with obsolete physical media.

The needs of the Digital Archives unit were often outside the scope of Emory’s desktop and IT support and the complex technical language frequently found in online resources drove her to pursue options that offered the chance to ask questions in real time and get hands-on practice. She embarked on research into local retrocomputing groups and has since been participating in meetings of the Atlanta Historical Computing Society, or AHCS.

Dorothy discussed the benefits of the hands-on approach at these meetings, one immediate benefit being a broader understanding of the development of personal computers. AHCS is a great resource for locating hard-to-find hardware—she’s been lucky enough to receive a number of floppy disk and Zip disk drives from its members—and for troubleshooting difficulties with obsolete or specialized hardware or software: for instance, Dorothy has worked with members to solve some minor issues affecting Emory’s KryoFlux, a tool that aids in the imaging of aging floppy disks. The group also offered problem-solving assistance on accessing content on some proprietary floppy disks formatted for a mid-1980s VideoWriter word processor.
16 computer
Throughout, Dorothy has taken full advantage of the opportunity to ask lots of questions, which have consistently been met with patience and enthusiasm. She stressed, though, some key differences between the retrocomputing community and the digital archives profession, not least the emphasis that archives place upon content as opposed to the retrocomputing community’s emphasis upon computers as cultural artifacts. As a result of these differences, Dorothy has learned the importance of being quite explicit about her objectives as a digital archivist, while remaining engaged with the retrocomputing community’s commitment to the preservation of authentic computing environments.

A remaining concern for Dorothy is how her participation with AHCS can evolve into more of a reciprocal partnership and she continues to seek ways by which she can achieve this. Her hope is that, in addition to serving as an opportunity for education and advocacy, these attempts will enable AHCS to identify additional opportunities for two-way collaboration between the two communities.

After the ten talks, I announced a new OCLC Research report, “Agreement Elements for Outsourcing Transfer of Born Digital Content,” by Ricky Erway, Ben Goldman, and Matthew McKinley. This document suggests elements that should be considered when constructing an agreement (or memorandum of understanding) for outsourcing the transfer of born-digital content from a physical medium while encouraging adherence to both archival principles and technical requirements. By being explicit about the expectations, processes, and deliverables we can hope to develop an effective community-based approach for transferring content off of legacy media and into a form that can be manage, preserved, and used by researchers.

We hope these vignettes inspire others facing challenges with archival content in obscure formats to find innovative options for dealing with obsolete media, so you can get on with the really important work of processing, preserving, and providing access to those unique collections.