As many of you are probably aware, OCLC and the University of Michigan announced last January that OCLC was taking over the OAIster aggregation of metadata harvested from OAI-compliant repositories. The University of Michigan was no longer able to support it, and was looking for assistance in sustaining this valuable community resource. As Kat Hagedorn remarked in regards to our agreement, “Hosting anything of this size quickly got out of hand for UM Libraries, and it took us a long time to realize it. Besides, greater access for more folks? Sounds win-win to me, as long as it’s continuously freely available.” [reported by Dorothea Salo]
I have heard lots of questions since we started contacting contributors with the most recent phase of the transfer plan, so the purpose of this post is to bring everyone up to date on why we are doing this, where things are, and what we hope to accomplish in the future.
- OCLC wanted to do whatever we could to ensure sustainability of this aggregation when the University of Michigan realized they needed assistance. We believe, as do many others, that OAIster provides a useful aggregation of millions of records representing millions of open access papers, journal articles, and other items with useful academic content. As a global non-profit library cooperative, we felt like we were the logical organization to provide support and maintenance of this service on behalf of the community at large. OCLC is committed to building on the success of OAIster by identifying open archive collections of interest to libraries and researchers, and ensuring that open archive collections will be freely discoverable and accessible to information seekers worldwide.
- We continue to collaborate with the University of Michigan during this transition period. The University of Michigan has been tremendously generous in their time and expertise as we take over this complicated and difficult process.
- Starting in October, the records will be freely discoverable along with all the other content in WorldCat.org. However, it will not be possible to limit a search to OAIster records alone.
- In FirstSearch, OAIster records can either be searched along with other FirstSearch databases, or selected to search alone. OAIster records have been searchable in FirstSearch since January 2009.
- Contributors of OAIster records can receive free access to the OAIster aggregation in FirstSearch by request. Contributors were recently contacted to offer them such access and many have already responded that they would like to have such access.
- Only data providers that request that we not harvest their records will be removed from the aggregation. We feel strongly that one of the main benefits of OAIster has been the aggregation of records from the vast majority of repositories worldwide. Therefore, unless a repository denies us permission to harvest their records, we will seek to include them.
- No money was exchanged in this transfer and OCLC is not making any money on the OAIster aggregation. OAIster records were added to FirstSearch at no extra charge to FirstSearch subscribers, and of course there is no charge for searching WorldCat.org, where they are also exposed. Rather than boosting revenue, in fact, OCLC is committed to making an investment in the kind of large-scale harvesting operation that OAIster represents.
- Harvesting is hard. As anyone who has done this work will tell you, harvesting records using the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is far from simple. There are all kinds of difficulties, not the least of which is the uneven support of the protocol by the wide variety of repository platforms. Community awareness of these problems led to the formation of an NSDL and DLF-sponsored working group that produced a web site devoted to “Best Practices for OAI Data Provider Implementations and Shareable Metadata”. Since this is a difficult process, we may not get everything right from the beginning, but with help from the University of Michigan during this transition we’re hopeful that we can not only reach, but eventually exceed, what has gone before.
- We are exploring options for machine access. Z39.50 access to OAIster is available to FirstSearch subscribers now, and we are considering whether additional options should be supported. The University of Michigan did not offer an OAI-PMH or Web Services interface, although they did offer an rsync option. Learning the needs of the community will help inform what we do in this area.
- We are seeking to provide long-term scalability for this service and we ask for the cooperation of data providers. Something that is likely not widely known is that the University of Michigan would perform specialized processing of the retrieved records because of standards noncompliance by some data providers. In order to sustain this service over the long haul, we will need to work with data providers to reduce the number of exceptions to standard procedures.
- We are forming an advisory board to provide us with essential advice. We know that this is an ongoing service that will require further development and support, and so we seek the advice of those knowledgeable and experienced within the community to make sure we get it as right as we can on behalf of our member institutions and the broader community of users.
We believe that we are uniquely positioned to maintain a production aggregation service of this scale in the service of information seekers worldwide. We welcome the advice and assistance of the OAI community in making this service as useful as possible for those seeking access to valuable academic content.
Roy Tennant works on projects related to improving the technological infrastructure of libraries, museums, and archives.