hangingtogether.org

Wikipedia gives the story of the running joke among Germans – that Bielefeld does not in fact exist. I began to wonder it if was more than just a joke after arriving on Monday by plane at the nearby international airport (Hannover), from which I then required two hours of train travel and an icy cold one-hour stop-over at Hannover main station before being able to shake off my doubts.

It does exist, and very pretty parts of it are too. The 9th International Bielefeld Conference is taking place in an impressively large convention centre near the city centre. The conference kicked off yesterday, with a good first half-day programme. One theme which came through interestingly in two of the presentations was that of virtual research environments. Juan Garcés from the British Library spoke about the Codex Sinaiticus, whose website states:

Codex Sinaiticus is one of the most important books in the world. Handwritten well over 1600 years ago, the manuscript contains the Christian Bible in Greek, including the oldest complete copy of the New Testament. Its heavily corrected text is of outstanding importance for the history of the Bible and the manuscript – the oldest substantial book to survive Antiquity – is of supreme importance for the history of the book … The Codex Sinaiticus Project is an international collaboration to reunite the entire manuscript in digital form and make it accessible to a global audience for the first time. Drawing on the expertise of leading scholars, conservators and curators, the Project gives everyone the opportunity to connect directly with this famous manuscript.

The Project involves a large international collaboration, with the four principal partners being the British Library, Leipzig University Library, St Catherine’s Monastery, Sinai and The National Library of Russia in St Petersburg.

It wants to create a scholarly user community around the Codex Sinaiticus. One of its tasks will be to develop an expertonomy (term coined by Garcés). He presented the Project as an example of the development of a special collection over time, portrayed in a community dimension as well as simply a resource dimension. Such items or collections have conventionally begun their digital life by being described and preserved by librarians and consulted in their original formats by a few expert readers. Via digitisation they can be transformed into a thriving communal resource co-developed by librarians and a large number of expert readers in the form of a virtual research environment. Juan Garcés built up this portrait eloquently and convincingly, in an animated slide which I tried to capture on my iPhone.

Will this community thrive? Ronald Milne of the British Library told me he was amazed at how web-active the papyrologist community is. Incidentally, Juan Garcés presented this work excitingly within the context of a recent decision by the British Library to mass-digitise its entire collection of pre-1600 manuscripts.

From papyrologists to bioethicists. The afternoon keynote was given by Wendy Pradt Lougee, University Librarian at the University of Minnesota (which, she announced, has just received the Association of College and Research Libraries (ACRL) 2009 Excellence in Academic Libraries Award). She discussed Minnesota’s Mellon-funded 2006 Multi-Dimensional Framework study , which I mentioned in an earlier post, and which we are using as a reference point in our RIM work. One of its blossomings has been the development of the myLibrary service within the myU portal. Customisation for service delivery has been developed by means of affinity strings which characterise granular groupings of faculty and students – described by Lorcan in a recent post – providing a high degree of customisation potential (there are 9,500 of them!).

Another important outcome is the Mellon-funded EthicShare virtual research environment for the global ethics community, with an initial emphasis on bioethics. Given a strong focus, virtual research environments, or communities, can be widely adopted. Any such undertaking must be given some time to see how much gravitational pull it exerts. Communities built around particular resources like the Codex Sinaiticus may have an immediate advantage in the gravitational attraction of the resource, and a toolset to accompany it. For EthicShare, Wendy intends that community building and community management will be a new role for subject librarians. With Minnesota’s librarians having access to a wealth of researcher behavioural data, and authoritative affinity strings, they ought to stand a good chance with their own bioethicists at least.

Reputation and trust are closely related and hard won. Two snippets dealing with the evolving landscape of reputation capital in universities caught my eye in this week’s Times Higher. The first relates to the proposed European Reference Index for the Humanities, funded by the European Science Foundation, which had announced it would grade journals into categories A (’high-ranking international publications’), B (’standard international publications’) and C (’publications of local/regional significance’). Rather as has happened in Australia whose league table of journals I mentioned in a previous post, there has been opposition to this idea - chiefly from academic editors of journals. So many of them have now threatened to boycott the index that the steering committee has been forced to drop the idea of the classification. It is doing so reluctantly, claiming that the classification was never intended to denote hierarchy. This might be indicative of a certain naïveté, or it may reveal, ironically, just how deep the concern about reputational damage potentially caused by rankings now runs, particularly in the UK where bibliometric measures of various kinds are being considered by the Higher Education Funding Council for England (HEFCE) for the new Research Excellence Framework (REF). Academics worry about any new measure which might be tossed into the bibliometric mix - and it is rather difficult to see how publication in a journal deemed of important local/regional significance but explicitly not of high-ranking international significance is a category judgement rather than a value judgement.

The second reveals that Evidence, a UK data analysis consultancy which has been working with HEFCE as it designs the REF, has been acquired by Thomson Reuters with whom it previously had a ’strategic alliance’. Since HEFCE was making use of Evidence to test the value of Thomson Reuters’ Web of Knowledge data relative to Elsevier’s SCOPUS data for the purposes of bibliometric analysis, it seems rather like a clear case of gamekeeper turned poacher. Evidence reject this, but HEFCE have confirmed that Evidence will now be confined to the use of Web of Science data, and

we will conduct an in-house analysis to compare the databases … taking independent external advice as we do this.

This sounds rather like an unexpected and probably unwelcome cost increase, but it is probably wise of HEFCE to do so. Employ a company to provide an impartial report upon its own products? That could tarnish its own reputation. As the landscape continues to shift, there are a few authoritative bodies whose products or services are coming to be trusted - albeit very cautiously in some cases. The Times Higher list of the world’s top 200 universities is one. The Shanghai Jiao Tong University index is another. University funding agencies cannot risk losing the trust of the academic community whose funding they administer. But ERIH’s task of earning the trust of European humanities researchers must now be a difficult one.

The excitement I remember when I was young as we awaited the announcement of the Christmas singles chart has been easily surpassed this year on university campuses. The RAE results are published today, and the general mood in UK HE seems to be one of celebration. David Eastwood, Chief Executive of the Higher Education Council for England, writing in the Times Higher says:

Amid the excitement of celebrating local achievements, we should not lose sight of an important story of national success: we enjoy the benefits of a well-established research base that stands among the world leaders in major disciplines. Statistics may be dulled by repetition, but for a country of our size to hold second place globally to the US in significant subject fields is no mean achievement, and we should not apologise for returning regularly to this leitmotiv.

Although the results this time were presented according to subject profiles, the Times Higher, like other newspapers, immediately compiled a league table based on a Grade-Point Average approach. Their Top 10 is as follows:

1 Institute of Cancer Research
2 University of Cambridge
3 London School of Hygiene & Tropical Medicine
4 London School of Economics and Political Science
=4 University of Oxford
6 Imperial College London
7 University College London
8 University of Manchester
9 University of Warwick
10 University of York

The Institute of Cancer Research and the LondonSchool of Hygiene & Tropical Medicine score very highly in a very few disciplines, and their specialisation allows them to take 1st and 3rd spots in the table. It is good to see six of our UK Partner institutions in the top 10. The University of York emerges as the leading 1994 Group institution (small, research-intensive universities).

Looking at our other Partners, Edinburgh climbs four places to 12th; Leeds jumps 12 places to 14th; SOAS drops one place to 31st; Glasgow drops four to 33rd; Aberdeen rises nine places to 38th; and Liverpool climbs one to 40th.

In Scotland, the ranking within the top 50 emerges as Edinburgh, followed by St Andrews, Glasgow, Aberdeen, Dundee, Heriot-Watt and Strathclyde. Scotland’s experiment in cross-university research collaboration also appears to have been vindicated. The Herald reports that:

As a result of weak performance in a number of subject areas in the 2001 RAE, the Scottish Funding Council has been instrumental in setting up so-called research pools, where departments from different universities work together.

Research pooling is based on the simple concept that, working in isolation, researchers can end up competing with each other, whereas together they can become an international force.

The first research pools were in the disciplines of economics, physics, chemistry, nursing, midwifery, allied health professions and some areas of engineering. The amount of research in all of these areas has significantly improved.

In particular, EaStChem, a chemistry collaboration between Edinburgh and St Andrews, emerges as one of the UK’s top five Chemistry schools.

This was the sixth and final RAE. The libraries of all of these universities will now be readying themselves to contribute within improved systems of research information management which universities are developing for the REF (Research Excellence Framework) which will now replace it. As they do so, they might glance with some puzzlement at the league table for library and information schools. It puts Sheffield at the top, and King’s College London in 2nd place. King’s College has no library or information school, but was permitted to enter its Centre for Computing in the Humanities in the Library & Information Management category. Below King’s come UCL, Wolverhampton, City University and The Robert Gordon University in Aberdeen. Loughborough sits in a surprising 9th position.

And not everyone is quoting the Times Higher. An alternative Power Table is produced by the international researcher publication ResearchResearch, which uses a different formula for its rankings. It puts Oxford ahead of Cambridge, and insists that the top 6 positions are unchanged from 2001 (Oxford, Cambridge, UCL, Manchester, Edinburgh and Imperial). The quibbling starts now.

Reputation in the university sector has never had a higher premium than at the present time, as the world’s universities compete with each other more overtly and more globally than ever before. Later this week, the results of the UK’s RAE 2008 exercise will be released by the UK university funding councils. Universities across the UK are holding their collective breath to see whether their stock has risen or fallen since the previous exercise, in 2001. The results affect more than simply reputation, since the sharing out of £1.5b of UK research funding by the government for the next several years will also be determined by them. Although they won’t be issued in the form of a league table, you can rest assured that the press will very quickly compile one. The Times Higher Education is appearing a day early to coincide with the publication of the results. This blog will give a quick summary of the key findings.

Will Oxford, Cambridge and London still predominate? Who will have jumped the furthest since last time, and who will have fallen most spectacularly? Will the Scottish universities, forcing themselves to collaborate in ‘research pools’ in order to compete with the southern ‘golden triangle’ find their efforts vindicated? And, further down the line, which Vice Chancellors might fall on their maces as a result of poor showings or failed strategies in the submissions? Which deans will suddenly find early retirement attractive? Which departments might have to close as universities desperately adjust to an altered financial picture ahead, and the pressure to concentrate on the strongest subjects in order to win back lost income next time round becomes even more intense?

Of course, it is not just the UK which gets itself into a frenzy over rankings. The Times Higher last week reported that the Malaysian opposition leader has declared that the country should be ashamed of the poor performance of its ‘leading’ universities in the Times Higher’s own ranking of the world’s top 200 universities. What is somewhat extraordinary about this is the credence given by academic leaders, surely among the most critical and intelligent people on the planet, to league tables whose methodologies are often criticised as being of dubious value. What we see is the reality of media control over impact: the rankings may count for little in themselves, but once they are published and in the media, they are very hard to refute. Those who do complain about their inaccuracies or criticise the methodologies concerned – and it is likely that the academic press will once again contain a flurry of such comment after this UK RAE – will be ignored, or accused of being bad losers. And academics, attempting to make points about methodologies and statistics, can of course be easily dismissed as indigestible to the media.

In our Workflows in Research Assessment project, we are drawing upon the expertise of two Australian colleagues in our Expert Advisory Group – Colin Steele (ANU) and Ross Coleman (Sydney). Colin recently drew my attention to a background paper issued by the Group of Eight - Australia’s association of top research universities - which very usefully summarises the efforts of a range of countries across the world to concentrate research excellence as far as possible. The reason for this is that many countries which fund research largely out of the public purse now believe that the UK model has proven its worth, and that concentrating research makes countries more economically successful. Research assessment is therefore no longer really the point; what the effort is now aimed at is research excellence concentration (hence the term excellence in both the UK and Australian new versions of the exercise). The paper states that

Research by Ellen Hazelkorn (2008) for OECD demonstrates that the new body of comparative information, especially institutional rankings and research output metrics, has rapidly become installed in the perspectives, performance measurement systems and objectives of both national governments and higher education institutions; and is entering into the funding decisions of corporations, philanthropists and donors. Hazelkorn surveyed and interviewed institutional leaders in 41 countries on their response to university rankings and league tables. Almost universally, respondents testified that ‘rankings are a critical factor underpinning and informing institutional reputation’, affecting applications, especially from international students; university partnerships; government funding; and the employer valuation of graduates.

Countering those in Australia who question the policy of research concentration, the paper provides the evidence for a worldwide trend in countries with large amounts of public funding provided to research:

The UK’s Research Information Network (RIN) has just published a document aimed at UK university Vice Chancellors, Presidents and Principals, Ensuring a bright future for research libraries. Jim Michalko, Lorcan Dempsey and I met with representatives of RIN, along with other European library and research organisations, after our recent European Partner meeting in Paris, and it seems clear – as Lorcan remarks in his blog - that our work agenda for the RLG Partnership coincides in various ways with the work being undertaken by RIN. Here are a few examples.

In Linking library content and collections to research strategies they state:

No single institution can provide all the publications and other information resources – digital and non-digital – that their researchers need to consult in the course of their research… HEIs therefore should … seek to exploit the potential for collaboration with other libraries, including the national libraries and the five designated major research libraries in England.

In our Shared Print Collections programme, we say that

A new business model is needed that will enable research libraries to establish partnerships capable of sustaining the long-term future of print collections, distributing the costs and benefits of acquiring and preserving content in tangible formats, and allowing aggregate holdings to be “right sized” in view of aggregate demand.

They also recommend that universities

should … develop and implement policies and procedures to determine which information resources should be managed and preserved over the long term and how; which can be disposed of within a shorter time, and how such disposals should be managed … establish polices for managing their holdings of low-use printed material where the content is available in digital form; and participate in the UK Research Reserve and other collaborative initiatives to ensure that they adopt a planned and coherent approach to disposal

Our project to Deaccession Materials held in Print and Electronic Form, which we are running with the help of Ithaka and JSTOR, takes as its starting point

There is clearly a need for aggregated information about costs associated with storing, preserving and delivering material from print back runs of e-journals, as well as data on the costs of discarding titles before and after they have been placed in storage. A comprehensive roster of print archives and access agreements would also be a worthy contribution to efforts in this area, particularly a title-by-title registry of which instituions are committed to retaining which materials, and providing access to them.

Under the same heading, they request that universities Explicitly relate the development and acquisition of special collections of rare material to the research strengths of the institution. Our new theme Mobilizing Unique Materials includes a project to Define the State of Holdings and Description for Archives. This will use datamining methods to provide data which should help with that explicit identification of rare materials and research priorities within institutions.

Providing institutions with a system-wide view of archival collection descriptions would provide a new input into these prioritization decisions and could help inform funding agency support.

Under the theme of Cataloguing, navigation, discovery, delivery and access they ask universities to

encourage their libraries to share catalogue records with other libraries; to make them available through collaborative catalogues and online discovery services, both national and international; and to ensure that they are exposed and made available to users through Google and other search engines

In our Share Best Practices for Metadata Creation Workflows Project (within the Knowledge Structure theme) we say

Information professions are eager to know what workflows work best in different environments that could be applied to their own and that would facilitate metadata flow in and among libraries, archives and museums.

Our Infrastructure theme, meanwhile, has a range of work going on within the Web Enablement programme.

RIN advises universities to encourage their libraries to work with others in developing innovative services that integrate into researchers’ workflows. In our new programme, Support for the Research Process, we are just starting on an Academic Research Landscape Project

As a foundational stage of the program, we are carrying out an analysis of research workflows and research information management practices, to ‘anatomize’ the area into its various components.

RIN has a strong focus on scholarly communication, patchily tied in to research evaluation in the UK via the national Research Assessment Exercise and its developing successor.

HEIs … should …develop clear policies and procedures as to the roles that institutional and/or subject-based repositories should play in promoting access to institutional research outputs, as well as in facilitating the creation of registers of these outputs for research evaluation

They go on to address the library’s potential role in the contentious area of bibliometric approaches to research assessment. Institutions should

draw on the expertise and advice of library and information professionals in making use of bibliometric and cybermetric tools, which are likely to play an increasing role in the assessment and evaluation of research outputs and impact at international, national and institutional levels.

Our new Workflows in Research Assessment programme is in the process of commissioning a Survey of Current Practice which will

survey the research information management landscape across its various dimensions - cultural (what are the research assessment drivers?), geographic (which countries have well-developed infrastructures and systems?), technological (what systems are being employed or developed?) and institutional (how are libraries embedded into research information systems?).

The scope document for that survey makes explicit reference to analysing bibliometric approaches in use in a range of countries.

Finally, we have recently categorised our outputs into four main areas: Change and community (challenging editorials, Partner events, workshops, etc); Best practice architecture & standards; Beta development & tools; and Evidence - business intelligence and user observation. Business intelligence in one form is represented by reports and other outputs based on datamining. RIN urges UK universities to

seek to benchmark their library and information services for the support of research against comparable institutions both in the UK and overseas; and participate in collaborative work that seeks to identify and where possible to quantify the benefits and returns from investments that they make in their library and information services

This emphasis on return on investment is also a key theme for RLUK, as stated in its Strategic Plan 2008-2011 (as Demonstrating Value). Our programmes and projects provide many opportunities for assembling data which support the demonstration of value both institutionally and at various levels of collaboration.

The RIN report boldly asks Vice Chancellors, Presidents and Principals to invest more in their libraries - and points to libraries as sources of leadership on campus in new areas where establishing that authority will take strong and concerted effort:

The services that librarians and information professionals provide have … changed fundamentally over the past decade. They can now do much more to provide leadership that brings improvements in research performance and effectiveness … Librarians and information services need the resources and the continuing top-level support within their institutions to ensure that they can fulfil their potential and meet these challenges.

Let’s hope they listen! We are keen that the work which we are undertaking within OCLC Research in so many similar areas can add breadth to RIN’s work, and can gain some depth of understanding of the UK context from it. In conclusion, they come down to earth with a well-understood library case for the cooperative approach:

recognise that there is scope for cost savings through the sharing of information resources and expertise, and through the development of collaborative services

We couldn’t have put it any better.

The Australian government is revising its research assessment system, and is in the process of setting up ERA, Excellence in Research for Australia. This new system was an early commitment of the Labor Government elected in November of last year, and is replacing the Research Quality Framework (RQF) which the previous Government had started to develop in 2006, and which was intended to carve up AU$600 million in block grant research funding. That system did not reach fruition, despite (and partly because of) being very costly. The new system is designed to benchmark Australian research better within an international context, and is - for the moment - not intended to lead to a ranking-based carve-up of the research funding pot, though that option has been left in for the future.

One of the first outputs of the new process is a single ranked list (which can be downloaded here) of over 21,000 journals, broken down into 157 subject groups classified according to the Australian and New Zealand Standard Research Classification. Each journal is graded on a 4-tier system, A* (top 5 %), A (next 15%), B (next 30%) and C (next 50%). The allocation of journals to these bands has been made by learned societies and disciplinary bodies. There has already been a lot of concern expressed by academics in Australia, arguing that the journals list is not sufficiently widely representative, is too Anglophone, will skew publication practices among academics, and is invidious because a percentage banding system ignores intrinsic quality for the sake of convenience. In an open letter to Senator Kim Carr, an international group of philosophers states ‘The problem is not that judgments of quality in research cannot currently be made, but rather that in disciplines like Philosophy, those standards cannot be given simple, mechanical, or quantitative expression. Publisher and journal rankings are no substitute for direct assessment of a scholar’s work by knowledgeable peers.’

Metrics used in the evaluation of research are often discounted by academics as being too crude, while administrators favour them for being objective and so capable of supporting resource decisions to advance policy objectives. The previous Australian government was influenced by the UK’s adoption of metrics of research excellence which has been credited with improving its economic performance. It would appear that governments are prepared to tolerate a degree of hand-wringing by academics and game-playing by their managers if it leads to increased global competitiveness. The nobler aims of the Academy have no compelling match for tables and rankings.

Journal rankings have been made over many years, though normally not in such a visible and comprehensive way as in a list of over 21,000 journals - a total which is larger than that of both Thomson Reuters’ ISI Web of Knowledge and Elsevier’s Scopus, each of which has around 15,000 journals. Indeed, Ulrich’s Periodicals Directory estimates that the total number of peer-reviewed journals is around 22,000, so this Australian list must be almost complete. What is perhaps more interesting, in light of the fact that both the Australian and UK governments want to make extensive research assessment exercises as lightweight as possible, is the Pareto Principle in this context. Thomson Reuters itself admits that ‘A core of 3,000 … journals accounts for about 75% of published articles and over 90% of cited articles’. So why consider 21,000?

It will be interesting to see whether the Australian list becomes a reference point for researchers, and is used internationally - or whether the criticisms increase. Without the justification from metrics of the citation impact factor, which governs the ISI ranking, there is a danger that a peer-reviewed list could be dismissed as immediately out-of-date. Small publishers could well protest at the injustice it represents to their up-and-coming titles which have not been graded A or A*. Larger publishers will surely seek to consolidate their top-ranked journals and invest in those with the potential to join them, possibly cutting others loose as they do so.

Thus rankings based on metrics could once again lead to the research community, for lack of sufficiently strong better judgement, behaving in a Jekyll and Hyde manner, as characterised by Jean-Claude Guédon in his 2001 essay In Oldenburg’s Long Shadow: Librarians, Research Scientists, Publishers, and the Control of Scientific Publishing, conforming to higher aspirations one moment, and the next giving in to the grubbier demands of gamesmanship within a system which is largely out of their control.

(And, just in case you were wondering, the A* Library & Information Studies journals you want to be published in are, according to this list: Journal of Documentation, Library Quarterly, Library Trends, Management Science, Annual Review of Information Science and Technology, Journal of Information Science, MIS Quarterly, Library & Information Science Research, Information Systems Research, Information Research-An International Electronic Journal, School Library Media Research and Journal of the American Society for Information Science and Technology).

Because it’s Friday (and because I have a cold!), this is just a round up of bits I’ve been meaning to blog about. They are piling up, and I figure it’s better to get out even a little bit on each, rather than try to find the time to blog about each one in depth.

Jennifer and I did our webinar yesterday (Assessing the impact of special collections) — I blogged about this earlier this month. I wanted to let you know that our slides are in Slideshare. The webinar itself will be posted later in the month, after some vacations. I’ll have more to say about the discussion, and the results of the poll later. For those of you who took the poll, thanks very much!

There was an interesting story on NPR about reCAPTCHA. Last summer, I blogged about our use of reCAPTCHA for validating the comments on this blog. Over time, these small efforts have added up. Something like 1.3 billion word, which adds up to enough text to fill up more than 17,600 books. Beneficiaries have been the New York Times and the Internet Archive / Open Content Alliance. So comment away — you are helping to do great things.

Finally, I was sad to hear that the Party Copyright Blog was shutting down — but even more disturbed that it subsequently vanished. Fortunately, the “voice of the people” was heeded, and the blog was mostly restored. It’s a reminder of the fleeting nature of information on the web and the importance of preserving valuable resources.

Following the Digitization and the Humanities Symposium, Jennifer Schaffner and I wrote a brief report, The Impact of Digitizing Special Collections on Teaching and Scholarship: Reflections on a Symposium about Digitization and the Humanities (PDF — 10 pages). The report acts as a summary of the symposium, and also gives some calls to action.

One of the topics that came up for discussion during the symposium was use metrics. Here’s a snippet from the report.

A major take-away point from the symposium was: know your scholars. The RLG Programs partners
engaged with the call for measuring and evaluating the impact of special collections on research.
Alice Schreyer (University of Chicago) spoke of the plusses and minuses of existing metrics in
special collections, and called for unified practices…. [Paul] Courant had a slightly different view… a provost is looking for a few top
faculty members to say that special collections (digital or not) are valuable for research. Without
faculty voices, all the metrics we have may be worthless. Ultimately, what matters is that scholars
find and use primary materials, and view the library (and its collections) as a valuable resource.

Call for action:
Although we need evidence of the impact of digitization and the unique collections themselves,
quantitative metrics aren’t enough. We must make sure libraries and archives both measure use of
special collections and work with faculty to demonstrate their value for excellent research.

As a follow up to this discussion, Jennifer and I will be hosting a Webinar on Thursday August 14th (8 am Pacific daylight time). Our Webinars are opportunities for us to discuss ongoing work in RLG Programs — this one will be a little different because we’d like to explore the topic with those of you who are out in the trenches. Do you need use metrics? If so, what? Why? Would data would you like to be collecting, and what keeps you from collecting it?

In preparing for the Webinar, I read a not-yet-published paper by Elizabeth Yakel and Elizabeth Goldman. They analyze interviews done with those who “measure” in archives. Then they categorized why institutions measure. Out of curiosity, and in advance of our Webinar, please take the poll below (this is completely non-scientific, and will only be used to help shape the discussion). I’ll share the results later. Please also note that you may need to scroll over the questions in order to see the whole thing — all part of the grand experiment).

If you are interested in joining the Webinar and participating in this conversation, please send me an email (proffitm@oclc.org) and I will email you the information.

Over the last year, Günter and Ricky have been examining models of collaboration across the cultural heritage community, working with RLG partners to identify the most significant obstacles and incentives to effective library-archive-museum partnerships. This is part of a program of work exploring cross-domain convergence in organizational structures and service requirements. Günter has reported on some of this work here (May ‘08), here (November ‘07) and here (July ‘07). A final report from this project is expected soon. It should help us to understand how cooperation contributes to improved institutional performance, including greater discoverability of collections, better integration of functions, and increased operational efficiencies. All of these objectives are important to research institutions that want to participate fully in the network information economy.

In a related vein, Dennis recently queried members of the long-running SHARES inter-lending partnership about the hallmarks of ‘high-performance’ sharing partners. A new Working Group on High Performing Lenders has drafted a survey to

learn what qualities [SHARES participants] value in a supplier and which … partners consistently display those characteristics

I was interested to see that the criteria under consideration include some social behaviors and delivery options that respond to expectations that have been shaped (or at least sharpened) by the larger network environment. These include [with my annotations]:

• Quality of scanning/copying [surrogates should meet or exceed quality of original] 
• Willingness to supply rare or hard-to-find materials [if content is discoverable, it is assumed to be available]
• Quality of holdings data in WorldCat [supply chains rely on accurate and reliable disclosure]

The library’s ability to meet end-user expectations is dependent upon the performance of its service providers — including its inter-lending partners. This is increasingly true in an environment where ‘local’ holdings are assumed to be continuous with the collective collection of library content. The larger information network imposes a set of collaborative imperatives that cross organizational, institutional and geographic boundaries.

All this has led me to wonder, as the ARL Library Assessment Conference (Seattle, 4-7 August) approaches, about the kinds of metrics we use to assess the value and performance of cultural heritage institutions, especially those that serve the research community. My colleague John MacColl attended a recent meeting of representatives from some of the organizations involved with with establishing and monitoring research library metrics, including ARL (123 institutions in North America), SCONUL (172 institutions the UK) and CAUL (41 institutions in Australia). There is evidently some interest in harmonizing these measures so that research libraries can begin to benchmark their collections and services against global indices. This suggests that libraries are seeking to situate themselves in an expanding network of information service providers in which performance standards reflect common operational requirements and social norms.

There are a couple of measures in the current array of library performance indicators that can be used to assess inter-institutional cooperation on a system-wide scale. Resource sharing statistics provide a gauge of network participation and institutional co-dependence. Annual expenditures on shared infrastructure are another useful index of collaboration. Since the mid 1990’s the National Center for Educational Statistics has tracked US library investments in ‘bibliographic utilities, networks and consortia’ – an interesting combination of social and technological support systems — as part of overall library operational expenditures. ARL added this category to its statistical measures in 2004. An acknowledgement, perhaps, that institutional performance and achievement in the research library sector is increasingly dependent upon collaboration.

In other sectors — notably supply chain management — the tangible benefits of cooperation have been studied more carefully. [References] Some fascinating work has been done to model instruments for measuring collaboration amongst “chain members” to enable reliable benchmarking and performance assessment. [Simatupang, T. M., & Sridharan, R. (2005).] The LAM community might benefit from a similar assessment framework, which acknowledges the multi-dimensional character of cooperation (information sharing, decision synchronization, alignment of incentives) and highlights its real operational value.

This slide caused the most discussion and comment during my presentation at the AALL workshop about which I posted previously. I return to it here for a few reasons.

Some of these assertions have attained meme status. In particular I’ve noticed that Roy’s characterization of searching and finding (which he’s been saying since at least 2005 - I’m sure he can tell us the exact date of the coinage) and Lorcan’s dictum about discovery were listened to with some skepticism and resistance only 12 months ago. They are now treated as common knowledge and an accepted starting point for discussions of our issues. This is good for us. It focuses us on change.

The next two about getting our services and assets into the work flow of the user on the network and about needing to present users with all of our system-wide assets aren’t yet memes but they have entered the vocabulary. Lorcan’s ‘networkflow‘ coinage I find helpful and apt in getting at the essence of the way we work and the collective collection phrase (about which Constance has blogged and spoken continuously) neatly and alliteratively captures what people really want to access. I hear other people use these phrases without expecting that they need to be explained. This is progress. These two observations are really about how we should change our services and invest our energy.

The last assertion about selection is far from a meme. In fact, it may not be true. But it could tell us more than any of the others about where we can choose to disinvest and redirect resources and effort.

The formulation arose in a group discussion led by my colleague, Arnold Arcolio, while reporting on user testing and interviewing that he was leading in connection with WorldCat Local at the University of California. While he has much analysis to do and considerable discussion yet to come with UC colleagues, one of the preliminary observations emerging is that the test partcipants overwhelmingly approach the local (or group catalog) with an item already chosen. Using the catalog as a research tool - a place to refine a general interest into a small number of selected ‘best’ items that answer an immediate need - seems to happen very infrequently. In these early interviews the idea seemed quite unusual to the faculty and graduate student users of the catalog.

During our group discussion this user behavior was capsuled as “Selection takes place without us.” We were intrigued with the potential import for our processes and practices should evidence emerge showing this to be generally true. Our investments in description and classification, in the functionality of the local/group catalog and many other areas could be re-examined and recast. If selection takes place without us then our efforts could be redirected to activities valued by our users and our institutions. I’m interested in spinning out the range of impact but, of course, we need evidence to take this beyond a thought-experiment.