Evaluating and Sharing Library Impact–with the OCLC Research Library Partnership

The OCLC Research Library Partnership recently held the first virtual meetings of its newly-launched Library Assessment Interest Group, and engaged RLP members from locales worldwide, including the US, Canada, the UK, Australia, and Hong Kong. This interest group accompanies and extends the three-part Webinar Series: Evaluating and Sharing Your Library’s Impact, which is coordinated by our colleagues at WebJunction, who have helped support community, collaboration, and lifelong learning for libraries since 2003.

Our goal with this interest group is to further scale learning and engagement on the topic of library assessment activities across Research Library Partnership institutions, by encouraging participating librarians to share their challenges, knowledge, and experiences as they participate in the webinars, virtual interest group meetings, and a dedicated email discussion group. The interest group is facilitated by OCLC RLP program officers Merrilee Proffitt, Titia van der Werf, and Rebecca Bryant, and to accommodate members across the globe, we arrange virtual interest group meetings in different time zones.

We know that library assessment is an important issue for many RLP institutions, and several institutions have signed up as teams, an important first step toward achieving what one participant described as “building a culture of assessment” at her institution. Participants have described their personal goals as seeking to “get past the metrics,” and to develop confidence in their assessment activities and communications. While all participants are partnership members from research libraries, they represent a broad range of previous experience with library assessment, offering opportunities for members to learn from each other as we work together to share and compare evaluation activities across libraries.

Our first interest group discussions followed the initial webinar in the series, User-centered Assessment: Leveraging What You Know and Filling in the Gaps, by Lynn Silipigni Connaway and Kara Reuter (recording and slides now available). We organized our discussion by using the publicly-available Webinar Series Learner Guide and focused on the first three steps of the assessment process as described in the webinar:

• Define the problem
• Environmental scanning
• Prepare the planning

In the course of our interest group discussions, I noticed a few recurring themes from librarians across the partnership:

• Libraries are increasingly seeking to align their strategic goals and reporting efforts with institutional goals. This can present new challenges about both the metrics that are collected, but also “what bits do you pick up to push outside the library?” We discussed the challenges of engaging with key stakeholders to learn more about how they form a view on whether or not the library is delivering value–to help librarians tell more persuasive stories to these audiences.
• Data overload can impede planning. Several librarians reported that their library – as well as library and university consortia – collected “tons of data”, but it was often unclear for what purpose, or if it was the right data or answering their local questions. Having so much data could lead to librarians feeling overwhelmed or as if they had to use that data for assessment—without first asking if it was the information needed. In response, one participant described how his institution is conducting a “data census” in order to catalog a canonical list of all the data collected by the institution. While driven initially by privacy concerns, it’s hoped that this effort will also help influence institutional attitudes toward accessing the right or “curated” data sets, rather than just trying to grab up everything.
• Librarians can make use of reference skills. The reference interview skills developed to support patron research and discovery are also of great use in this setting, although with the additional challenge of having to recast the questions back to ourselves.
• Research libraries are dedicating resources to assessment, and several participating institutions have a dedicated assessment librarian who provides leadership and expertise to help support quality assessment. In this role, one participant described her responsibility as often “refining the questions” to arrive at the right assessment program.

Following the webinar and interest group discussion, I offer a couple of high level takeaways for our partners beginning their assessment efforts:

1. Look at  institutional strategic plans, and consider the library’s role. While we won’t talk in depth about communications until the third webinar in the series, it’s essential to understand the institutional stakeholders who will be hearing–and judging–the library’s future assessments and communications.
2. Step away from the data you already have so you can take a fresh look at what you want to know. Challenge your existing assumptions and don’t assume that since you have “tons of data” that you have to use it. Ask if it’s the right question(s).
3. Don’t get bogged down with questions about “how” you will do the assessment before you have finished defining the problem. There is a time to focus on the “how” of data collection and analysis (in fact, it’s the focus of the second webinar in the series, coming up on August 14), but it needs to come after you have clearly articulated the assessment questions.

Our RLP interest group will be virtually meeting again in July, to further discuss member assessment activities. Throughout the next few months, we invite our interest group members to:

1. Engage with the questions in the learner guide
2. Define their assessment question(s) and begin planning
3. Share with others in the interest group, through our online community and in our next virtual meetings

While this small Library Assessment Interest Group is limited to librarians at Research Library Partnership member institutions, please note that the webinars and learners guide are made publicly available by WebJunction.