OCLC RLP institutions are addressing library assessment challenges

Two people talking, Selena Wilke, Wikimedia Commons, public domain
Two people talking, Selena Wilke, Wikimedia Commons, public domain

The OCLC Research Library Partnership recently held the third virtual meeting of its Library Assessment Interest Group, engaging RLP institutions from the US, UK, Australia, and Hong Kong. This interest group accompanies and extends the three-part Webinar Series: Evaluating and Sharing Your Library’s Impact, which is coordinated by our colleagues at WebJunction.

Our interest group discussion followed the second webinar session in the series, Digging into Assessment Data: Trips, Tricks, and Tools of the Trade (recording and slides publicly available), and focused on case study discussions as to how RLP institutions are honing their assessment activities as a result of the webinar engagement.

Several institutions shared how they are planning or conducting usability testing of their current ILS systems, to better understand user behavior in order to continue to serve patrons well following future system migrations. Webinar presenter and library assessment expert Dr. Kara Reuter, Digital Library Manager at the Worthington Libraries (Ohio), provided useful feedback to participants on their assessment planning.  Institutions that have recently migrated to new ILS systems also shared how they are struggling with the discontinuity of their data, necessitating a lot of clean up and efforts to make the data understandable to stakeholders inside and outside the library.

Overall, several institutions report trying to move from a system of tracking transactions (reference desk visits, etc.) to a more informative system that will help them judge impact and outcomes, which requires greater expertise, teamwork, and different types of assessment. Many of our participating libraries shared how they are working to improve their assessment activities by developing teams, hiring dedicated assessment librarians to guide others, and working with other units within their institutions. They are also looking to move beyond the metrics, and to increase their ways of knowing by triangulating quantitative assessments with qualitative assessments—helping to capture a more complete story about the impact of the library.

For instance, La Trobe University in Melbourne, Australia, shared how they have partnered with others on campus to insert questions about the library into an annual student survey of more than 34,000 undergraduate and post-graduate students. By partnering with others on campus, the library should benefit from a larger sample size than their independent efforts can yield. They also receive external supporting in data analysis and coding. The La Trobe libraries are using this opportunity to try to answer the overall question as to why students hold the library in high regard (or not), by asking how the library contributes to research success, and if it facilitates feelings of “connectedness” to university life.

Assessing impact is extremely difficult. Dr. Reuter emphasized that when you are asking users questions about library impact, you are asking users about their lives. As a result, she advises librarians to focus less on the service/organization/library, but on the life of the individual user—and how the intervention in question impacted them. Melissa Bowles-Terry, Head of Educational Initiatives from the UNLV Libraries (and presenter in our upcoming third webinar entitled Take Action: Using and Presenting Research Findings to Make Your Case) further emphasized the importance of specificity—asking questions about behaviors and specific challenges. A lot of the information yielded won’t necessarily be about the library, but that’s okay because users may not understand the library’s offerings anyway. But it will bring back answers about impact.

We also discussed how impact assessment can be creative and fun. For instance, the St Andrews Library (Scotland) invites users to write library love letters or break-up letters on Valentine’s Day each year. Using post-its, pens, and a large poster in the library entrance, the library invites users to share what they love and hate about the library.