CHRIS: Custom Holdings Ranking Information System Implementing Value, Service, and Community in Interlibrary Loan (ILL)
Traditional ILL Custom Holdings • Three-tiered approach to managing Custom Holdings in Interlibrary Loan – Cost is king – Service is second – Rely on consortia (IDS, LVIS, Rapid) to monitor cost and performance evaluation.
Custom Holdings Defined as Queries • All libraries that will lend to us for Free. • All of those libraries that are in New York • All of those libraries that use LAND • All of those libraries that are in IDS
Custom Holdings Defined as Queries • All free libraries in proximate states • …That use UPS
Custom Holdings Defined as Queries • All libraries that will lend to us for $5 • That are proximate • …Ugh. Forget it. Too many queries to repeat regularly.
Problems • Requires a lot of legwork and labor to set up and maintain. • Ideally, custom holdings would be updated frequently to verify information • Hours of tedious, monotonous labor on an annual or semi-annual basis • Limited in scope; if the custom holdings list is short, you’re rolling the dice on cost and service
Problems ( con’t ) • Even after all that labor, it’s uncertain what you actually get for it. – It’s hard or impossible to efficiently extract the data you need from OCLC Policies Directory – Can’t even be certain that data is correct, cost is frequently misrepresented • Even the best custom holdings are based on individual knowledge (craft or anecdotal information, depending on your perspective).
SYB’s Approach to Custom Holdings • Network (Consortia or OCLC group affiliation), Cost, Service, Billing • IDS takes precedence • Free is better than Not free; $10 is better than $15 • Post receipt processing labor is discouraged – IFM favored over Invoices – Odyssey favored over Email
SYB’s Approach (con’t ) • Limited accounting for how GOOD a service is provided to us, only what level of service is provided • Limited accounting for geography and shipping methods for book borrowing • Both factors can add days (or even weeks!) to an ILL request • How to assign borrowing priorities that account for this?
Poor service to us = Poor service provided by us • If another library takes a week to respond to an article request, we have provided a poor service to our patrons • If a geographically remote library uses the slowest available shipping to get us an ILL loan, we have provided a poor service to our patrons
Introducing CHRIS • Currently MS Access based; shareable and adaptable • Optimize cost, service, and community- building • Highly adaptable and easy to update • Adjusts for the quality of service provided to us
Construction Methods • Simple scoring algorithm • About two dozen manual searches performed against the OCLC policies directory • Less laborious on the front end – maybe 1-2 hours of Excel cut-and-paste
Attributes for Articles • Delivery • EMST Method • Billing • Institution Type • Performance • Cost
Attributes for Loans • Delivery Method • EMST • Geographic • Billing Region • Performance • Institution Type • Cost
Attribute Collections • Cost – Free, then $5 increments • Performance – Based on Average Performance • Institutional Characteristics – Minimize back end labor
Supplemental Measures • OCLC Policies supplemented with several sources • Custom Holdings Helper Worksheet
Supplemental data pulled from OCLC Usage Statistics • Reciprocity Reports from OCLC Stats: do we have a “Trade Deficit” or “Trade Surplus” with another institution • Lender String and Transaction Reports from OCLC Stats: real time lender performance data • Fee Management Reports from OCLC Stats: cost per transaction grouped by Institution, Request Type (Loan Or Copy) • All data pulled from one year
Why OCLC Usage Statistics? • Provided most accurate representation of lender performance (even when they didn’t fill). • Easily generated and updatable without complex searching.
Scoring • Grouped by preferred attributes • Bins are based on empirical scrutiny: an attempt to predict who will give good service • Attributes are combinatorial in nature
Main Attributes in Scoring • 50 Total Points • Cost – Free, then $5 increments • Performance – Based on Median, Average Performance • Institutional Characteristics – Minimize back end labor
Attributes Comprising Total Score • 50 Total Points • Cost – 15 Points • Performance – 15 Points, Two Categories • Institutional Characteristics – 20 Points, Four/Five Categories
Supplemental Measures • Prefer Actual Data > Proxy; if we have real cost information, use that instead of Policies reports • Don’t punish (or reward) a few bad transactions. Require 5 Borrowing Requests for actual Performance Data to be Validated
Institution Type (Articles) Type Avg Response Days Total Percent Unfilled Score Assigned Medical 1.27 0.12 1 Academic 1.67 0.28 2 Research 1.75 0.27 2 Government 2.7 0.22 3 Community 2.68 0.55 4
Reciprocity Correction • If More than 50 Transactions between Institutions: • If Borrows : Total > 0.75: – Add 5 points • If Borrows : Total < 0.25 AND reciprocally free: – Subtract 5 points • I.e., attempt to correct imbalances by shifting institutions to a different CH group
Cost is Still King • But performance is just as important • Cost: 30% of Total, Radical jumps in score to reflect preference for low cost • Performance = Response Days + Accuracy = 30% • Institutional Characteristics: 40% Accumulated
Results • 8753 Ranked Institutions • SYB has transacted with 2473 of these • Most institutions are transacted infrequently
Number of Transactions NumRequests Loans Articles < 5 1775 2201 5-10 350 104 10-100 289 133 100 > 59 35 5 > 607 245
Summary of Transactions • Loans, max requests for a single institution: 1404 (NYP). Second: 1158 (BNG) • 75 Percentile for number of loan requests: 5 • Articles, max requests: 620 (BNG), Second: 565(BUF) • 75 Percentile for number of article requests: 1
Institution Types Institution Type Count Academic 3312 Community 2497 Government 771 Medical 36 Association of Research Libraries 102 Total Extracted 6718
Cost Range Count Free 2590 $0-$10 5934 $10-$15 142 Over 15 87 Total Extracted 8753
Top Ten Ranked Institutions Symbol Cost InstType DelMethod Region EMST IFM Performance Accuracy Rank VYT 2 2 1 1 0 0 1 3 15 YGM 2 2 1 1 0 0 1 3 15 ZGM 2 2 1 1 0 0 1 3 15 YJM 2 2 1 1 0 0 1 1 16 XNC 2 2 1 1 0 0 1 2 18 VVV 2 2 1 1 0 0 1 2 18 BNG 2 2 1 1 0 0 1 2 18 VVB 2 2 1 1 0 0 1 2 18 YJA 2 2 1 1 0 0 1 2 18 ZVM 2 2 1 1 0 0 1 2 18 YPM 2 2 1 1 0 0 1 2 18
Missing Data • Use aggregate averages for missing values (i.e. incomplete policies directory info.) • Finer grained picture of our custom holdings set • Much larger set than is possible by coding Custom Holdings by hand • Missing Data can be reduced further by collaborating with other IDS institutions
Low-Lying Outliers, <$10 + Email symbol Cost InstType DelMethod EMST IFM Performance Accuracy RANK CHM 6 1 3 0 0 2 1 26 XII 6 2 3 0 0 2 1 28 CAI 6 3 3 0 0 2 1 30 PFM 6 2 3 0 0 2 2 30 TXA 6 2 3 0 0 2 2 30 CQL 6 3 3 0 0 2 1 30 UBY 6 2 3 0 0 2 2 30 YAM 6 3 3 0 0 2 1 30 YUH 6 2 3 0 0 2 2 30
Initial General Analysis of Custom Holdings • Major Consortial Partner Libraries still Highly Represented, but allows for other high performers to blend with IDS. • Comfort in Prioritizing IDS Libraries in Custom Holdings —it’s based on data.
Updating Values • OCLC Policies Directory subject to change on a day to day, seasonal, or other random basis • Can pull institutional characteristics on a quarterly basis: about an hour cut and paste; data translation SQL stored for quick re-use • Transaction, Performance, and Fee Reports show a rolling aggregation of the past year with monthly pulls from OCLC
Updating Values • Scoring system can be altered as easily as changing digits in a query; what works for SYB may not work for all institutions • But the model and underlying data can still be uniform, allowing for data sharing
Other Advantages • No more manual checking of IFM reports — CHRIS can provide a report of what libraries have costs different than “expected”.
Assessing CHRIS • Turnaround Time (excluding parts of workflow affected by borrower performance — i.e. time from awaiting req. processing to sent). • Cost-reducing (IFM and shipping) while improving turnaround time. • Variety of lenders used and reciprocity • Percentage within consortia • Reducing “outlier” transactions • “End Labor” Processing needed
Further development • CHRIS does videos and other formats • Analyze loan periods, and significantly factor in longer loan periods for loans. • Move from Access to online database that can be easily shared and configured
Recommend
More recommend
