January 10, 2005


Dear Colleagues,

There has been some concern about the data in the Statewide Assessment, and the Statewide Assessment Task Force (formed in 2002 to plan and implement a second portion of the statewide assessment) would like to clarify some of the concerns and reinforce the overall validity of the data in the assessment. There are several points about the validity of the data that we would like to affirm. First, there is some misunderstanding about the data that OCLC contracted to provide to the ICCMP. OCLC has fulfilled it contract with the ICCMP. In its contract with us, OCLC agreed to provide statistical reports for the following assessments, “statistical tables and graph view of the collections’ age, content and overlap, based on the parameters specified in the APG[ACAS Planning Guide], delivered via the World Wide Web, for each library to access.” They also agreed to provide, “two analysis databases, one according to the WLN LC Conspectus and one according to the WLN Dewey Conspectus.” OCLC also specified in the contract that analysis would be done at the Conspectus subject level (5,000 subjects). At the end of the project, OCLC provided, for no cost, analysis of 32 languages at all of the subject levels.

Other points about the data that may be helpful are the following.

• The data counts titles for each of the 86 libraries in the project.
• The data includes language, subject coverage to the 5,000 subject level, publication date, uniqueness and overlap data that is accurate for individual libraries and for libraries to compare themselves with other individual libraries.
• The data was collected in 2003. The data itself belongs to the state and will be available through the World Wide Web indefinitely.

Another concern that was mentioned at the Bibliographers’ Meeting on December 10 has to do with one column in the Statewide Assessment, the “All Libraries” column. This column sums all of the individual libraries counts of titles into one total count. Because it is the sum of the individual library counts, it is not deduplicated. In other words, if three libraries own the same title, they will all be counted in the “All Libraries” sum. When the committee to write the report on the statewide collection realized this, we asked OCLC if they could deduplicate this column. OCLC said that the deduplication required a complete re-running of the data and they could not at that time commit to that. The committee then looked at the Uniqueness study to see if we could use the data there to complete our task. We discovered that by dividing each of the counts in the columns in Uniqueness by the number of libraries who own the title, we could come up with a deduplicated count and use that for the statewide collection assessment. (A step-by-step guide to doing this is being put up on the website.) Although it would have been convenient to have a deduplicated statewide column, we were able to do the analysis without it.

Concern has also been expressed that libraries cannot compare themselves to a peer group in the manner allowed by such programs as the AMIGOS product. The ACAS product does allow for peer group comparisons, but the original committee felt that the libraries within Illinois are so diverse that establishing useful peer groups would be difficult or impossible in some cases. The committee also felt that having an LC and Dewey analysis was more important than a peer group analysis. Cost was also a factor. We felt that we already had specified a number of expensive projects. More might place the price out of our grasp. We stress that most of these decisions were made with extensive consultation with the membership through library conferences and presentations. In addition, peer analysis at the individual library level is available. Any library can be compared to any other library, and therefore benchmarking can be done.

Another wish has been to have a title list. This again was discussed, but most of the librarians on the task force felt that a title list was not as important as other factors, such as subject analysis. Title lists are often used as selection lists, and the bibliographers on the group felt that because of their age (remember we are looking at the entire collection over time here), title lists are not particularly useful. In addition, title lists are available through reports from Voyager, and that might be an avenue for libraries who wish to look at title lists.

In short, the Task Force on the Statewide Assessment wishes to assure each library that the data is valid, that it can be used for extensive assessment of your collections, and that the contract with OCLC was fulfilled as stated.

Task Force on The Statewide Assessment
Mary Munroe, Chair
Carla Tracy
Charlotte Johnson
Karen Schmidt
Kathy Walsh
Nancy Hessler