BULLET Public Library Benefits Valuation Study
Appendix B - Evaluations from Partner Library Systems

Evaluations from Partner Library Systems

Partner Libraries completed a survey in October 2000. These are their responses to the questions. St. Louis Public's director did not complete an evaluation because he was the principal investigator.

1. Would your staff be capable of implementing the measurement methodology developed in this project?

Baltimore County Library:  No. I feel very fortunate to be able to use the team's expertise. This methodology is complicated to implement, and the results (beyond the basic) are hard to explain without some outside assistance.

Birmingham Public Library:  No. No one on staff has the expertise needed.

King County Library System:  No. We couldn't do the whole thing from scratch. We understand the methodology and could work with a consultant who would do the actual calculations.

Phoenix Public Library:  Yes. Phoenix Public Library can create the sample patron database and is currently working with a local firm to conduct the survey.


2. Would your staff be capable of implementing the measurement methodology with the assistance of an economist?

Baltimore County Library:  Yes. We have a certain level of experience with numbers and evaluation but would need this type of help. Don E has spoiled me for life.

Birmingham Public Library:  Yes.

King County Library System:  Yes. We thoroughly understand the methodology.

Phoenix Public Library:  Yes. It has been a good learning experience working with Don Elliot. His professional insights and guidance were essential to the success of the project.

3. Will the results assist you in allocating resources and tracking benefits to specific user groups?

Baltimore County Library:  Yes. It allows the public's dollar value perceptions to be part of our service/collections decision-making in a way that was never possible before.

Birmingham Public Library:  Yes. The survey helped us to understand that collections are good but without the expertise of a well-trained staff, patrons would suffer. Recognizing this, we reorganized and developed a coordinator of staff Development.

King County Library System:  Yes. I'm not sure yet how we will utilize the information with regard to future allocations for budget purposes. We have not yet had a discussion with our Collection Management Services staff about the implications and whether, based on the results, we should reallocate our collection budgets.

Phoenix Public Library:  Yes. Results can be used to garner public and city support for library activities, programs, initiatives, and increase financial support. We will also use the results to re-examine resource allocation.

4. Do you believe that the methodology provides a "defensible floor" for the estimated return on public investment in library services?

Baltimore County Library:  Yes. As long as the questions are basic. More sophisticated questions require more complex answers that are harder, sometimes impossible, to deliver.

Birmingham Public Library:  Yes. Because it is directly from users and is demonstratively conservative.

King County Library System:  Yes. Don convinced us! By utilizing the most conservative approach we are indeed demonstrating that investment in library services is an excellent investment for tax-paying public.

Phoenix Public Library:  Yes. The conservative approach to the study makes the results totally defensible to budget analysts and city management.

5. Do you believe the information gained through this research will positively influence constituent attitudes about your library systems performance?

Baltimore County Library:  Yes. I feel it reinforces the value of the library beyond the philosophical. While the emotional impact should not be ignored, this adds a very powerful element to a library's arguments.

Birmingham Public Library:  Yes. This information provides an excellent tool for communicating to a library's arguments.

King County Library System:  Yes. Bill presented the results to the King County Council and reaction was extremely favorable. he will also include it in the presentations of the proposed 2001 budget, which he does every year in several community locations. Everyone who has heard the presentation has been positive; none have expressed skepticism.

Phoenix Public Library:  Yes. The result will be shared with the Library Board, friends, Foundation, City management, and citizens. We do believe the information will have a positive impact.

6. Do you consider this methodology transportable to other libraries? If no, could or should the research be scaled to a more modest size?

Baltimore County Library:  No. Not easily, at any rate. It could be with outside assistance. A "scaled down" version would be better IF the results could still be defended.

Birmingham Public Library:  No. Without access to an economist, this would prove to be rather difficult. Lack of funding and staff time is also a deterrent.

King County Library System:  Yes. With the understanding that it is not a turnkey package that can be implemented without professional help.

Phoenix Public Library:  Yes. It is a very straightforward, conservative, and an excellent approach to data gathering and analysis that can be used by any size public library. However, it does need to be scaled to a more moderate size to meet the needs of most public libraries.

7. Do you believe that this research had produced a policy tool worth the investment of federal tax dollars and your institutions times and effort?

Baltimore County Library:  Yes. This type of information is critical to public libraries as they "pitch" decision-makers and make tough allocation decisions for limited resources but... (see #9 below)

Birmingham Public Library:  Yes. This research is worth the investment but the methodology needs to be streamlined.

King County Library System:  Yes. It is a useful tool for reasons mentioned above, primarily as a tool for explaining our worth and value to the community, officials and staff. I believe that if federal dollars are invested that the process will be improved and other libraries, that perhaps have more difficulty explaining their value will benefit from being able to use it.

Phoenix Public Library:  Yes. Studies prior to this one addressed local issues rather than universal commonalties of service offered by public libraries. The study is based on proven methodologies by the academic community with a proven track record in the private sector.

8. Please estimate approximately how many hours your library staff spent working on this project.

Baltimore County Library:  470 hours

Birmingham Public Library:  Approximately 420 hours of staff time plus the time spent by the Associate Directors for travel and attending meetings.

King County Library System:  Well, it was a fair amount. Outside of the meetings, which we all attended and I believe that was three full days, I probably spent two to three hours following each full meeting gathering information needed and communicating with SLPL staff and consultants. We also spent three or more hours preparing the power point presentations. It wasn't a huge investment of time and it was spread out over a long period. As I add up what I've mentioned, it seems low. I didn't keep track, but guess it would be more in the range of twenty hours in addition to the meetings.

Phoenix Public Library:  Approximately 180 hours.

9. Please provide any additional comments regarding your institution's participation in this research.

Baltimore County Library:  This has been a very worthwhile experience. We received a vast amount of information that we never would have had without this initiative. Everyone involved was very knowledgeable and collegial. I am grateful for having been included.

My concern is expressed in comments above. This is a very special sophisticated approach, not generally nor easily understood by most libraries. Having been involved with other planning and evaluation tools that employed nomenclature and concepts based in libraries and seeing that libraries struggle with THOSE tools, I don't think that many will be able to pick this up and run with it - despite the team's excellent effort, preconference, binders and presentations.

Outside assistance and/or a simpler without being simplistic approach are necessary - perhaps, something of s "Cost-Benefit Analysis" Lite.

Birmingham Public Library:  Unlike many other institutions, our city government structure is more political than management oriented. Every thing is driven by politics. No one, at the present time, is interested in this kind of fiscal accountability. Once a year, we are required to explain why we want "more" than they have forecast on the budget. If the time is right, we will get it. If not, we will get what is forecast.

We appreciate having been asked to participate. We have learned a lot from this experience. Internally, it was great for staff morale and it helped us in justifying the need for changes in the organizational structure. This research allowed us to view ourselves through the eyes of the public. We were able to see what they thought in comparison to what we were actually doing. We feel that this experience was definitely a net gain for us.

King County Library System:  We enjoyed it very much. It has been a pleasure to be able to include it in many of our contracts with local groups.

Phoenix Public Library:  The initial results of the study in the Phoenix community forced a close examination of the sample database. That examination provided additional information about our community, its rapid growth, transient nature of the population in areas of the city, and dispelled some assumptions about our Hispanic population.

Complied 10-31-00

<Appendix A

Table of Contents

Appendix C>