Archives

Measures bloody measures

Dr. Bill Irwin I owe you. You talked about performance measures and the logic model work by the Kellogg Foundation in February at OLA’s Superconference, and I didn’t follow up on it. Thankfully you were one of the highly informative speakers at Designing New Metrics for Libraries this past week and you again talked about Kellogg. This time I looked into it. BAZINGA!

Moe Hosseini-Ara @ City of Markham and I have been working on measures and, more specifically the logic model, for libraries for the last number of years. In fact, we are teaching a course on planning & measures at CLA on May 28th, and as a “working” course those attending will actually “work” on measures for their library (academic, public, government or whatever type of library).

www_fsg_org_Portals_0_Uploads_Documents_PDF_Evaluating_Community_Info_Projects_pdf_cpgn_WP_DL_-_Evaluating_Community_Info_Projects

There are many sources and tools to refer to as you are assessing services and programs, including articles Moe & I wrote for libraries using the logic model last summer.   But it is so important to see how other organizations offering similar information-based services develop their measures. Have a look at the report shown here: IMPACT: A Practical Guide to Evaluating Community Information Projects by the FSG Knight Foundation.  Better yet, follow FSG’s works.

Impact – that’s what we want. Community Information – that’s the business we are in, and certainly want to be seen as being in. So what can we learn from FSG? Lots.

“This guide will take you through the essential steps for designing an evaluation of your community
information project. These steps explain what to do and consider at different stages of the
evaluation process.”  The difference in how they design and develop their program or service from the way that many libraries develop their program or service is that they build evaluation and measurement in at the design phase.  If you already take this approach, then congrats – you are ahead of the curve.  But many libraries do not.  Evaluation is a critical element of any program or service.

The questions FSG asks as they develop a program can be, and should be asked by all those in the community change business – and that’s libraries, correct? In fact, the project FSG addresses is one that is so dear to the heart of many libraries: “residents are hungry to contribute to local news and information and engage in community issues but lack the tools to do so…… the project will  provide training, platforms and support to residents in order to empower underrepresented voices among the people of the specific community.”  

The questions they address as they design the program are the questions libraries address:

  1. What change do we want to see in the community? 
  2.  What is the issue? Who is affected by it? Why are you taking action? 
  3.  What will we do to address the issue?  And, I’d add here — Who else should we work with to address this? 
  4.  Although FSG asks “If the project is successful”, we should ask “When the program succeeds, what change, among which members of the community, do you hope to see?

And then we must ask:

What resources do we need?
How do we best use these resources? 
What activities need to occur? What are the tangible products of these activities?

 

FSG doesn’t ask some of these questions, but I encourage you to ask,

What’s our pilot phase? 
What changes do we expect to occur within the short term? (and define “short term” – 3 weeks? 3 months?)

 What changes do we want to see within the mid-term? (define mid-term)

What changes do we hope to see over time? (yep – define long-term or over time) 

This tool, and many of the tools from FSG are very valuable as we learn from other, community focused organizations.  Keep FSG in your sights. Keep Bill Irwin in your sights.

Oh – and don’t forget.  The last question you should always ask as you design a new service or program is:

“How will we celebrate when we complete our pilot phase? ” 

Just as we tend to forget to factor in evaluation and pilots into the design, we also overlook the celebration. And in this profession, we need to celebrate. A lot.

 

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>