October 16, 2011
Measuring Success: From Usage to Satisfaction to Impact
You can tell a lot from how an organization measures success – what they value, what they know about their customers, and the tools they use, for example. You can also understand the dominant paradigm of whatever industry you’re looking at and then compare across them.
There are three types of success measures: usage, satisfaction, and impact. Each successive measure is harder to obtain and indicates the relative maturity of an organization or industry in understanding how well it is doing, whether it’s a dental practice, a library or an online retailer.
USAGE statistics are the starting point: how many people go to a website, click on a link, visit a place, check out a book, or ask for help? This provides a basic level of understanding about what is being provided and whether it is providing some value when utilization is taken as a proxy for need, usefulness, and desirability. Think back 10+ years to when the number of members or subscribers was the only statistic used to take a company public or how libraries and museums advertise their gate count as evidence of their value. The trouble is of course that people could be visiting and clicking for a variety of reasons: perhaps they have no other options or don’t know any better, perhaps they dread every second of it, perhaps they only come once and never come back, or perhaps it barely helps them do what they need to. So, we need to dig a little deeper.
SATISFACTION is one such measure with which we can get a better sense of not just how much something is used but how well its users think it is working. The advent of simple and cheap survey tools (e.g.: survey monkey and Google Forms) and now mining social media platforms (e.g.: Twitter and Facebook) have made it remarkably easier to get a sense of what customers think about your products and services. This is where most organizations stop though. If you’re a dentist you’ll know how many patients you’ve seen and how satisfied they are. If you’re a library you’ll know how many people visit, how much they check out, and you can use instruments like LIBQual+ to see how satisfied people are. You can use customers’ satisfaction as a proxy for the impact you have, but you don’t really know.
IMPACT is an even more elusive measure, telling you whether or not you’ve made a difference for your customers, enabling them do what they otherwise couldn’t. Not enough industries and organizations are at this stage. Transactions over the web make this easier than other domains, correlating clicks not only with time on page and satisfaction but also conversion rates for purchases or even referrals to others. In the bricks and mortar world, things are much harder as it is difficult to control for and correlate physical objects and activities with specific outcomes. Several promising initiatives are underway trying to catch up, such as ACRL’s Value of Libraries Study or the EDCAUSE Seeking Evidence of Impact program. Value-Added Analysis is also an attempt at this for education reform.
Of course, there will always be some things that cannot be fully measured – values, emotions, culture, and creative expression come to mind. However difficult, organizations must strive to move up the chain from usage to satisfaction to impact if they want to add value for their customers and the broader world. The areas that are lagging behind can take cues from the more advanced ones; for instance, months ago I heard Dr. John Falk astutely observe that most museums still don’t know who their visitors are on a given day. I’d say they should talk to convention organizers as they’ve known who’s on the floor, what booths they visit, and how to get in touch with them for decades. Where is your organization on this journey and how are you moving ahead?