May 9, 2018
How Can the Quality of a Student’s Experience be Measured?
How can the quality of a student’s experience be measured?
This was the topic of our Q1 Hack-A-Thon, a quarterly learning day where we set aside time to work through a question as a team that has been puzzling us and whose solution can help our clients transform higher education.
What did we learn from looking back on our work improving the student experience and thinking ahead about a new tool to measure it?
- We think measuring a student’s experience is an opportunity for the university not just to assess and analyze but also to demonstrate that it cares about each student as an individual.
- We noted that who a “student” is has changed considerably over the years in terms of gender, race, and age – for instance, 38% of undergrads are over 25 years old according to the Lumina Foundation.
- We also determined that any good tool should pair subjective data we gather like % satisfaction with objective data we mine like the student : faculty ratio.
- While any solution should be written from the student point-of-view (I am a basketball player celebrating Ramadan), not the language of bureaucracy (We provide excellent resources to meet the unique needs of our diverse student population), it needs to tie directly to common organizational structures within colleges and universities so they can readily act on the results.
How did we get there?
First we conducted research on existing assessment tools. Then we broke up into three different teams to generate a range of solutions. Each team created an approach to define different “dimensions” of student experience, and then developed an assessment methodology to determine what data would be needed and how it’d be gathered, analyzed, and visualized. Each team then created mock-ups and prototypes of their tool for feedback and testing. We discussed and critiqued at the mid-point and end-point of the process to learn from and challenge each other.
From the research, we learned that The UK and Australia both have national student experience surveys. Times Higher Education also has a national survey for the UK. In the US, we have NSSE but that only gets at academic engagement. We have the SERU survey but that doesn’t provide a holistic view either and only goes to consortium members. Many institutions are creating their own; such as the University of California system’s new undergraduate experience survey.
Our hacks included (A) a survey rubric tied to the different touchpoints in the experience like spaces, services, programs, and people to (B) an app that could take the ongoing pulse of the student body one student at a time to (C) a physical place where students could gather and witness this measure of them expressed as a dynamic informational graphic. While the form-factors varied widely, our different teams had much in common. We all wanted something from the student point of view that was useful for universities to act on. We all recognized that some needs are more important than others and concluded that different criteria should be weighted in some way. We all thought that no one tool could do it all; for instance, a survey should be coupled with focus groups to help interpret its results.
To start understanding the dimensions of student experience, we looked at existing approaches to codifying and quantifying human needs such as Gallup’s Q12 or – even better – Maslow’s Hierarchy of Needs that moves up from physiological needs to safety to belonging to esteem to self-actualization. From there, discussions ranged from understanding assumptions and potential biases behind what has traditionally shaped the student experience, in comparison to the ‘multiplicity and diversity’ of the contemporary student body. By openly acknowledging tropes, we liberated ourselves to set aside our own biases and apply what we have learned through our work as brightspotters about the contemporary student and how their experience is changing.
In the end, the office landed on a survey that we are excited to pilot in the field. If your institution is looking to measure student success, ping us. We’ve laid the groundwork and are ready to test this in the field!
