November 6, 2019

Why and How to Conduct Post-Occupancy Evaluations on Your Spaces

Achieving Impact

By Elliot Felix

Colleges and universities are evolving to be more data-driven. Architects and engineers are embracing lean design. Sustainable, high-performance buildings are becoming the norm. Institutions are becoming more strategic about their building projects to ensure they achieve campus goals. So, for these reasons and more, post-occupancy evaluations that measure performance and success are needed now more than ever.

Unfortunately, most institutions lack the key components for conducting post-occupancy evaluations: a consistent, rigorous methodology for conducting post-occupancy evaluations, baseline data for comparison, benchmark data for context, and the culture of assessment and continuous improvement.

As a campus facilities leader, if you want to better measure and improve the performance of your projects and teams, this post will help you understand what a post-occupancy evaluation is, why you should do them, what you should consider measuring, the methodology and tools you can use to measure it, the process to use, and tips for getting started.

Post-occupancy evaluation, NYU Tech Spaces

What is a Post-Occupancy Evaluation?

A post-occupancy evaluation measures two somewhat overlapping things: first, the performance of the building in areas like energy usage and occupancy, and second, the success of a facility in achieving its goals like enabling active learning and supporting specific activities like seamless student services. It does this by combining quantitative and qualitative data drawn from surveys, interviews, observations, workshops, and mining other sources of data such as utilization of energy or services. For example, to measure the success of new creative, collaborative technology spaces for NYU, we compared student satisfaction surveys taken before and after the renovation, mined anonymous software usage data, observed the new spaces in use, interviewed users, and conducted focus groups. All these activities then came together to tell the project’s success story, including 93% satisfaction with technology and 63% less use of email and social media in keeping with the project goals to create a place for creative projects, not just checking email and printing.

Work+ Flexible Working Program, University of Minnesota

Why Conduct a Post-Occupancy Evaluation?

Post-occupancy evaluations can help you measure performance and success, better communicate these results, build momentum through communication and participation, and enable a culture of evidence-based design. They can also create a mindset of continuous improvement on campus so that each project informs the next. This is critical because buildings are always evolving, and so space should be thought of as a dynamic service, not a fixed product that design teams deliver and is complete when you move in. For instance, when we developed the University of Minnesota’s Work+ Flexible Working Program, we evaluated the success of the initial pilot with the Office of Human Resources to better understand and communicate its value as well as inform how to scale and evolve the program. One surprising finding: staff response times decreased 69% and this indicated that the right combination of space, technology, and policies enabled much more efficient and effective work.

What Do You Measure in a Post-Occupancy Evaluation?

There are generally three kinds of measures in a post-occupancy evaluation, and these can be thought of as tiers since each is more difficult to obtain and analyze but also more valuable as a metric.

  1. The most basic measure is utilization – how much something is used – typically by mining usage and transaction data; for instance, we planned a library and campus center at Young Harris College with VMDO Architects whose visits increased 113%. We also planned a next-generation library at NC State University with Snøhetta and Clark Nexsen whose reference questions increased 10% and technology device lending increased 16%.
  2. The next tier up is measuring satisfaction – how happy are people with what they are using – typically gathered through surveys. For example, 93% of students are satisfied with NC State’s library and 89% of students are satisfied with NYU’s technology spaces.
  3. The third tier is measuring impact – what are the outcomes of using the space and service? What did they enable? For instance, our workplace strategy at the University of Michigan yielded an average productivity savings of 4.26 hours per week, our training program to support the tech spaces at NYU increased staff confidence by 39%, another project achieved 89% staff engagement, and another project reduced operational costs by 30%.
Change in Space Satisfaction for Weiser Hall at University of Michigan

What Should You Consider as You Decide What to Measure?

A great starting point is to think about what you will do with the data you gather, what you are willing to change as a result of your findings, and your comfort hearing some potentially “bad news” that your new, expensive project might not be perfect. Be clear about this from the start; you’ll want to set people’s expectations that you might refine policies and operations as well as make minor space adjustments such as moving a group but not knocking down walls you just built. Another important consideration is how important are peer or industry comparisons versus asking specific, bespoke questions? Finally, think about what platforms have performance data that you already have in place; for instance, data on room utilization staff absenteeism and energy usage are commonly available.

If comparisons are important, then a standard assessment survey such as the Center for the Built Environment (CBE) might be the way to go, or you could create a standard process at your institution and use it consistently to compare results. As you go about gathering data, knowing what existing data is available is important, and so is deciding whether you’ll be using a representative sampling of users or a convenience sample. When to conduct the post-occupancy evaluation is also important to consider – we recommend about six months after opening so you have time to work out operational kinks that might unduly influence perceptions (you don’t want the fact that someone is having trouble logging into their phone leading them to trash the space overall). Finally, it’s best to go into the process understanding that there are trade-offs and value judgments; for instance, is it okay for people to report greater distractions if their overall satisfaction and productivity go up?

Weiser Hall, University of Michigan

What Methodology Should You Use to Measure Performance?

Once you understand what you can measure and you have thought through the implications, you’ll need a rigorous methodology for evaluating performance. We have four principles to guide you:

  1. Assess your project’s performance relative to its targets and its success relative to its goals. You’ll want to determine how well the facility met its stated objectives; for instance, if a goal is interdisciplinary research, what was the change in grant applications or papers representing multiple disciplines? Having specific goals upfront to measure against is what enables you to use space strategically.
  2. Compare performance before and after a change. Conduct a pre-occupancy evaluation to determine the needs and then compare the post-occupancy results to see how well they’ve been met. (The pre-occupancy data also helps you create your vision and strategy – for the project and broader campus initiatives.) If you don’t have pre-occupancy data, you can ask people about what they perceive as impacts; for instance, is it easier to collaborate than before?
  3. Compare your data to get the full picture. Your analysis should combine quantitative and qualitative data, compare subjective data like satisfaction with objective data like transactions, and ideally compare internal data from your institution with external data from peers or industry averages. The more standard metrics that can be used the better because you’ll be able to compare across projects and institutions while design teams then become accustomed to working towards a common set of metrics, such as student engagement.
  4. Measure activities when you can’t measure outcomes. In a laboratory you might be able to isolate a causal relationship between a space and an outcome (e.g., higher ceilings lead to more creativity). But, in the real world, there are often too many intervening factors (e.g., a professor is sick and off her game the day you evaluate your new active learning classroom). So, activities are a good proxy; you can see whether people are doing more or less of activities that are correlated with success and then ask to what degree people see the facilities as inhibiting or enabling those activities.
Hunt Library Service Point, NC State University (Credit: Snøhetta and Clark Nexsen)

What is the Process for Conducting a Post-Occupancy Evaluation?

First, define the research questions you want to answer and the initial hypotheses that you aim to prove or disprove. Then, compile and analyze the data that are already available, which may include things like pre-occupancy data, goals, service transactions, space booking/usage data, or computer logins. Next, it’s time to gather new data through online surveys, conduct observations of spaces in use with intercept interviews, and facilitate workshops with students, faculty, and staff. It’s usually best to conduct the survey first so that the other methods can be used to better understand the survey results. From there, it’s time to bring the existing and new data together to create findings and recommendations to address them. For example, on NYU’s tech spaces mentioned above, our survey and focus groups uncovered that the space could be more welcoming if staff were more visible (i.e., not hidden behind monitors and roving throughout the space) so this was an operational change NYU made as a result.

What Should You Keep in Mind as You Get Started?

Even though conducting post-occupancy evaluations on building projects is unfortunately the exception rather than the rule, you don’t have to start from scratch; for instance, the Royal Institute of British Architects (RIBA) has compiled resources and guidance on post-occupancy evaluations you can reference. You can also learn from industry publications like this article in Metropolis or this one in Building Design + Construction or this one in Architectural Record. There may also be existing resources on your campus you can tap; for instance, every institution has an Office of Institutional Research (or similar) who are skilled at assessment and analysis.

RIBA’s schedule of activities for post-occupancy evaluations

A good place to start is by gathering pre-occupancy data so you have a baseline to compare against. So, on your next project be sure to get a sense of how things are working today through mining data and conducting a survey, interviews, and focus groups with users. When you are gathering pre- and post-occupancy data, try to mix methods so that you test the validity of your analysis; for instance, ask the same question in a survey and a focus group to compare results. Make sure whoever is conducting the analysis is sufficiently neutral so you can get honest feedback from users and unbiased analysis. Finally, don’t do the post-occupancy analysis in a vacuum; report back to occupants to review, validate, and act on findings, either for that space or to inform subsequent projects

Our hope is that the design process will become more data-driven, high-performance buildings become the norm, and campuses and architects alike embrace a culture of feedback and continuous improvement. Then, post-occupancy evaluations will become the norm tomorrow rather than the exception they are today. As a facilities leader, you can make this happen on your campus by using post-occupancy evaluations to measure success, better communicate project results, build momentum, and enable a culture of evidence-based design. Good luck as you move ahead!

=======

Acknowledgements: I’d like to thank Dina Sorensen for her comments on this article and sharing insights from recent efforts within the American Institute of Architects (AIA) to make POEs a standard practice.

Related articles