Measure What Matters

The attached article, Return Policy, in Architect Magazine, is a reminder that learning design organizations need to measure what matters to them.  If what matters is high performance buildings and spaces that foster the performance of users, then as the article highlights, a post-occupancy evaluation can be an effective technique to measure that performance.

Citing best practice recommendations from three design firms, the article lays out both methods and strategies focused on revisiting the spaces that you have designed and built.  Janice Barnes from Perkins and Will sets out a perspective on bringing post-occupancy into a design practice. She offers four key steps a designer should take to create effective evaluations that both provide design direction and measure design effectiveness.

1.  Make sure it's a priority.  Does your firm have a mindset that this is important?  Because if not, it will always be pushed aside.

2.  Don't reinvent the wheel.  Invest once in developing a consistent protocol that includes a diverse set of tools for research and evaluation, such as focus groups, interviews, surveys and on-site visits.

3.  Refine your standard protocol for specific industries.  Consider the issues that consistently arise in that typology and build the necessary research into the process.

4.  Get the protocol evaluated.  Perkins and Will established their standardized PPOE and then vetted it through a research university.

Barnes offers a further insight that the key to successful post-occupancy evaluation is actually the pre-occupancy research.

'In order for the (POE) data that you're collecting to give you valid results, you have to first measure the problem you're solving, then design to solve that problem, and finally see if you solved it.  It's collected data pre- and post-occupancy.  That's why (at our firm) we call it PPOE instead of POE.'

While there are a number of factors that may make it complicated to integrate POE's into practice, the benefits to clients and design firms are becoming more clear.  In order for our practice to grow and learn, no matter what POE method is used, designers need to know how well their design solutions are addressing the challenges they have set out to solve for and with their clients.

Design is a Research Process

In the last few years there have been a number of articles focused on the integration of research into the design process. The majority of these articles have been framed from the perspective of a healthcare practice, however a design research perspective is applicable to any client project.  Two of these articles: How to Convert Key Design Issues into Research Questions by Mardelle Shepley and 6 Steps to Integrate Research into Healthcare Design by Upali Nanda and Tom Harvey offer advice from experienced researchers and practitioners regarding the challenge of integrating research into design and making it a formative part of the process.

As you look at the articles you will see that Mardelle Shepley acknowledges that designers are prolific generators of research topics, and that these important design questions are formed during the programming phase when primary design goals are formed. She shows how a design goal can become a design hypothesis by identifying examples of goals, defining components of a hypothesis and then sharing examples from three case studies. Her case study examples are particularly useful for taking a typical, somewhat general design goal, and crafting it into a design hypothesis which identifies a clear design feature and states a clear outcome. From my experience, this is one of the most important steps in clarifying design intentions.

In the second article Upali Nanda and Tom Harvey from HKS/CADRE take on many of the

same practice issues, but make an effort to link research issues with traditional design

phases. Their simplified framework is broken into six steps which are summarized and displayed in a graphic to show how the steps interface with typical design phases.

1. TARGET - Create designs based on key performance goals of the organization: set targets based on important client goals and define performance metrics to assess these goals.

2. EXPLORE/EXPERIMENT - Gather knowledge, understand users, simulate scenarios and test prototypes using tools that balance technology with empathy : explore and experiment with the potential of design options to answer design goals and design questions leading to a design hypothesis.

3. DEFINE - Link design solution to performance hypothesis: every design decision is a performance hypothesis and based upon project importance can be crafted into a clear testable hypothesis.

4. MEASURE - Identify key metrics for design and performance and collect baseline data. All measures are not quantitative: match design phase metrics/simulations with outcomes/performance metrics.

5. MONITOR - Ensure design is implemented as planned, aiming for targeted performance goals: track the project through documentation and construction to protect design features and hypotheses.

6. TEST - Test the hypothesis: to what extent did the project achieve the outcomes hypothesized and why? Remember that design doesn't "cause" improved outcomes; it creates compelling conditions for improved outcomes.

Both articles provide a useful perspectives to explore design as a research process.  Mardelle sums it all up at the end of her article.

"The generation of hypotheses during the design process is likely to clarify design thinking, support problem solving, and provide the base for potential research."