Design Thinking and Rapid Impact Assessment: A Match Made in Evaluation
Abstract
Between November 2018 and April 2019, the Evaluation Unit at Employment and Social Development Canada engaged the ESDC Innovation Lab to help conduct a Rapid Impact Assessment (RIA) of the New Horizons for Seniors Program (NHSP). NHSP provides funding for innovative projects enhancing seniors’ well-being. The ESDC Innovation Lab is a human-centered design lab within Employment and Social Development Canada (ESDC), which uses qualitative and mixed-methods research to improve government programs and services for Canadians, with Canadians. The Evaluation Unit tasked the Lab to lead the pilot RIA, believing the Lab’s expertise in user research would be of substantial benefit. The Lab paired Design Thinking with Rapid Impact Assessment methodology to develop a new approach to evaluation and user-centered alternative scenarios.
A Rapid Impact Assessment evaluates program outcomes relative to a counterfactual – an alternative scenario that is plausible, feasible and legally compliant. The innovative potential of a RIA lies in its participatory process, efficiency, flexibility, and ability to complement other lines of evidence through triangulation. RIAs have been used a handful of times by the Government of Canada. In preliminary research for the NHSP RIA, the Lab consulted reports and individuals involved in past Government of Canada trials. These RIAs used the absence of the program as counterfactual. The Lab sought to build on past approaches to RIA by developing a more complex, alternate scenario, co-developed with beneficiaries of NHSP funding, seniors organizations, internal stakeholders, businesses and subject-matter experts.
This RIA intended to fill knowledge gaps regarding program outcomes and the needs of recipients/end-users through balanced integration of multiple perspectives. The Lab used design-thinking methodology alongside the RIA: this approach assessed the past and present of the program, while suggesting design solutions for future program iterations. The Lab engaged stakeholders through multiple workshops, co-creating and stress-testing solutions to current challenges with participants for the development of counterfactuals.
Design thinking and Rapid Impact Assessment marry well. In practice, each methodology follows parallel steps; Design Thinking includes empathy, framing, solutioning, prototyping and testing; Rapid Impact Assessment includes qualitative research, thematic definition, scenario creation, counterfactual creation, and assessment. When these analogous processes come together, Design Thinking embeds greater understanding of users into an evaluation process.
