Reflections on 2020 and the Last Stretch of EW2
Happy New Year! As we move into the last stretch of Experimentation Works 2 (EW2), we thought we’d share some reflections. This is normally a good opportunity to take stock and assess what is working, what isn’t, and adjust as needed: the same principles that underpin experiments. This year, like everyone else, our project teams and all those supporting them have already had to recalibrate, adjust, and evolve on the fly due to the pandemic.
Maintaining momentum amidst change
So, where are we? Since May, we have connected with the full EW2 Cohort through a total of 21 virtual events. These events included monthly cohort check-ins, project presentations, and monthly learning events. We also launched a fall series focused on reviewing early modules of the experimentation course being developed by EW2 experts and our team. Our learning event topics ranged from design choice, GBA+, ethics and featured experiments (materials accessible here). The course material will also be available in the coming months.
Several of our projects had to withdraw due to changed priorities. That said, a full 9 are making progress, though many with changes to scope or timing. We still have over 30 experts involved in a variety of ways. These experts have been instrumental in supporting projects, hosting learning events, and helping us develop the experimentation course mentioned above.
Making our materials publicly accessible
We’ve also posted all EW-related materials on a new publicly-accessible GCWiki page. This resource provides an overview of EW as well as detailed links and resources for both Cohorts (EW1: 2018–2019; EW2: 2020–2021). Our GCWiki also includes a link to our new infographic, with a few screenshots below:
Pain points and challenges
Running EW2 has given us a unique perspective into the experimentation journey. This includes seeing common pain points and challenges. For example, exploratory work is essential in setting the groundwork for a successful experiment. And this can take a lot more time than anticipated. Also, experimentation can sometimes be confused with public opinion research. If definitions between these two things are not clear, teams can spend more time having conversations about this than is ideal. Lastly, teams seeking to contract research and experimentation expertise would benefit from more support and examples of success in this space. This includes the need for newer research methods, such as ethnography, to be recognized and supported.
How are we supporting these challenges? EW2 was always designed to support pre-experimental research as well as experimental research. As such, our model is flexible by design. We can easily pivot our resources to support and showcase the pre-experimental work as much as the experimental. To help reduce the confusion around public opinion research, we also worked with TBS colleagues to update the definition of public opinion research on Canada.ca. This definition now includes experimental research and what distinguishes it from public opinion research.
Experimentation and Covid-19
A final thought: the Covid-19 pandemic has demonstrated the value of the experimental approach. At least two of our EW2 experts have been re-deployed to support an evidence-based response to the pandemic. All 9 project teams are building skills and gaining experience that will support future research and experimental approaches in government. Together, we are learning how to collaborate across departmental boundaries and use our cross-functional diversity of skill-sets to tackle complex problems. The pandemic has also shown us a wide range of issues that will require research and evidence to address. To respond, the public service will need to continue to rapidly evolve and find new ways of working. This means there is no shortage of areas in need of rigorous testing to see what works!
Keep reading and stay tuned!
As we go into the final months of Experimentation Works 2, we’re also thinking about what’s next for EW and how we can keep supporting experimentation across the public service. We’ll have more to share on this soon. In the meantime, be sure to read the past and upcoming posts on individual projects. They each have their own story to share!
* * *
Post by the TBS Experimentation Team (Nick Chesterley, Pierre-Olivier Bédard, Sarah Chan, Alana Couvrette)
Article également disponible en français ici : Réflexions sur 2020 et le dernier tronçon d’EO2 | par L’expérimentation à l’œuvre| Février, 2021 | Medium
