ESDC’s Experimentation Journey with Grants and Contributions
Since the 2016 introduction of the Experimentation Direction, the field of experimentation has been steadily growing across the federal government. While not new for Employment and Social Development Canada (ESDC), the last few months have seen a major shift towards experimentation in the Program Operations Branch (POB) with the creation of the Research and Innovation Team (R&I) and the launch of two experimentation projects. One experiment led by R&I on Incomplete Applications, and another led by the Youth Employment and Skills Strategy (YESS) on youth facing barriers accessing employment programs.
In September 2020, I joined POB as a student with the R&I team. Being new to the public sector, I find myself intrigued by how experimentation is being applied to program delivery and have learned that it is a new and under-explored field for Grants and Contributions (Gs&Cs). The work being done in the team and the Branch helps build the first steps towards a culture of evidence-based decision-making through a human-centered lens. To help me understand and explain this work, I have conducted five interviews over the last several weeks with public servants linked to R&I’s work on Incomplete Applications. I have decided to focus on this project since my student placement is with the R&I team. You’ll read the thoughtful processes behind the people directly involved in this project, as well as from others playing key leadership roles.
What is Experimentation? Not just Scientists in a Lab!
Experimentation is able to take old ways of looking at something and make room for change and innovation. Speaking to the specifics of experimentation with respect to this process, Eric Beauchamp, the project lead and Senior Business Advisor in the R&I team, finds, “it’s mostly about culture. It’s about culture and it’s about strong, informed decision-making. This project is really important because it’s a part of a broader trend currently in government towards evidence-based decision making, towards using tools like research, data and data analysis, like experimentation, which means actually testing and validating innovations and improvements before scaling them and implementing at scale.” Experimentation takes the traditional way of doing things, be it in government or elsewhere, and challenges it. It challenges our thinking in assuming we have the answers; it allows us to explore different avenues of change. This change is the culture shift that is needed particularly in large organizations such as government to get them to modernize.
TBS Definition of Experimentation
“Activities which seek to first explore, then test and compare, the effects and impacts of policies and interventions in order to inform evidence-based decision-making and improve outcomes for Canadians, by learning what works and what doesn’t. Experimentation is related to, but distinct from innovation (the trying of new things), because it involves a rigorous comparison of results. For example, using a new website to communicate with Canadians could be an innovation, while systematically testing that new website against existing outreach tools (or even an old website) to see which one leads to more engagement is an experiment.”
Experimentation also allows us to rigorously test the impact of our possible options before we implement them at scale. Speaking about this, Farhia Ahmed, Manager of the R&I team, states, “there is one thing that I would like to highlight to our colleagues across the Branch who may not know about experimentation: just like you would in a lab, when you try to do something differently, you first see and study where the problem comes from and what previous iterations of solutions looked like. What were the reactions? What was the outcome? This allows you to really concretely move forward with your theory, and then investigate further. The entire notion of experimentation is taking many scientific concepts and applying it to a process that many would not see as a scientific process. And it really is.”
Experimentation challenges traditional ways of thinking. It allows room to try new things and possibly fail forward. Speaking to this concept of failing forward, Ahmed further mentions, “through experimentation, we go through the motions of problem, to idea, to solution. We make sure every step of the way is accounted for and inventive. The idea is that, if we give ourselves room to challenge ourselves, and how we have been doing things, then we have more room to grow and expand our thinking. We might fail, and we might fail again, but every time we will learn from what went wrong and why. Failure is only an opportunity to see what you can do right next time.”
Erica Vezeau, the project’s sponsor and the Director of Integrated Planning and Business Analysis, discussed the concept of modernization, “[it] just brings us into a more modern space as a public service, and it brings us closer to our objectives of exploring, doing better, being client focused, and data driven.” As our society changes, experimentation allows its public service to explore and implement what works not only to make public servants work more effectively, but to get to better services to Canadians.
A Project is Born — An Experiment on Incomplete Applications
Established in early 2020, the Research and Innovation (R&I) team has a mandate to conduct research focusing on a variety of subjects ranging from client experience, to barriers for populations in vulnerable situations, to program processes and outcomes. One of the team’s landmark projects is the design and delivery of an experiment in collaboration with the Treasury Board Secretariat (TBS) through their Experimentation Works 2 (EW2). This initiative is designed to build the capacity of the public service to experiment and displays a variety of projects where selected departments run experiments while paired with experimental design experts for the duration of their project.
As part of this project, the R&I team is researching why some grants and contributions programs within our Branch receive a high number of incomplete applications (for example, if they are missing mandatory documents or key information) resulting in delays in providing a decision to applicants and an increased workload for programs. This ultimately reduces the quality and speed of the services provided to Canadians.
The project researches and measures elements of the application experience and how they affect completion rates of Gs&Cs applications. As the project progresses, the team has demonstrated creativity through the new collaborative online tools they are using (for example, Miro, an online whiteboard service), as well as through novel research tools (for instance, Zotero) or through the creation of a common library of resources to be used for training purposes.
Through this project, the R&I team will also increase the Branch’s experimental capacity, and broaden its understanding of a key part of our interactions with Canadians, namely their applications for grants and contributions funding. Ultimately, it will allow us to better understand our clients’ behaviours and be more user-centric in future strategies and decision-making.
Building our Capacity
The Incomplete Applications experiment is considered a trial run. In this context, it is especially beneficial to those in the public service interacting with it. It allows us to learn and grow, and as Vezeau states, “lay the foundation” for future projects. It provides a clear example for not only our team going forward, but also to anyone who hears of our involvement with this new perspective towards experimentation.
Experimentation is also part of a more inclusive, diverse government that strives to, as Reem El-Attar who works on the literature review for the R&I team, puts it, “[it] helps build our capacity and foster a culture of learning and innovation.” The unique nature of experimentation will invite new ideas. It allows for more than one way of thinking, which in turn, means more than one way of doing. It gives organizations and clients room to get services the way they need them, and the way they want them. It eliminates single-minded, single-way of doing things and makes room for a multitude of solutions.
Pierre-Olivier Bédard, the project’s assigned expert with the TBS experimentation team, builds on this as he mentions that experimentation often requires a diversity of skillsets and opinions. He says, “I feel like this project has a really well-rounded team and a really good mix of people working to make it progress and I think that’s really key. Seeing that we have subject matter experts, you have some experimental design experts; you have a whole range of people that all contribute something to the project. Having a multi-disciplinary team is key as well.” Without a doubt, an experimental project not only takes diversity in method, but also takes a diverse group of skills to execute it. With employees from all different professional and educational backgrounds, we hear different opinions and recommendations that we might not have heard otherwise.
Design by Humans for Humans
Beauchamp has been involved in program design for a number of years. He says, “I’m a big fan of human centered design, of design thinking, of these new trends that are coming in where the businesses look beyond the processes and deliverables to understanding the human emotions, the human drivers of behaviours in play.” In the R&I team, it is especially important to us to consider multiple perspectives. We utilize experimentation to understand all aspects of the exchange of services we provide, in order to better our execution when we provide these services.
Speaking about the client’s satisfaction as the end goal, El-Attar explains that, “the literature review [for the Incomplete Applications experiment] could uncover some solutions that could help the organizations that we serve. Especially those that serve vulnerable groups and have a limited capacity.” We are learning that every organization is different and not all will need the same type of help. Experimentation allows us to try new things and centre on the human in our service design processes. In our case, this is especially important for vulnerable populations such as seniors who may not want to, or who may not be able to access funding through only digital channels of communication.
Vezeau notes that “the most key element of this project … is [that it is] laying the foundation for us to think differently in this Branch about the way that we do our work, and the way that we deliver these programs, and the way that we design these programs. If we have success on that level, then we’ll see more projects like this. And even if this project doesn’t have a direct or measurable impact on clients, on Canadians, then maybe one of our next ones will. But we’ll never get to the next ones until we get through this one.” Our human-centred service design sets an example that things can be done differently, that processes can be designed for humans by humans.
What’s Next?
Especially given the current pandemic situation, our public services need to connect and listen to Canadians in order to better serve them. This is what makes our contributions so valuable. In hearing from those directly involved, the Incomplete Applications experiment has many facets and diverse voices contributing to it. These diverse voices are what drives the project and allows innovation.
Every person interviewed involved with this project has a unique perspective on what they see as most beneficial about it. There are multiple lessons to be learned from experimentation and these lessons are important on the road to more modern government. As mentioned throughout this article, EW2 allows for collaboration, exploration and room to fail forward. What is our future vision for our Branch in this domain? The Branch is excited about experimentation and, in future years, it intends to expand the work to include other innovative approaches to program design and explore what the future of grants and contribution programming could look like. We are testing new ways of working and leaving room to learn — and that is what experimentation is all about.
* * *
Post by the Research and Innovation EW2 project team (Emma Peters, Eric Beauchamp, Farhia Ahmed), Program Operations Branch, ESDC.
Article également disponible en français ici: Parcours d’expérimentation d’EDSC avec les subventions et contributions | par L’expérimentation à l’œuvre | Décembre, 2021 | Medium
