How to optimize the collective impact of a devolved initiative using a collaborative formative evaluation process
Some initiatives consist of a central agency initiating an initiative that is made up of a number of different parties working at different sites while undertaking similar activity (a devolved initiative).
Formative evaluation (an aspect of implementation evaluation) is a type of evaluation aimed at optimizing initiative implementation. It is done by a formative evaluator, or team, working alongside an initiative playing the role of a critical friend to the initiative.
A formative evaluator working as a critical friend critiques an initiative, but at the same time offers to assist the initiative to improve the things that the evaluator(s) has identified as problems.
A formative evaluator’s relationship to an initiative is different from the traditional hands-off relationship of external evaluators to the initiative they are evaluating. Ideally, you one evaluator, or team of evaluators, works as formative evaluators, while a different evaluator, or team, works on outcome or impact evaluation (measuring the impact of an initiative in terms of achieving its outcomes). This is desirable because a formative evaluator can ultimately end up being so involved with an initiative that it is hard for them to view it independently.
The Collaborative Ongoing Formative Evaluation COFE process is a formative evaluation technique specifically developed for devolved initiatives. It is based on the Group Action Planning approach to collaborative planning for devolved initiatives.
The traditional approach for implementing a devolved initiative - one where similar activity is taking place at a number of sites - is for a central agency to have many one-on-one interactions with the different parties implementing the initiative at different sites.
The COFE process takes a more collaborative approach. In the COFE approach, at the start of the developed initiative, a workshop is held where the central agency meets with those who are going to implement the initiative at different sites. The group then meets regularly over the course of the initiative. The philosophy is similar to the Collective Impact approach. It is one where the task at hand is a shared task between the central agency and all of those who are implementing the initiative at different sites.
The advantages of the COFE process are:
- It uses the energy of committed enthusiasts who just want to get things done and builds off peer-to-peer interaction between parties implementing the initiative at different sites.
- The presence of the formative evaluator changes the dynamic that can exist between central agencies and those implementing an initiative at different sites.
- The formative evaluator can play the role of injecting information about what works into the discussions.
- Subgroups of those implementing the initiative can come together to work on specific tasks that will benefit those working at many different sites.
- The COFE process is self-documenting in that the report-backs to each meeting from those implementing the initiative at different sites can be captured. This information can then be used to describe the process of how the initiative is being implemented.
The COFE process tightly embeds planning within action. It’s agile planning collective impact planning that gets people to roll up their sleeves and ‘just do it’. It combines a collective impact approach with the benefits of formative evaluation.
Here’s how it is run:
Bring them together for an initial meeting. Tell them that they're going to be meeting every three (or six months) either virtually or physically until the issue is fixed (i.e. over several years).
Identify a small group of key dynamic people working at different sites or in different organizations. Don't just bring in figure-head organizational leaders. The criteria is that group members should be motivated and impatient for change.
Have the meetings independently facilitated by a formative evaluator who is familiar with the COFE process. This changes the dynamic from a central agency 'telling' people what to do to a group working collectively to solve an issue.
Throughout the process, the formative evaluator and others provide input about the best way of addressing the issue(s).
This input might, for instance, provide evidence to inform evidence-based practice; information on legal requirements the group needs to take into account; or it might be best practice suggestions.
At the first COFE meeting, identify the outcomes the group is trying to achieve and the steps required to achieve these. Prioritize where the group, and each member, will focus in the period until the next meeting.
This can be done by group members assigning themselves to work on specific issues either in their own organization or site, or in conjunction with other group members on issues that could benefit by members of the group working together.
At each meeting, allow time for group members to share the best-practice approaches they're using in their organizations or at their sites.
Encourage them to set up working groups to address issues that need collaborative work.
Identify any system-level issues they're facing in their organizations or at their sites that need to be addressed collectively.
Make representations from the COFE group to external stakeholders who can improve these system-level issues.
Start each COFE meeting by members reporting back on the progress they've made in their organization or at their site.
Keep a record of these reports as documentation of the progress that's being made as part of documenting, monitoring, evaluating and improving the work of the group.
Get COFE group members involved in working out how they're going to monitor and evaluate their progress.
Work out how to monitor and evaluate overall progress on the issue.
Encourage electronic networking between meetings so that COFE members can share best-practice; keep motivation and morale up; and identify any issues that need to be addressed during the next stage of action.
FC2018