An example of the successful introduction of Outcome Mapping

The case of Vredeseilanden: an example of the use of Outcome mapping

We were delighted to have Christ Vansteenkiste from the Flemish NGO "Vredeseilanden" (VE) present the case of VE. Christ is responsible for the Planning, Learning and Accountability department of VE.  

We invited him to share the story of how and why VE decided to adopt OM. Other questions that were addressed during his presentation were: what have been the successes and challenges until now? What current issues are they dealing with? What is the difference with what they did before? And how has OM affected their relationship with partners, donors and internally? Christ presented an inspiring story of how OM contributes to learning within VE and with partners, and how they have adapted the method to make it work for them.

The participants were impressed by the Christ's confidence in the VE decision to use OM. His story was a realistic and practical one. People asked themselves how it could be that such a big change in the organisation apparently took place without much resistance.

Christ relates this to the fact that OM was introduced at a moment when people were ready for change; they were dissatisfied with the Logical Framework Approach that was used at the time. Besides, VE was working on a new strategy.

An important aspect in the process of applying OM is that VE decided not to start with a few pilots, but introduced the method from the start as an organisation-wide practice. This was in order to avoid confusion.

According to Christ, it was also significant that VE reserved budget to invest in OM experts for each region. Another success factor was that they were clear about their strategic choices when applying OM.

The main adaptations they made in the method were:

• they do not distinguish between 'expect to see, like to see, love to see' in their progress markers. They ask partner organisations to prioritize yearly what progress markers they would like to work on that year.
• they focus on outcome journals and do not use strategy journals. They added a few items from the strategy journal to the outcome journal.
• each objective is treated as a vision
• strategies are only formulated in a very general way, not using the strategy maps as presented in the OM manual

 Group discussion

During the second half of the day, we discussed the use of OM on the basis of questions formulated in groups. Below, the answers given by Christ to various questions from participants.

How can OM be phased in...or... is there a way of "growing it", rather than installing it? 

VE chose not to ask their partners to plan in terms of OM. It is applied to the relationship between VE and the partner organisation, but it is not expected to be applied to the relationship between the partner and their boundary partners. Christ thinks it makes sense to "grow" OM since it is only a method, with all its limitations. The sense making takes place between the people who use it.

What is the difference between the Logical Framework Approach (LFA) and OM?

OM is not meant to measure progress. That is also how progress markers differ from indicators. Although it is possible to reformulate progress markers into indicators, they serve a different purpose. They are important in order to follow change in the relationship and to be able to adapt strategies and practice when (the expected) change does not take place. It could be a pitfall that people using

OM start to use progress markers as indicators. Christ refered to an article on the Logical Framework Approach and OM.

Was there any resistance within VE when introducing OM?

Christ described the situation of VE when OM was introduced. At the head office, as well as the regional offices, there was the will to do something different. Although staff at all levels was familiar with the LFA, and had to be re-trained in OM, they still very soon felt comfortable with the use of progress markers.

How do you deal with aggregation?

 
This is where VE still encounters difficulties. For the annual reports it has not been easy to present a coherent picture of what they are doing and what has been achieved. For donors it is important to hear how the situation of the target group has been improved. Christ explains that they have doubts whether this can best be presented by showing illustrative cases or by drawing an overall picture. However, Christ does observe that OM has helped the process of aggregation since working with 'outcome challenges' has made the information more coherent. VE considers it important to be able to aggregate the lessons learned. In the discussion, participants stressed that quantification is still important not only for the donor, but also for the internal process of an organisation.  

What is the place of a context and power analysis in OM?

Christ explained that the selection of boundary partners is an essential process in which it is important to act on the basis of a thorough context and stakeholders analysis. It is a pitfall to involve only like-minded partners and/or partners that organisations are already working with. It may be necessary and useful to involve boundary partners that you have not been involved with before. Another misunderstanding is that you should always have a financial relationship with boundary partners. Donors are very much focused on this way of thinking. Christ considers the context to be integrated in OM through the Outcome Journals: what internal or external factors hindered or contributed to the process of change?

How to get information on the beneficiary level?

Outcome Mapping does not use indicators. VE uses indicators and stories. They would like to make the use of stories more lively. Up to now the VE country offices produce the stories, but not regularly. They are now experimenting with involving schools of journalism in the story writing. Christ states that it is a misconception to think that the impact level is not important in OM. Impact is the 'guiding star' of everything we do, not a 'measuring stick'.

How are Outcome Journals made?

VE sees Outcome Journals as key in the monitoring process. The idea is that the journals are living documents that are filled in every time you come from the field. VE aims at having at least two reflection meetings every year in which the information from the Outcome Journals is guiding.

How to change to OM?

In a situation heavily concentrated on implementation it is not easy to find the space to change one's  mindset on PM&E. How does one find this space? Christ acknowledges that this might be difficult. For VE it was not difficult because the time was right and people where open to change. VE was already in the process of designing a new program. Christ further mentions that people often do not consider monitoring as a regular organisational function.

However, when done well, monitoring will save time and provide important information as a basis for reflection. VE has made chosen to invest in the shift to OM by investing in regional OM specialists. The challenge is to integrate OM in the relationship between Head Office - regional offices. Furthermore, it remains difficult to find reflective moments.

Do we have to lobby for OM with DGIS?

Both Christ, as well as the group of participants feel that it is quite early to lobby for OM, since most of the organisations do not have the experiences on which to base their recommendation of Outcome Mapping methods. However, it would be good to involve DGIS in similar events in the future.

For those interested in learning more about Outcome Mapping or who wish to partake in a new network, please contact Cristien Temmink, temmink@pso.nl

Why a 'Pit stop'?

The Pit stop on OM was the first of its kind. When starting the session, some time was taken to reflect on  the meaning of the term 'pit stop': a chance to re-fuel and to do maintenance where needed. The person who joins the pit stop is in the driver's seat.

This pit stop was part of a series of sessions about qualitative and participatory methods of PM&E organised by PSO. The first session took place in April 2008 and was on learning from innovation.The second one took place in October 2008 in collaboration with the Institute for Research & Evaluation in International Cooperation. 

For 2009, PSO has planned two introductory workshops: one on Outcome Mapping (OM), that took place on the 26th and 27th of May 2009 (see general impressions) and one on Most Significant Change (see invitation), that will take place in September 2009.