PSO members learn about the ‘Most Significant Change’ or ‘story’ approach

Learning events

In the framework of the meetings PSO facilitated in 2009 pertaining to qualitative methods for PM&E, two sessions took place on 29 September and 1 October 2009 concerning the ‘Most Significant Change’ (MSC) technique. This MSC technique was developed a number of years ago by Rick Davies to be able to monitor social change.

The technique’s principal characteristic is the collection of stories of meaningful changes people have experienced (the most significant change stories) and the systematic selection of the most meaningful story involving the various actors.

The meetings aimed to acquaint organisations with the technique and to let them consider whether application could be of interest to them. On the first day, four of WASTE’s southern partners were present. A number of them already used MSC, other wished to know more about the method. Thanks to the joint participation in the meeting WASTE expects to be better able to make a decision about how to jointly utilise MSC.

War Child had also experimented with MSC before. Endry van den Berg took part in the PSO event to refresh and update her knowledge, and to share her experiences. Other participants indicated that they had attended to see whether this method would be suitable for their organisation and their work. A southern participant humorously indicated that he/she attended 'because this training was planned in our Logframe'.

The ‘Pit stop’

The ‘Pit stop’ meeting on 13 October 2009 in Madurodam was the follow-up to the meetings held on 29 September and 1 October. 'The ‘Pit stop’ formula offers participants the opportunity to study the method in more detail,' says Cristien Temmink, PSO Facilitator of Learning. 'The learning days are therefore not a goal in and of themselves; this is only the beginning. Our ultimate focus is on the application of methods and the joint learning of these in daily practice.'

This characterises PSO’s approach according to Cristien. 'Ultimately we want to be able to compare the experiences gained in practice together with our members and southern partners to be able to find the lessons learned and to share these with the entire national and international development sector.'

Issues pertaining to MSC

The ‘Pit stop’ centred on a number of important issues with regard to the use of MSC which participants had identified during the prior MSC sessions. These included:

• The importance of clearly formulating why MSC has been selected (the objective in mind) before actually starting to use it. Does the organisation wish to use MSC for accountability purposes or to stimulate learning, or both? This determines, to a great extent, how the method can be applied.

• It should be properly ascertained how MSC can be incorporated into the existing PM&E system. The link with planning and the combination with any possible other methods is important.

• Good facilitation and a conscious choice of internal or external facilitation for the method is crucial to the process.

• Time, human resources and funding investment in relation to the potential yield.

• The importance of the selection process: who tells the stories (whose stories are told)? How do we select the stories? Who select the stories? Balances of power at any level can play a role and influence the process.

• How will the stories, and the selection process and the choices made be recorded (in writing, on video or using sound recordings)? Who will do this; which capacities are required to this end?

• Feedback loop: when and how long after the stories have been told will feedback be provided? The other question being: who will provide feedback and concerning which aspects?

• How many domains of change will be examined? Will these be laid down in advance or afterwards on the basis of the substance of the stories?

Sri Lanka and Afghanistan

The ‘Pit stop’ meeting started with presentations by Wim Hiemstra of the ETC Foundation and Endry van den Berg of War Child Nederland. Wim presented the case: ‘ETC COMPAS: Collecting quality of life change stories’. This case concerns ETC’s experiences with the MSC technique in the COMPAS (Comparing and Supporting Endogenous Development) programme in Sri Lanka. The project focuses on organic farming, building on local means, world views and local institutions.

ETC cooperates with three local NGOs led by a network coordinator. The latter was also closely involved in preparing the presentation provided at the ‘Pit stop’. ETC decided to use the MSC technique because the indicators formulated within the Logical Framework Approach to monitor social change and spiritual empowerment provided insufficient insight. Using MSC it proved possible to demonstrate the improvements in the farmers’ wellbeing from the perspective of the various beliefs (Buddhist, Hindu and Christian) that occur within the communities.

In 2009, field promoters of the three organisations collected 154 stories across gender (men and women) and generation (young, adult, old). Working with MSC also led to unexpected results: 'The stories proved to motivate young farmers to focus on organic farming,’ Wim explained. 'Their vision on the world and on agriculture became very apparent thanks to the stories.'

Wim also indicated that the application of the MSC technique needed time for the field promoters to probe deeper into the rural people's quality of life changes.  'Documenting the stories of the members of the community was a learning process in itself.' However, the time invested in this was not perceived as a problem by those involved. Both the programme’s coordinators, the storytellers and the partner organisations’ field promoters experienced the process and the application of MSC as empowering. 'We have noticed that working with MSC clearly provides results,' concludes Wim.

Challenges ETC still faces include how to best present the MSC stories to donors. 'Summarising them or converting them into figures robs their spirit; the strength lies in the stories themselves.'

Endry van den Berg provided insight into War Child Nederland’s experiences with MSC in Afghanistan. War Child used the MSC technique for the mid-term review of a project related to the welfare of Afghani children.

The objective was to provide insight into the (un)expected results. The MSC technique was combined with other evaluation methods such as interviews, observation and creative sessions. The principal target group were children. They were asked to make a drawing and to tell a story based on it. The evaluation took three weeks; this proved to make it difficult to properly apply the MSC technique. For example, there was limited time to train staff and to coach them in facilitation and interview skills.

But the use of the MSC technique in the review in Afghanistan definitely provided results. It gave a general impression of which changes had been brought about by the project. The MSC technique also enabled insight to be obtained in unexpected or even negative changes. It is important to incorporate these in a review. Furthermore, the stories generated many interesting quotes for the report.

Exchange

After the presentations, the twelve participants – PSO, Niza, WASTE, War Child Nederland, Oxfam Novib, ICCO en Kerk in Actie, Mensen met een Missie, IRC, Cordaid and the ETC Foundation staff – were subdivided into two groups. The smaller groups discussed the examples suggested by the presentations. The participants were also asked to view the cases from the perspective of the issues surrounding MSC, which had been formulated earlier. A number of questions and comments arose from the group discussions:

- Margreet Houdijk (Cordaid) wanted more insight into how the MSC technique works if it is applied to a heterogeneous group. She hereby referred to a programme which brought together local, regional and national women’s groups. The women have varying levels of education, ethnic backgrounds, political ideas and interests. The group arrived at the conclusion that the stories can help to explain the women’s ideas and interests to one another. This helps them foster understanding for each others’ points of view. An important question remains who is to tell the stories and how these are to be selected. Consciously dealing with balances of power is a challenge in this.

- It has been observed that when using MSC, the stimulation of learning at partner organisations and between partner organisations should have priority. The process should be structured to this end.

- Once again, the importance of the question how MSC suits what we do in relation to PM&E has arisen. When deciding whether or not to use MSC for M&E it is good to do so in relation to other methods already in use. Which information is collected using those other M&E methods? Which existing information need does MSC fulfil? MSC is a method which has to be combined with other M&E methods.

- How do you prevent MSC stories being viewed as success stories? The group agreed that negative experiences are just as important to learning. How do you ensure that negative stories also gain attention and can be used? You do not want to enter them as a separate domain because this gives rise to the risk of them becoming a file of lost cases which have no other substantive linkage. The conclusion was that the quality of the facilitator is crucial. Facilitating the interview correctly, enables negative experiences to also arise as meaningful stories.

- If we examine the MSC stories which a process gives rise to, it is hard to determine what a story’s value is. The latter has a direct relation to the process’ quality.

What's next?

The ‘Pit stop’ was concluded with the agreement to continue to exchange our experiences with the use of the MSC technique. This is part of the learning process concerning the value of the use of qualitative methods facilitated by PSO. Wim Hiemstra raised the idea to involve a number of organisations in the shaping of the MSC process within the COMPAS programme. Dialogue will take place with COMPAS partners concerning this during the first quarter of 2010.