The Transformative Power of Story-Telling: An Edutainment Guide for Social Change - Step 9: Monitoring & Evaluation

"Evaluation is the last stage of your Edutainment project. But it should not be the last thing you think about, budget for or plan!"

This chapter explores monitoring and evaluation (M&E) in order to assess and gauge the results and impact of an evidence-based entertainment education (EE) production that addresses situations of complex social justice and reconciliation. M&E can have multiple purposes: to ensure the proper orientation of the process, to gather evidence to make adjustments along the way, to verify achievements and accomplishments, to learn from what has been done, to have evidence to manage new funds, among others. The resource clarifies the difference between the M and the E of M&E: Monitoring focuses on how, when, and where the planned activities are carried out, as well as who executes them and who benefits from them, whereas evaluation focuses on what has been achieved. The resource defines types of evaluations, lists some typical evaluation questions, and describes quantitative and qualitative methods (both types of data are valuable).

As outlined here, the evaluation needs to be done by independent, professional evaluators who are external to your organisation. It is complex to establish a simple relationship between someone's behaviour change and an EE initiative; good evaluators know how to control for these many variables. The type of evaluation to be conducted is determined by the available funds: 8% of a project budget should be allocated for evaluation.

Case studies include:

1. Colombia: At the time of the writing of this manual, Imaginario's Romper el Silencio ("Breaking the Silence") is not at the stage when M&E is in place, though a plan has been developed. With the support of Singizi Consulting Africa, a series of M&E workshops were held to elaborate on the "Breaking the Silence"'s theory of change, based on the findings of the formative research. This exploratory work will inform the television series (with three series in total) and podcasts, whose launch will be accompanied by the introduction of the educational package in schools and Weeks to Break the Silence outside the classroom, designed by youth groups and other collectives.
2. South Africa: Day-to-day project monitoring on the "Beyond the River" project included the development of a project plan, regular project status meetings, stakeholder meetings, and monthly reporting to ensure that project implementation was progressing successfully. In addition, the project team tracked budgets closely. Heartlines also contracted Singizi Consulting Africa to evaluate the success of the #WhatsYourStory? programme, which was implemented from July 2017 to March 2020 in churches and workplaces. (See Related Summaries, below.) Overall, evaluation "findings were very positive and point strongly to the robustness of the theory of change and the positive impact of Beyond the River and #WhatsYourStory? methodology."

• Clearly define your goal from the design stage, as there are several possible methodological designs depending on the type of M&E to be conducted and the objectives that were set.
• Note that M&E requires a methodological design that includes the theory of change, the objectives of the evaluation, the methods and instruments, the audiences that will be consulted, and the manner in which the results will be reported.
• Have a plan to share evaluation results that includes: (i) feedback to stakeholders, (ii) feedback to partners, (iii) publications in academic journals, and (iv) feedback to the team for future projects.