The Transformative Power of Story-Telling: An Edutainment Guide for Social Change - Step 6: Pre-testing

"...the messages are tested, the characters are tested, the narrative arcs are tested, the scripts are tested and then the final products... This will guarantee that the content we produce carries the messages, that there are no unintended or contradictory messages and that it engages our audiences." - Jair Vega, Research Director, "Breaking the Silence"

This chapter examines the process of pre-testing all materials that are part of an evidence-based entertainment education (EE) strategy to address situations of complex social justice and reconciliation. Pre-testing refers to the process of measuring or assessing what a group makes of a set of messages before they are aired. The aim is to establish the intended audience's reactions to the messages and identify systematically which are the most effective ones and what needs to be modified or improved.

The resource details the process of pre-testing, including via a series of questions to be asked about the EE materials (see page 113). A funnel approach to pre-testing means the initial questions are non-leading, open, and exploratory, and they allow audience responses to emerge spontaneously before content issues are probed and details are sorted out. As noted here, there must be documentation of findings for each pre-testing that goes back to the people in charge of the processes. Not everything that the audience thinks needs to be changed. However, the decision must go through careful analysis to identify which feedback must be incorporated to enhance the production and which comments are incidental.

Case studies include:

1. Colombia: Each key element of Romper el Silencio ("Breaking the Silence") was pre-tested. The methodology was designed by the Universidad del Norte (UNINORTE) with support from Imaginario's research team. The focus group technique was used with the three intended audiences: a group of teens and young people, a group of parents and caregivers, and a group of teachers. After each evaluation, the moderators and observers presented their impressions to the project group. Subsequently, UNINORTE produced a pre-testing report, and results were shared with the creative teams responsible for design and production.
2. South Africa: Heartlines used a vigorous testing approach throughout the project to ensure that the final "Beyond the River" film was authentic and resonated with the key intended audiences, consulting with an organisation called Young and Able for the pre-testing. Heartlines also developed a questionnaire for audiences to complete after viewing the rough cut screening.

Lessons learned from Step 6 experiences:

• Recognise that pre-testing is a characteristic and indispensable process for edutainment that serves to ensure the clarity, relevance, and appeal of each piece of content.
• Ground pre-testing in a methodological approach that has clear objectives and provides the researchers with a deliberative guide to what they are looking for.
• Consider qualitative methods (e.g., focus groups and in-depth interviews) for pre-testing because they allow for a deeper conversation with the intended audiences.
• Carry out pre-testing throughout the entire design and production process to see how the negative comments or disagreements expressed by audiences decrease each time.
• Let people with experience in this type of research perform the validation exercises and data analysis.