CRC Advocacy through Radio-Television Programs for Children and Youth – End of Project Evaluation

Background   
The ‘CRC Advocacy through radio and TV and youth for children and youth’ project is implemented as part of Save the Children’s Child Rights Governance Programme in Laos. The project is the second phase of support for the Lao Youth Union’s (LYU) mass media department to run a child and youth-led radio and TV programme on child rights since 2007. The first phase of the project was implemented with support from Save the Children Norway (SCN) from 2007 – 2009 and a new phase started in 2010, following an assessment workshop to expand and further develop the radio and TV activities of the LYU. The project has been managed by Save the Children International (SCI) since transition to SCI in September 2011. There has not been a formal evaluation conducted before, only an assessment workshop conducted in 2009 to develop a second phase of the programme. Therefore the first purpose of this evaluation is to assess and document the progress to date of SC’s support to the LYU TV and radio activities. This should also include the development of a Timeline to document the history of the project and SC support. In addition, as this project is now coming to the end of a second phase (with the MoU ending in December 2012), a new project phase will be developed and so therefore this requires some key recommendations for the next  phase of the project.

Purpose/objective (including evaluation questions)   
• Assess the Efficiency and Effectiveness of the project, reviewing achievements against planned objectives and outputs
• Assess the direct management of the project by LYU and the indirect management and technical support role by SC, including analysis of levels of understanding and capacity, challenges and lessons learnt
• Assess the intended and unintended Impact of the project, using Save the Children’s Theory of Change
• Assess the Relevance and Sustainability of the project model for the next phase of the programme, developing key recommendations for the next phase.
• Document the historical timeline of SC’s support for the LYU TV and radio programme

Methodology   
Desk review of project records and reports to gather data relevant to outputs, and field work including participatory exercises to collect data on the project outcomes. Field work used a combination of participatory research tools and methods to collect primary information from a sample of stakeholders, both linked and not-linked to the project (i.e. semi-structured interviews, focus group discussions, paying attention to specific ethnic, gender and cultural contexts) and qualitative analysis.
The participation of children and youth in the evaluation was essential and condultations were conducted with both children and youth.

Key findings   
 The overall objectives of the project have been met although for future planning it is advised that both parties (LYU and SC) develop clearer objectives and more tangible outcomes that are clear, realistic and achievable. With support from SC, and Lao Youth Union the volunteer DJs have undergone trainings and workshop in the areas of child rights, child protection, and child participation.  Lao National Radio has provided technical support to the DJS in the areas of interviewing, storytelling, reporting, and overall general child-youth radio skills.  From the Evaluators interviews, observations, and sampling of content, it is clear that the volunteer child/youth DJs’ have the skills to design and produce media, with adult support, for children and young people. When talking to the DJs it was very clear that they were eager and happy to participate in this project.
1. Overall Management of the Project (including technical support)
Save the Children: The overall management of the project from 2007-2012 has been generally strong. SC has strengthen the CRG staff capacity, by sending them to media (mainly radio) and child trainings, workshops and study visits. However, due to the transition to SCI between 2010-2011 the CRG section has seen a large turn over off staff who have been responsible for this project resulting in a new CRG team downsized from having two project officers to now only one. The new team has a comprehensive understanding of the project and overall objectives, but lack the technical knowledge of radio and TV to fully support the project, as well as the means to properly monitor the programming of radio and TV programs.  Over the project period there has varying levels of monitoring of the project.
Lao Youth Union
Like the SC management, the Lao Youth Union management has been overall strong in this project. Within the Lao Youth Union the Department of Mass Media is the responsible for project activities. The project receives technical support for editing and broadcasting from the Lao National Radio and the Department of Pioneers help coordinate volunteer children DJs. Central LYU staff coordinate with LYU Provincial and District who support the Listener Groups, an extension of the LYU network.  Project reports produced by the LYU which are submitted to Save the Children on a quarterly, six months and annual basis lacks details of child rights content. 

2. Content of the Radio Programme
The length of the evaluation did not allow for a full review of six years worth of radio content. However, it can be confirmed, through project documentation and sampling of scripts, that relevant child right messages, addressing the four pillars of rights, have been incorporated into the media produced the LYU.  Child rights are introduced into the children’s radio programs through CRC stories and songs. 

3. Relevance
SC has played an important role in feeding child rights related content to the LYU as well as assisting with an action plan to increase child/youth participation in the design and delivery of the LYU Radio and TV.  Overall the project objectives have a high relevance in country in which there is a still limitation to children and youth participation as well as for a media environment which has minimal programming for children and youth and content relating to the rights and protection of children. As part of the follow-up audience survey or new audience survey it would be important to ask key questions about problems facing children and youth, and their interests, and then aggregate this data by gender, ethnicity, geographic locations in order to create media programs that target specific populations.

4. Sustainability
The project has provided technical and financial support which has allowed the LYU to open its own “doors” in the areas of child/youth participation and child rights related content.  Children and youth involved in LYU media production now have the capacity to actively participate in the making and presenting of radio and television. LYU management has a better understanding on how to create more spaces for participation for children and children. Content, of the media produced by LYU, now incorporates child rights related messages or themes.

The evaluation found that if funding and technical support to this project were to stop then there would be a dip in child rights content in LYU radio and TV.

Recommendations   
Operations
MoU & Logical framework (SC & LYU) – when developing a 3rd phase of this project, it is essential that the two partners agree on clear, realistic and achievable specific objectives, outcomes and indicators. In addition, it is advised that a monitoring and evaluation tools/ including an M&E Plan is developed.
Communication Plan –SC and LYU develop a separate communication plans for Radio and Television. Development of a Media Data Base – Could be simple Microsoft Access database that could be populated with information such as previous content topics, and feedback from listeners, enabling aggregation and analysis.
Capture what is being produced – SC has failed to pull out key examples of creative content for radio and just now in the format of video/television which could be used for donor reporting or additional fundraising. Also suggested that the budget includes specific budget for English dubbing of production or translation of scripts.
Equipment –The Evaluator suggests SC and LYU single out Listener groups who are actively providing the field reports and provide the group with a digital recorder and training on how to use the. 
Between 2010 and 2011 the LYU received a new Apple Desk top computer, worth between 1,200-1,700 USD, for the designing, editing and production of LYU TV. During the interviews it was made clear that the equipment that was received is very good, but was not used because the mass media staff needed training on how to use the computer and also associated software. At present they still use an old but functional PC computer to edit and produce TV programs.
Third party partnerships – if the LYU decides to partnership with corporate companies to further finance LYU radio and TV, and SC continues to support as well, it would be vital that SC supports this process to ensure these private partnerships are with child-friendly companies as well as to ensure accountability and transparency.
Program
Follow-up Audience Survey – SC should provide support to the LYU to conduct a follow-up to the 2008 Baseline Audience Survey. The follow-up or even a new survey would provide current data on listeners preference that can be used to develop content and formats that children and youth people like. It is also recommended that listener groups and LYU Provincial staff are given further training in data collection so that more frequent “mini’ surveys can be conducted throughout the project period. 
Moving Radio support to the Provincial Level: It is recommended that SC and LYU discuss the possibilities of moving radio support to the provincial level-either via the Provincial LYU or the Provincial radio stations, opening up the opportunity to support child rights radio programming in ethnic languages. It is recommend to in order to reach the most marginalized children and communities that SC explores community based radio program.
Skill up CRG team – the new CRG team is strong with a large amount of potential to move SC CRG initiatives forward. However, in regards to support to any future radio and TV activities it is recommended that SC invest in providing the CRG with necessary training and understanding on how to support these activities.
SCI Communications
The Evaluator recommends SC Country Program Office to further review Communications as well as the use of radio in its overall operations and program work. It is fair to say that the program office has an almost non-existent communication department, and are only project specific. 
Some areas to considered:
• Radio and Television can be used by the country program to advocate broader SC messages and work
• Explore developing a communication unit that would work across sectors (Health, Education, Child Protection, Emergencies, Disaster Risk Reduction, Primary Health Care and Child Rights Governance).
• Insuring that future grants have a realistic communication budget line.
- Explore the many innovative ways to use Media in program work (radio modules as follow-up support to new preschool teachers; support to the PHC program – tip and reminders for mothers with newborns; compliant mechanism for SC programs/call-in system/especially for Emergency Response; radio could be used as means to inform communities of the work that has been done SC and its partners, or work that is being planned; Consider sustainable Education Entertainment; look at other partners for radio and television- e.g UNICEF

Follow up (with reference to Action Plan) 
A new project phase will be developed in March 2013, in partnership with the LYU and other relevant media partners. The baseline audience survery, which was conducted in 2009 will be repeated as part of the design of a new phase, in order to assess and develop new content for the radio and TV programmes.