UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.
Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.
And we never give up.
For every child, evaluate
The Evaluation Office seeks to issue a contract to eligible consultants to be part of a 5-person core team that will conduct an evaluation of UNICEF work in early childhood development (ECD) and early childhood education (ECE). The evaluation aims to interrogate whether UNICEF-supported ECD/ECE programmes for early stimulation and learning do achieve intended developmental and school readiness outcomes. The evaluation will assess whether there is clarity in UNICEF country offices about what constitutes inclusive and effective learning programmes for caregivers and children in the early years – from birth to the pre-school year (pre-Grade 1 year). The evaluation will assess whether the choice of an integrated ECD approach has resulted in the intended cohesiveness, efficiencies, and complementarities within UNICEF goal areas, and between UNICEF and partners in the ECD/ECE subsector, to provide holistic developmental support to young children and their families. It will also interrogate, whether UNICEF-supported programmes prepare ECD/ECE pre-school centers and schools seek out and engage with the most vulnerable populations of young learners and their caregivers.
The evaluation will focus on ECD and ECE programmes that address the policy development, system strengthening and service delivery objectives, and contribute to children’s developmental readiness and school readiness for children from 0 to 8 years. The five components of the Nurturing Care Framework will be used to interrogate the content of the programmes, and to assess whether there is clarity in UNICEF country offices about what constitutes an inclusive and effective early learning programmes for caregivers and children to achieve the intended developmental outcomes. On the other hand, the evaluation will look to the Build to Last framework for the ‘how to’ – i.e., to tease out the role of the education sector and key components that are necessary to prepare ECD/ECE centers and schools to engage young learners and their caregivers. The evaluation draft theory of change, provided in Annex 2 of the attached terms of reference, attempts to bring these two aspects together. However, more inputs will be required to update the draft theory of change and to provide an exhaustive catalogue of UNICEF-supported programmes, interventions, and/or activities during the inception phase, while the data collection phase will be used to update the theory of change further against the workplans from the countries that will be selected to participate in the evaluation, and to validate it for use in making evaluative judgments.
While ECD was previously viewed as one of many interventions, programmes, or even as a sub-sector in many organizations, there is now wider recognition of early childhood development as the intended outcome of the work that governments, national stakeholders, UNICEF, and the development community do in the ECD and the ECE sub-sector. Put differently, positioning children to reach their developmental potential in the early years is identified as a key ingredient for enhancing the achievement of almost every child outcome that we can think about.
Within UNICEF, notable developments have taken place, key of which was the increased visibility of ECD due to being championed by the Executive Director during the period of UNICEF Strategic Plan (2018-2021). This had a positive effect on the early learning agenda. Also, UNICEF has since articulated significant conceptual frameworks around which the ECD and ECE work is organized, namely the Nurturing Care Framework in 2018 and the launching in 2019 of the Build to Last framework, to support governments in strengthening systems to provide pre-primary education. At best, an examination of these investments is necessary, both through formative assessments of the more recent commitments, and summative assessments of the work that was done from 2018 (underpinned by approved strategic plans, results, operational modalities and staff structures) for accountability purposes. This evaluation contributes towards that effort.
How can you make a difference?
EO seeks to issue a contract to eligible consultants to be part of a 5-person core team that will conduct an evaluation of UNICEF work in early childhood development (ECD) and early childhood education (ECE). You are invited to apply, to join the evaluation team as Team Leader.
The purpose of the evaluation is twofold – to enhance organizational learning and to strengthen organizational accountability. On the learning side, the evaluation will interrogate whether UNICEF has leveraged the organization’s investments on the integrated ECD approach and used its knowledge resources and experiences to support countries in strengthening their ECD and ECE programmes. The evaluation will also advance the accountability objectives of the organization by interrogating and/or verifying UNICEF achievement of reported results, aimed to increase access to early childhood education opportunities and the care that supports learning. Evidence from the evaluation will be used to facilitate decision-making, advocacy, and resource mobilization and to contribute to global learning within the ECD and the ECE sub-sector.
The evaluation will pursue three specific objectives, namely:
The evaluation will occur in four phases (i) inception; (ii) desk-based document review and analysis; (iii) field-based data collection; and, (iv) data analysis, reporting and communication of evaluation results. Each of the stages is described below.
Phase 1-Inception: The inception face will feature five activities, as described below:
Phase 2-Desk-based document review and country self-assessments: This phase will be data collection from secondary sources, featuring three activities:
Comprehensive search and compilation of documents from 15 country offices selected for the desk-based document review: Continuing from the search described in the inception phase, relevant UNICEF documentation from the global and regional levels, will be collected. These will include work plans, corporate policies, strategies, programme guidance, position papers, inter-agency and global education cluster documents, studies, reviews and evaluations from UNICEF and other organizations. From the country offices which will be a focus of the desk-based review, the team will collect planning and programming documents; situation reports (SitRep); needs assessments; surveys and statistics; donor proposals and reports; field monitoring reports from UNICEF and partners; country audits and financial data; data and reports. A significant amount of data is readily available from existing UNICEF and non-UNICEF databases. These must also be reviewed before any requests for data can be made to UNICEF country office staff, government counterparts, or any other key informants.
Conduct a detailed desk-based document review and analysis and light-touch country self- assessments: The focus of this activity is to review and analyse secondary data and other documentary evidence, and data from 15 selected countries. Task for this activity are: (i) to pre-populate the evaluation matrix for the 15 desk review countries; (ii) execute self-assessments by the ECD/Education teams in the countries that will selected for field-based data collection (using and existing/adapted self-assessment tools) to ensure that the evaluators understand the context and the ECD/ECE programmes fully and refine the methodology for the theme-based case studies accordingly; (iii) refine data collection and data analysis tools for Phase 3; and, (iv) to orient and inform countries that will be asked to participate in the evaluative case studies and host primary data collection missions.
Drafting and submission of the document review report: To be reviewed only by the evaluation manager, the desk review report should present an updated chapter of the global context and UNICEF role in ECD/ECE, with an updated methodology section, and a 3-page summary on the context of each of the 15 countries featured in the desk-based review, confirm the themes for the evaluative case studies (2-3 themes). The format for this report is discussed in Section X of the attached terms of reference.
Phase 3: Primary data collection: The activities proposed for this phase are presented below. However, evaluation consultants are free to augment or propose alternatives data collection approaches.
In depth interviews with key informants at the global and regional level: Using the stakeholder analysis generated in Phase 1, a list of key informants will be updated by the evaluation team, in conjunction with the evaluation manager, UNICEF staff, and members of the Evaluation Reference Group (ERG). Primary data will then be collected through key informant interviews, in part to answer a subset of questions for which these informants will be the primary sources, and also to triangulate and/or corroborate some of the data from the desk-based review.
In-country visits: Field visits will be undertaken in a subset of countries that were designated for theme-based evaluative case studies (3 or 4 of the 15 countries, depending on the budget), also for the purpose of collecting primary data. The duration for each visit will be up to 10 working days. Each field visit will commence with a briefing meeting with UNICEF country office education teams, where the 3-page summary about the country context can be shared and subjected for review by the country office. Field-based data collection methods will include validation of the self-assessment, a country level stakeholder analysis, field observations, key informant interviews for UNICEF staff and implementing partners, as well as interviews and/or focus groups for beneficiaries. It is recommended to pilot the methodology and data collection tools during the first field mission before proceeding to subsequent missions. Two debriefing meetings will be conducted, one with the UNICEF country office, and another with interested evaluation participants to present key observations from the field missions, and where feasible, preliminary findings. The debriefing meeting will also be used as an opportunity to fill data gaps, and/or correct the record on factual information, were necessary.
Phase 4 – Data analysis and reporting: The activities proposed for this phase are presented below. However, evaluation consultants are free to augment or propose alternatives data analysis and reporting approaches.
Updating the data analysis plan: Accuracy, triangulation and disaggregation of data will be of crucial importance for the comprehensiveness, usefulness and credibility of the evaluation findings. hence data analysis and interpretation methods proffered in the earlier stages will be updated accordingly. Also, the evaluation will seek to identify contextual and structural/systemic factors explaining the evaluation findings – supporting or impeding performance of UNICEF. Some of the bottlenecks may relate to corporate characteristics, system-wide processes and cross-sectoral issues that lie beyond the education sector. These need to be listed but not examined in great depth.
Drafting and submission extended outline (zero draft): The data phase will conclude with the submission of an extended outline as described in Section X of the attached terms of reference, which will be a mock-up of the final report, and if possible, a collection of insights and/or preliminary findings from the field visits, which will be subjected to more systematic and comprehensive analysis.
Confirmatory online survey of findings: At the reporting stage, a single round of an online survey using the Delphi methodology, or a comparable technique will be executed to validate preliminary findings, and to establish the extent to which there is consensus and/or generalizability of findings across UNICEF offices. All UNICEF country and regional offices will be included (approximately 150 offices, which makes this a census approach). Delivered online, the survey will be addressed to the Education/ECD Specialist who will be requested to complete it, ideally with inputs from other programme leads. Each country office will be asked to return only one completed questionnaire.
Drafting and submission final report: Based on the outline in Section X of the attached terms of reference, the report will be presented in four drafts (completed first draft, revised draft, penultimate draft, and final report). The first draft will be reviewed by the evaluation manager and second reader within the Goal Area 2/5 team, while the revised draft will be presented to ERG for review. To be submitted for clearance by the Director, the penultimate draft should incorporate feedback from the ERG. The final draft will be submitted for copyediting and publishing.
To qualify as an advocate for every child you will have…
Attributes listed in a through f, are required for Team Leader role, while g will be considered an added advantage.
Former UNICEF staff or consultants that have worked on education and/or ECD programmes may be members of the evaluation team if they meet the technical qualifications for skills. However, any prior involvement with UNICEF should be declared during the bidding/contracting stages so that prospective team can be vetted and cleared of possible conflicts of interest.
The overall duration for the evaluation team is estimated to be 9 months, from January 2024 to September 2024. After studying the terms of reference and the responsibilities of the position they are seeking, applicants should propose the total level of effort (in person days), that they will contribute in executing of the evaluation.
How to Apply:
The evaluation team will be made up of a team leader, two evaluators, two researchers, and national consultants that will be recruited at a later stage from each of the 4 countries that will be selected to host field-based data collection missions. Each prospective evaluator or researcher should submit an application as an individual. The application packet should contain the following:
Applicants are free to indicate the names of persons that they would like to team up with if selected. However, all prospective team members should submit an individual application, specific to the role they want to fill in the evaluation team. Also, contracts will be issued to individual consultants, and not consultancy firms or institutions. Additional details about the application process are provided in Section XII of the attached terms of reference.
Only shortlisted candidates will be contacted and advance to the next stage of the selection process.