JOB DESCRIPTION

Please specify the position you are applying for in your application 

 

Purpose of the Consultancy:

To undertake the endline evaluation for the project, “My Body, My Life, My World – Empowering Young girls and women to decide over their own bodies and Ensure universal access to comprehensive adolescent and youth friendly SRHR information and services (EYE universal SRHR)” implemented by UNFPA in partnership with CARE and Marie Stopes Uganda (MSU) in Kamuli and Mayuge districts with support from Norwegian Government.

How you can make a difference:

UNFPA is the lead UN agency for delivering a world where every pregnancy is wanted, every childbirth is safe and every young person”s potential is fulfilled.  UNFPA’s strategic plan (2022-2025), reaffirms the relevance of the current strategic direction of UNFPA and focuses on three transformative results: to end preventable maternal deaths; end unmet need for family planning; and end gender-based violence and harmful practices. These results capture our strategic commitments on accelerating progress towards realizing the ICPD and SDGs in the Decade of Action leading up to 2030. Our strategic plan calls upon UN Member States, organizations and individuals to “build forward better”, while addressing the negative impacts of the Covid-19 pandemic on women’s and girls’ access to sexual and reproductive health and reproductive rights, recover lost gains and realize our goals.

In a world where fundamental human rights are at risk, we need principled and ethical staff, who embody these international norms and standards, and who will defend them courageously and with full conviction.

UNFPA is seeking candidates that transform, inspire and deliver high impact and sustained results; we need staff who are transparent, exceptional in how they manage the resources entrusted to them and who commit to deliver excellence in programme results.

Scope of Work

BACKGROUND

The United Nations Population Fund works to deliver a world where every pregnancy is wanted, every childbirth is safe and every young person’s potential is fulfilled. We are also the leading United Nations agency that supports the Government of Uganda to generate population data for planning and decision making. We ensure the empowerment of women, adolescents and young people to live their dreams and fulfill their potential.  UNFPA is active in 56 districts across Uganda.

With support from the Norwegian Government, UNFPA in partnership with CARE and MSU the EYE project is aimed  at  increasing  the utilization of integrated SRHR services by adolescents and youth aged 10-24 in Uganda with a focus on the two high-SRHR burdened districts of Eastern Uganda (Kamuli and Mayuge).  The two districts have a population of about 1.1M people, where  poverty levels are 45% and 49% for Kamuli and Mayuge respectively; and Teenage pregnancy rate is at 23.8% and 20.9% for  Kamuli and Mayuge respectively.

The EYE project responds to identified problems of high maternal mortality rate; high levels of teenage pregnancies, low use of contraceptives; and weak programming for GBV prevention and response. The project’s goal is to ensure adolescents and young people (10-24), especially those most vulnerable and in hard-to-reach areas, can exercise their sexual and reproductive health and rights (SRHR) by empowering them to make informed choices, accessing high-quality integrated SRHR and GBV services through strengthened health facilities, and fostering a supportive enabling environment at the sub-national level.

MAIN PROJECT RESULTS

The main project results include the following:

1. Impact: Reduced maternal deaths and unintended teenage pregnancies
2. Outcome: Increased utilisation of quality integrated sexual and reproductive health and rights and GBV services, and reduced GBV and harmful practices in Kamuli and Mayuge districts.
3. Output 1: Empowerment for demand generation for adolescent SRHR: Adolescents and young people aged 10-24, including those in hard-to-reach communities and those most at risk, are empowered to decide over their own bodies, make informed choices, and utilize high quality, integrated SRHR information and services.
4. Output 2: Health facilities in the target districts have increased capacity to provide universal access to acceptable, high quality, integrated SRHR and GBV services, particularly for the most vulnerable adolescents and youth
5. Output 3: An enabling environment for adolescents and young people to exercise their SRH rights, make informed choices, and demand SRH services is strengthened at district and community levels.

 PURPOSE AND SCOPE OF THE STUDY 

UNFPA is seeking qualified Ugandans to conduct an endline study of the achievement of the EYE universal SRHR project during the three years of implementation (2023-2025).

The purpose of this evaluation is to assess to what extent the EYE programme outcomes and outputs were achieved including the overall results, effects/changes realized, good practices, challenges faced, and lessons learned. The endline evaluation will assess the relevance, effectiveness, efficiency, impact, coherence, and prospects for sustainability of the EYE programme based on the OECD/DAC criteria and UNFPA evaluation guidelines. These will also be assessed to determine the contribution of the EYE programme towards 9^th Government of Uganda/UNFPA Country Programme outcomes.

The evaluation will apply appropriate methodology for assessing principle cross-cutting elements of equity and vulnerability, gender equality and human rights. It will be based on the guiding principles, norms and standards for evaluations adopted at UNFPA.

Objectives

1. Assess the extent to which intended outputs have been achieved, and the extent to which these outputs have contributed to the achievement of the outcomes (Effectiveness)
2. Analyse the relevance of project components in increasing access and use of SRHR and GBV services among target beneficiaries. (alignment to national, UNFPA, and donor priorities; appropriateness and acceptability to users, and the existing systems)
3. Assess the potential for sustainability of the project results and interventions: the extent to which the achieved project results and the interventions aimed at increasing access to GBV prevention and response services are likely to be maintained.
4. Examine the extent to which the project was implemented efficiently; specifically, how funding, personnel, administrative arrangements, time and other inputs contributed to, or how it hindered the achievement of results.
5. Identify challenges, barriers which have impeded implementation of project components; and enablers that have facilitated successful implementation of the project.
6. Identify and document good practices and lessons learned during the 3-year life of the project.

Scope 

In relation to the above, the endline evaluation will cover the 2 programme districts of Kamuli and Mayuge.  Contribution analysis will be employed as the overall analytical framework of this evaluation to identify the contribution that the EYE programme has made to the observed set of changes (i.e. positive/negative, intended/unintended effects) among targeted beneficiaries and communities. Mixed methods will be employed to generate and triangulate qualitative and quantitative data across multiple sources to answer the evaluation questions.

• Health Facility Assessment on Youth Friendly Services: The health facility assessment will be conducted on a sample of targeted facilities in Kamuli and Mayuge districts regarding the capacity, availability, and accessibility of quality SRHR services that are responsive to the needs of women, girls and boys. Observations at facilities will be conducted as part of the facility assessment. The SRHR/YFS consultant will lead the health facility assessment survey under the overall guidance and supervision of the Team Leader.
• Comprehensive document review: The consultants will undertake a comprehensive review of EYE programme documents (proposal, work plans, progress reports, best practices/lessons learned documents, website articles etc.), national SRHR/GBV strategies and plans, and other relevant documents. The information contained in the documents will be analysed and triangulated with other information sources and methods.
• Analysis of national and district level SRHR and GBV data The above will be complemented with quantitative analysis which will include the profiling of district-level data (from the 2 districts) from data management systems such as HMIS, NGBVD, SAUTI and other programme data to assess indicators of the EYE results framework.

The evaluation will follow a collaborative and participatory approach ensuring close engagement with programme beneficiaries, implementing partners, district local government leadership, MDAs, and other key stakeholders as will be informed by an initial rapid desk review and inception meetings with key stakeholders which will support the refinement and finalisation of the evaluation methodology and analytical framework to be included into the inception report.

All information will be analysed and triangulated using standardised methods (e.g. structured coding and content analysis). Results from the endline evaluation will be validated through a (virtual or physical) workshop with stakeholders from national and district levels and disseminated through a regional dissemination workshop (physical).

Specific Tasks

1. Work with evaluation team members to develop an inception report containing the evaluation approach, methodology, evaluation matrix, review of the project theory of change, data collection tools, and submit it to the Evaluation Reference Group for review.
2. Present the inception report to UNFPA, the donor and the evaluation reference group and thereafter finalise the report based on feedback received.
3. Recruit and train research assistants to assist with the collection of relevant quantitative and qualitative data during the field mission.
4. Undertake desk review and analysis of key background documents, including the Project documents, Results framework, Project progress reports, baseline reports, SRH Integration framework, SRH Integration Score Card
5. Seek ethical clearance from an Institutional Review Board and the Uganda National Council for Science and Technology
6. Collect primary quantitative and qualitative data from selected sites (schools, health facilities, and community levels etc). A list of key stakeholders to be engaged by the evaluation team will be established at the beginning of the inception phase but will include Donor staff, UNFPA staff, implementing partners, district officials, and key beneficiaries.
7. Conduct an in-depth analysis and triangulation of primary and secondary data. The analysis should reflect the progress concerning the evaluation criteria above. Data on indicators should be disaggregated for sex, age, and key vulnerability groups (PWDs, KPs, etc.) as guided by UNFPA M&E advisors.
9. Write a draft evaluation report summarising the background, evaluation approach and methodology, findings, lessons, best practices, conclusions, and recommendations.  (max 50 pages excluding annexes).
10. Present the draft report Evaluation and Health Facility Assessment Report to UNFPA, the donor and the Evaluation Reference group, and representatives of stakeholders who participated in the Endline, at a virtual validation workshop, and finalise the report based on the feedback received.
11. Disseminate the Endline findings to the EYE partners during a regional l dissemination workshop.
12. Update the Results framework with endline indicator values

Time horizon

The EYE universal SRHR project is a three-year project, which is in its final year of implementation. Therefore, within the framework of the above evaluation objectives, this endline evaluation will cover the period from January 2023-December 2025.

Geographic scope

At the national level, the main sectors to focus on will be the Ministry of Health, Ministry of Gender Labour and Social Development and Ministry of Education. Otherwise much of the data collection will be from the 2 districts of focus; Kamuli and Mayuge.

EVALUATION CRITERIA AND PRELIMINARY EVALUATION QUESTIONS

Evaluation Criteria

In accordance with the OECD/DAC evaluation criteria and UNFPA global evaluation guidelines, the endline evaluation will examine the following evaluation criteria: relevance, effectiveness, efficiency, impact and prospect for sustainability. It will also use the evaluation criterion of coherence to assess the extent to which the UNFPA Uganda CO harmonized interventions under the NORAD programme with other actors, promoted synergy and avoided duplication amongst SRHR and Health Development Partners (HDP) and  the framework of the UNCT Delivering as one.

 

Relevance	The extent to which the project design was relevant for its different stakeholder groups (including primary beneficiaries) and to national policies of the GOU, and Country Programme. How effectively the programme adapted its strategies and interventions to contextual changes to achieve results.
Effectiveness	The extent to which the EYE universal SRHR project results and targets have been achieved, including both intended and unintended effects, and the extent to which these outputs have contributed to the achievement of the Country programme outcomes.
Efficiency	The extent to which the project outputs and outcomes have been achieved in the most economic and timely way, including an assessment of how funds, expertise, personnel, and implementation modalities contributed to, or hindered the achievement of results.
Coherence	The internal and external coherence of the EYE universal SRHR project, including the coherence between specific project components, other UNFPA programmes, as well as coherence with other partners’ SRHR and GBV interventions in the Kamuli and Mayuge districts

 

Sustainability	The continuation of benefits from the EYE universal SRHR project after its termination, linked to their continued resilience to risks.
Impact	The extent to which the EYE universal SRHR project has generated or is expected to generate significant positive or negative, intended or unintended, impact for its beneficiaries

 

 

Indicative Evaluation Questions

The evaluation questions presented below are indicative and preliminary. Based on these examples, at the inception phase, the evaluators in consultation with the Evaluation Reference Group (ERG), are expected to develop a final set of evaluation questions. The examples here serve the purpose of illustrating to the evaluator’s key questions of interest to the ERG.

Relevance

1. To what extent was the EYE universal SRHR project aligned with specific national policies and sectoral priorities (Health, Gender, Education, including Vision 2040 & NDP III).
2. To what extent do planned interventions adequately reflect the outputs and outcomes stated in the CPD and aligned with UNFPA Strategic Plan and national priorities.
3. To what extent did the project integrate intersectional approaches (gender, disability, socio-economic status) in promoting SRHR and GBV services

Effectiveness

1. To what extent did the interventions supported by EYE universal SRHR in all programmatic areas contribute to the achievement of planned results (outputs and outcomes)^[4], including the extent of reach of planned geographic areas and target groups successfully reached?
2. What factors facilitated or hindered the achievement of programme results?
3. How did the EYE results contribute to the 9th CPD results?
4. To what extent did the programme reach vulnerable groups, such as persons living with disabilities with information and services? What factors are facilitating and hindering the reach of these groups?

Efficiency

1. How and to what extent has the EYE universal SRHR project facilitated the use of its funding, personnel, administrative arrangements, time and other inputs to optimize achievement of results.
2. To what extent did the intervention mechanisms (Engaging Men Through Accountable Practices (EMAP), partnership strategy; execution/implementation arrangements; partnerships, etc) foster or hinder the achievement of the project outputs. What adaptations were made to ensure the achievement of results, including those specifically related to advancing integration, and gender equality and human rights dimensions?
3. Were EYE project results achieved on time and according to the implementation plans? This includes assessment of timeliness in the release of funds, implementation rates, reporting, fund utilisation rates, etc.
4. What decisions were taken during the implementation period which helped to enhance efficiency? Were the logistics and procurement decisions optimal?
5. What factors facilitated or hindered the efficiency of the project, i.e. the timely and economic implementation of work plans and achievement of results?

Sustainability

1. What specific capacity-building strategies were implemented to enable government and partners to sustain interventions beyond the project period?
2. How have partnerships built by the project enhanced national ownership, and what are the risks to sustaining this ownership?
3. What mechanisms have been established to ensure continuous community participation, monitoring, and accountability beyond the project cycle?
4. To what extent has the project leveraged on the existing government programs, especially the Parish Development Model?
5. What are the lessons learned from the project? How can the lessons learned be used for strategic positioning for future programming.
6. What best practices and lessons learned can inform scalability and replication in other SRHR/GBV programmes and be applied in future programme and policy development?

Coordination

1. To what extent has the EYE universal SRHR project contributed to the functioning and consolidation of coordination mechanisms at national and sub-national level for SRHR/GBV?
2. To what extent has the project fostered cross-sectoral collaboration to enhance SRHR and GBV integration within Uganda’s development agenda

Coherence

1. To what extent were the EYE project interventions interlinked with other UNFPA projects and other SRHR and GBV partners in Kamuli and Mayuge districts?
2. How effective was the way of working with and delivering through the Government sectors (MDAs) at national and district level and implementation partners coherence with needs of target groups^[1], SDGs, and UNFPA Strategic plan (2022-2025)

Impact

The evaluation will assess the extent to which the project had an impact (positive or negative, intended or unintended) on the targeted beneficiaries.

1. What difference did the EYE project make in the lives of beneficiaries?
2. What is the specific contribution of the EYE project to the observed higher-level results (outcome/impact level) in the target districts?
3. Were there any unintended (positive or negative) impacts of the EYE project and what lessons can be drawn from these?
4. What are the outstanding successes and innovations that can be adopted/reciprocated for further implementation and available opportunities for further engagements?

The final evaluation questions and the evaluation matrix will be finalized by the evaluation team in the design report.

APPROACH AND METHODOLOGY

Evaluation Approach/ Design

The evaluation will use a cross-sectional mixed-methods design, which involves both quantitative and qualitative approaches of data collection. The Evaluation design will build on the baseline assessment methodology to ensure consistent measurements of the pre and post project intervention impact.

Theory-based approach

The evaluation team will be required to verify the theory of change underpinning the EYE universal SRHR project and use this theory of change to determine whether changes at output and outcome levels occurred (or not) and whether assumptions about change hold true. The analysis of the theory of change will serve as the basis for the evaluators to assess how relevant, effective, efficient and sustainable the support provided by the EYE universal SRHR project was during the implementation period.

Contribution analysis 

As part of the theory-based approach, contribution analysis will be employed as the overall analytical framework of this evaluation to identify the contribution that the EYE universal SRHR project has made to the observed set of changes (i.e. positive/negative, intended/unintended effects) among targeted beneficiaries and communities. Specifically, the evaluators will use a contribution analysis to explore whether evidence to support key assumptions exists, examine if evidence on observed results confirms the chain of expected results in the theory of change, and seek out evidence on the influence that other factors may have had in achieving desired results.

Accordingly, the evaluation team will reconstruct the logic behind the EYE universal SRHR project interventions (theory of change) for the period under evaluation. The theory of change (ToC) reflects the conceptual and programmatic approach taken by UNFPA over the period under evaluation including the most important implicit assumptions underlying the change pathway. The ToC will include the types of intervention strategies or modes of engagement used in program delivery, the principles/guiding interventions, the elements of the intervention logic, the type and level of expected changes and the external factors and influence and determine the causal links depicted in the theory of change diagram. The ToC will be tested during the field and data collection phase.

Evaluators will analyse and interpret the logical consistency of the chain of effects: linking programme activities and outputs with changes in higher-level outcome areas, based on observations and data collected along the chain. This analysis should serve as the basis of judgment by the evaluators on how well the programme has contributed to the achievement of the intended results (outputs and outcomes) in the EYE universal SRHR programme document.

The indicative and preliminary evaluation questions listed above must be complemented by questions on sets of assumptions that capture key aspects of the intervention logic. The data collection for each of the assumptions will be guided by clearly formulated quantitative and qualitative indicators which must also be included in the evaluation matrix.

Methodology and tasks

The consultants will detail their methodological approach in their Inception report, as per the TOR, including the tools they will use. The methodology and approach must incorporate human rights and gender equality perspectives.

The evaluation team shall develop and conduct the evaluation in accordance with the UNEG Norms and Standards for Evaluation, Ethical Guidelines for Evaluation, Code of Conduct for Evaluation in the UN System, and Guidance on Integrating Human Rights and Gender Equality in Evaluations. The evaluators will be requested to sign the UNEG Code of Conduct  prior to starting their work.

The methodological design of the evaluation shall include in particular: (i) a theory of change; (ii) a strategy for collecting and analyzing data; (iii) specifically designed tools for data collection and analysis; (iv) an evaluation matrix; and (v) a detailed evaluation work plan and agenda for the field phase.

Evaluation Process

The evaluation process can be broken down into three different phases that include different stages and lead to different deliverables: inception phase; field phase; reporting phase; and phase of dissemination and facilitation of use. The evaluation manager and the evaluation team leader must undertake quality assurance of each deliverable at each phase and step of the process, with a view to ensuring the production of a credible, useful and timely evaluation.

Inception Phase

In the inception phase, the evaluation manager will lay the foundation for communications around the evaluation. All other activities will be carried out by the evaluation team, in close consultation with the evaluation manager and the ERG. This phase includes:

• Evaluation kick-off meeting between the evaluation manager and the evaluation team
• Desk review of background information and documentation on the country context and the EYE universal SRHR project, as well as other relevant documentation.
• Detailed review of the theory of change underlying the EYE universal SRHR project. This includes an analysis of the assumptions on which the theory of change is based; contextual factors in which the programme is implemented (how it affects activities and result); indicators of progress in achieving results; links where the causal chain seems to break or are not well established; how results are expected to be sustained after the interventions end, etc.
• Prioritization and formulation of a final set of evaluation questions based on the preliminary evaluation questions provided in the ToR.
• Development of a final stakeholder map and a sampling strategy to select sites to be visited and stakeholders to be consulted in Kampala and the target districts through interviews and group discussions.
• Development of a data collection and analysis strategy for the different components of the evaluation etc. as well as a concrete and feasible evaluation work plan and agenda for the field phase.
• Development of data collection methods and tools.
• Development of an initial communication plan by the evaluation manager, in consultation with the UNFPA communication officer to support the dissemination and facilitation of use of the evaluation results. The initial communication plan will be updated during each phase of the evaluation, as appropriate, and finalized for implementation during the dissemination and facilitation of use phase.

At the end of the inception phase, the evaluation team will develop an inception report that presents a robust, practical and feasible evaluation approach, detailed methodology and work plan. The evaluation team will develop the inception report in consultation with the evaluation manager who will submit it to the ERG for review. The ERG members will provide feedback and comments to the inception report during the 1^st ERG meeting.

 Field Phase

The evaluation team will collect the data and information required to EYE universal SRHR project the evaluation questions in the field phase. Towards the end of the field phase, the evaluation team will conduct a preliminary analysis of the qualitative data to identify emerging findings that will be presented to the CO and the ERG.^[1] The field phase should allow the evaluators sufficient time to collect valid and reliable data to cover the thematic scope of the EYE universal SRHR project  evaluation and be able to compare baseline and endline data. A period of up to 3 weeks for data collection (HFA, KII, FGD, site visits) is planned for this evaluation. However, the evaluation manager will determine the optimal duration of data collection, in consultation with the evaluation team during the inception phase.

The field phase includes:

●        Meeting with the UNFPA Uganda CO staff to launch the data collection.

●        Meeting of the evaluation team with relevant programme officers at the UNFPA Uganda CO.

●        Data collection at national and sub-national levels.

 At the end of the field phase, the evaluation team will hold a debriefing meeting with the CO and the ERG to present the emerging findings from the data collection. In addition to the ERG members, selected stakeholders who participated in the data collection, including representatives of DLG and IP field teams will be invited to participate in the meeting virtually. The meeting will serve as a mechanism for the validation of collected data and information and the exchange of views between the evaluators and important stakeholders. It will enable the evaluation team to refine the findings, which is necessary so they can then formulate their conclusions and develop credible and relevant recommendations.

 Analysis and Reporting Phase

In the analysis and reporting phase, the evaluation team will continue the analytical work initiated during the field phase. The quantitative data collected through the HFAT, HMIS and NGBVD will be cleaned, entered and analyzed using appropriate software to be determined by the evaluators. Qualitative data will be entered and coded using Nvivo, Dedoose or similar software to facilitate structured content analysis. The evaluation team will jointly conduct a virtual or onsite analysis workshop to triangulate draft findings and conclusions by evaluation question based on the various sub-components of the endline evaluation (HFA, HMIS/NGBVD, qualitative etc.).

The evaluation team shall develop a draft evaluation report that summarizes the context, methodology, findings, conclusions and recommendations). The draft report will take into account the comments and feedback provided by the CO and the ERG at the debriefing meeting at the end of the field phase. Additionally, the evaluators should write a stand-alone report on the HEALTH Facility Assessment, which will be annexed to the draft evaluation report. The final report will summarize and triangulate the key findings from the HFA as part of the overall contribution analysis.

Prior to the submission of the draft report to the evaluation manager, the evaluation team will perform an internal quality control against the criteria outlined in the Evaluation Quality Assessment (EQA) grid. The evaluation manager will subsequently review the draft evaluation report, using the same criteria (defined in the EQA grid). If the quality of the report is satisfactory (in form and substance), the draft report will be circulated to the ERG members for review. In the event that the quality of the draft report is unsatisfactory, the evaluation team will be required to revise the report and produce a second draft.

The evaluation manager will perform his/her review of the draft final report against the completed evaluation matrix (to ensure that the analysis – responses to the evaluation questions – rests on credible data and information and is, in fact, evidence based). S/he will also collect and consolidate the written comments and feedback provided by the members of the ERG. Based on the comments, the evaluation team should make appropriate amendments, prepare the final evaluation report and submit it to the evaluation manager. The final report should clearly account for the strength of evidence on which findings rest to support the reliability and validity of the evaluation. Conclusions and recommendations need to clearly build on the findings of the evaluation. Each conclusion shall make reference to the evaluation question(s) upon which it is based, while each recommendation shall indicate the conclusion(s) from which it logically stems.

UNFPA may decide to conduct a physical or virtual co-creation workshop to draft the recommendations jointly with UNFPA and IP programme staff, representatives of the Embassy, or the wider ERG. If this option is chosen, the evaluation team is still responsible for formulating the final recommendations based on the co-creation workshop deliberations.

The evaluation report is considered final once it is formally cleared by the ERG.

Dissemination and Facilitation of Use Phase

In the dissemination and facilitation of use phase, the evaluation team will develop a PowerPoint presentation of the evaluation results that summarizes the key findings, conclusions and recommendations of the evaluation in an easily understandable and user-friendly way.

The evaluation matrix

The evaluation matrix is center-piece to the methodological design of the evaluation. The matrix contains the core elements of the evaluation. It outlines (i) what will be evaluated: evaluation questions for all evaluation criteria and key assumptions to be examined; and (ii) how it will be evaluated: data collection methods and tools and sources of information for each evaluation question and associated key assumptions. By linking each evaluation question (and associated assumptions) with the specific data sources and data collection methods required to answer the question, the evaluation matrix plays a crucial role before, during and after data collection.

•  In the inception phase, the evaluators should use the evaluation matrix to develop a detailed agenda for data collection and analysis and to prepare the structure of interviews, group discussions and site visits. At the inception phase, the evaluation team must enter, in the matrix, the data and information resulting from their desk (documentary review) in a clear and orderly manner.
• During the field phase, the evaluation matrix serves as a working document to ensure that the data and information are systematically collected (for each evaluation question) and are presented in an organized manner. Throughout the field phase, the evaluators must enter, in the matrix, all data and information collected. The evaluation manager will ensure that the matrix is placed in a Google drive and will check the evaluation matrix on a daily basis to ensure that data and information is properly compiled. S/he will alert the evaluation team in the event of gaps that require additional data collection or if the data/information entered in the matrix is insufficiently clear/precise.
• In the reporting phase, the evaluators should use the data and information presented in the evaluation matrix to build their analysis (or findings) for each evaluation question. The fully completed matrix is an indispensable annex to the report and the evaluation manager will verify that sufficient evidence has been collected to answer all evaluation questions in a credible manner.

As the evaluation matrix plays a crucial role at all stages of the evaluation process, it will require particular attention from both the evaluation team and the evaluation manager. The evaluation matrix will be drafted in the inception phase and must be included in the inception report. The evaluation matrix will also be included in the annexes of the final evaluation report, to enable the evaluation report’s users to access the supporting evidence for the answers to the evaluation questions.

Data collection methods

To be able to make meaningful assessment of trends, the methods used at baseline and MTR will be adopted at endline.

Field visits – Site visits will be conducted on sample basis during which group interviews and focus group discussions (FGD) will be conducted with direct and indirect beneficiaries, including in-school and out of school young people ^[2], men and women, health care workers, teachers, parents, religious, cultural and political leaders. The evaluators will be required to take into account ethical considerations when collecting information. The proposed field visit sites, stakeholders to be engaged and interview protocols will be outlined in the inception report to be submitted by the evaluation team. When choosing sites to visit, the evaluation team should make explicit the reasons for selection. The choice of the locations to visit at sub-national level needs to take into consideration the implementation of the project components in those areas and done in consultation with the UNFPA project technical team and the evaluation manager.

The evaluation team is expected to dedicate up to three weeks for data collection in the field. The data collection tools that the evaluation team will develop, shall be presented in the inception report.

 Data validation

The evaluators will check the validity of the collected data and information and verify the robustness of findings at each stage of the evaluation, so they can determine whether they should further pursue specific hypotheses (related to the evaluation questions) or disregard them when there are indications that these are weak (e.g. contradictory findings or lack of evidence).

The evaluators will use a variety of mechanisms to ensure the validity of the information and data collected^[3] including but not limited to:

• Systematic triangulation of data sources and data collection methods and tools
• Regular exchange with the UNFPA programme staff at the CO
• Internal evaluation team meetings to corroborate data and information for the analysis of assumptions, the formulation of emerging findings and the definition of preliminary conclusions
• Discussion of the emerging findings during a debriefing meeting with the ERG members and key district level stakeholders at the end of the data collection process (2^nd ERG meeting)
• Validation of draft findings and conclusions when the evaluators present the draft evaluation report (3^rd ERG meeting).

The specific validation mechanisms will be further elaborated in the inception report.

 Sampling strategy

The evaluation will use different sampling strategies suitable for the variables and data sources identified in the evaluation matrix. For the qualitative primary data collection, purposive sampling will be used to sample districts and key informants for the KIIs, group interviews, and FGDs, and sites to be visited. UNFPA will provide an initial overview of the interventions supported by the EYE universal SRHR project, the locations where these interventions have taken place, and the stakeholders involved in these interventions.

Building on the initial stakeholder map and based on information gathered through document review and discussions with UNFPA staff, the evaluators will develop the final stakeholder map. From this final stakeholder map, the evaluation team will select a sample of stakeholders at national and sub-national levels who will be consulted through interviews and/or group discussions during the data collection phase. These stakeholders must be selected through clearly defined criteria and the sampling approach outlined in the inception report. In the inception report, the evaluators should also make explicit what groups of stakeholders were not included and why. The evaluators should aim to select a sample of stakeholders that is as representative as possible, recognizing that it will not be possible to obtain a statistically representative sample.

The evaluation team shall also select a sample of sites that will be visited for data collection for the KIIs, FGD, group interviews and direct observations, and provide the rationale for the selection of the sites in the inception report. UNFPA will provide the evaluators with necessary information to access the selected locations, including logistical requirements and security risks, if applicable. The sample of sites selected for visits should reflect the variety of interventions supported by the EYE universal SRHR project, both in terms of thematic focus and context.

The evaluation team should consider how to include non-beneficiaries in the sample, with a focus on using qualitative methods to assess the effectiveness of the programme strategies in reaching beneficiaries with SRHR/GBV information and services, creating demand and changing health seeking behaviours. With regards to programme reach and coverage, the evaluation team could consider including questions to measure dose-response relationship at programme sites for different interventions – and then explore reasons for non or under exposure (recall bias notwithstanding).

The final sample of stakeholders and sites will be determined in consultation with the evaluation manager, based on the review of the inception report.

Ethical clearance

The Evaluators will seek Local Institutional Review Board (IRB) approval of the inception report and protocols using a fast-tracked process. The evaluation team shall finalise the inception report based on feedback from IRB/UNSCT.
Deliverables

• Inception Report – detailing the research methodology including sampling procedures, data collection tools, data analysis plan and detailed work plan and schedule.
• PowerPoint presentation of the inception report. The PowerPoint presentation will be delivered at an ERG meeting to present the contents of the inception report and the agenda for the field phase. Based on the comments and feedback of the ERG, the evaluation team will develop the final version of the inception report.
• PowerPoint presentation for debriefing meeting with the CO and the ERG. The presentation provides an overview of key emerging findings of the evaluation at the end of the field phase. It will serve as the basis for the exchange of views between the evaluation team, UNFPA CO staff (incl. senior management) and the members of the ERG who will thus have the opportunity to provide complementary information and/or rectify the inaccurate interpretation of data and information collected
• Draft report for comment and feedback
• Copy of the Inception Report shared to the MOH Committee on Evaluation and Research, MGLSD and NPA
• Produce an acceptable quality of the final evaluation report, approved by the MOH Committee on Evaluation and Research. The report should be a maximum of 50 pages in length, excluding annexes.
• PowerPoint presentation of the evaluation results. The presentation will provide a clear overview of the key findings, conclusions and recommendations to be used for the dissemination of the final evaluation report.
• Cleaned-up datasets in readable format.
• Utilization matrix l outlining how key findings from the report will be shared and used by different stakeholders, including the government, UNFPA, and implementing partners

Monitoring and Reporting

The consultants will provide periodic updates (in writing) on progress of the assignment, at a frequency to be agreed upon by the M&E and technical teams from UNFPA. The consultant will present all key milestone products to UNFPA M&E and technical teams, who will in turn share them with the MOH Research and Evaluation Committee.
Management of the study

Expected travel
The consultants will be expected to travel widely, to the 2 target districts

Evaluation Quality Assessment Checklist   1. Structure and Clarity of the Report To ensure the report is user-friendly, comprehensive, logically structured and drafted in accordance with international standards. 2. Executive Summary   To provide an overview of the evaluation, written as a stand-alone section including key elements of the evaluation, such as objectives, methodology and conclusions and recommendations. 3. Design and Methodology To provide a clear explanation of the methods and tools used including the rationale for the methodological choice justified. To ensure constraints and limitations are made explicit (including limitations applying to interpretations and extrapolations; robustness of data sources, etc.) 4. Reliability of Data To ensure sources of data are clearly stated for both primary and secondary data. To provide explanation on the credibility of primary (e.g. interviews and focus groups) and secondary (e.g. reports) data established and limitations made explicit. 5. Findings and Analysis To ensure sound analysis and credible evidence-based findings. To ensure interpretations are based on carefully described assumptions; contextual factors are identified; cause and effect links between an intervention and its end results (including unintended results) are explained. 6. Validity of conclusions To ensure conclusions are based on credible findings and convey evaluators’ unbiased judgment of the intervention. Ensure conclusions are prioritised and clustered and include: summary; origin (which evaluation question(s) the conclusion is based on); detailed conclusion. 7. Usefulness and clarity of recommendations To ensure recommendations flow logically from conclusions; are targeted, realistic and operationally feasible; and are presented in priority order. Recommendations include: Summary; Priority level (very high/high/medium); Target (administrative unit(s) to which the recommendation is addressed); Origin (which conclusion(s) the recommendation is based on); Operational implications. 8. Sharing and Utilizing Key Findings: A Stakeholder Utilization Matrix To ensure the insights from this assessment are effectively disseminated and acted upon, a utilization matrix will be developed. This matrix will outline how key findings from the report will be shared and used by different stakeholders, including the government, UNFPA, and implementing partners. The aim is to maximize the impact of the evaluation by fostering a clear pathway from findings to actionable strategies and improved programming.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Duration and working schedule

The evaluation team will share workdays (person-days) as per the following tentative plan:

Task	Number of Days required
Team Leader	Gender/Youth Expert	Total # of Days
Inception Phase – writing inception report	4	4	8
Development of tools	2	2	4
Training RAs	2	1	3
Testing/ validation and refinement of the draft tools	1	1	2
Data collection	7	5	12
Data entry and analysis	7	2	9
Report writing	5	3	8
Up-dating the Results Framework	2	2	4
Total – Consultancy (man-days)	30	20	50

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The evaluation study will commence in August 2025 and is expected to be concluded by November 2025. The consultant will present an inception report to the MOH Research and Evaluation Committee not later than September 1st, 2025 Data collection will commence by September 17th.  A draft report is expected for review before October 6th^th; and a final report by October 30^th 2025

Payment schedule

The consultant will receive 20% upon submitting the inception report and 30% upon submission of a draft report of the survey. The balance of 50% will be paid upon submission and acceptance of the final report.

Team composition and qualifications

The End Term Evaluation will be conducted by an independent multidisciplinary evaluation team composed of two national consultants with expertise in SRHR/Public Health and Gender/Adolescent Youth, supported by Research Assistants.

The team leader will be overall responsible for the evaluation process and the production of the draft and final evaluation reports. S/he will lead and coordinate the work of the evaluation team during all phases of the evaluation and be responsible for the quality assurance of all evaluation deliverables.  She/he will liaise with the UNFPA M&E Specialist on various issues related to successful completion of the evaluation exercise. The Team leader is expected to be an SRHR expert. S/he will have overall responsibility for providing guidance and leadership in: development of the evaluation design including approach, methodology and workplan; drafting the design, draft and final reports, as well as the brief summary for presentation to the Evaluation Reference Group..

In addition to his/her role as Team Leader, she/he will also take part in the data collection and analysis work during the field phase of the Evaluation. She/he will be responsible for drafting key parts of the design report and of the final evaluation report, including (but not limited to) sections relating to the national context and gender equality.

Qualifications and experience of the Evaluation Team

Team Leader: Public Health or Sexual Reproductive Health Expert

• An advanced degree in Medicine or related fields, Health Economics, Epidemiology or Biostatistics.
• Specialization in public health;
• 8 years’ experience in conducting evaluations in the field of development aid for UN agencies and/or other international organizations;
• Substantive knowledge of sexual and reproductive health as a thematic area
• Knowledge of evaluation methods, data collection and analysis
• Excellent data analysis skills in qualitative and quantitative methods.
• Past experience as evaluation team leader in a related assignment(s).
• Ability to provide deliverables on time
• Excellent written and spoken English Language skills
• Evidence of having carried out similar work before, especially within the East African Region, particularly in the health, gender/Youth and human rights sectors
• Proven experience of conducting studies using mixed methods approach (quantitative and qualitative)
• Expertise in developing or updating a M&E Results Frameworks
• Excellent analytical, writing and communication skills
• Excellent written and spoken English
• Familiarity with UN operations is an added advantage

Gender/Adolescent and Youth Expert

• An advanced degree in Gender and Development, Human Rights, Sociology, Social Work.
• Substantive knowledge of Gender Equality as a thematic area
• Good knowledge of the national development context
• 5 years’ experience in conducting complex evaluations in the field of development aid for UN agencies and/or other international organizations including experience in leading evaluations
• Extensive knowledge and experience in the application of quantitative and qualitative evaluation methods with a strong gender focus.
• In-depth knowledge of evaluation methods, data collection and analysis
• Excellent data analysis skills in qualitative and quantitative methods
• Familiarity with UN operations
• Excellent analytical, writing and communication skills
• Experience working with a multi-disciplinary team of experts
• Excellent written and spoken English
• Experience and understanding of UN programming processes.
• Experience on evaluation of UN supported programmes will be an added advantage.

Languages: 

English

Required Competencies: 

Values:

• Exemplifying integrity,
• Demonstrating commitment to UNFPA and the UN system,
• Embracing cultural diversity,
• Embracing change

Core Competencies: 

• Achieving results,
• Being accountable,
• Developing and applying professional expertise/business acumen,
• Thinking analytically and strategically,
• Working in teams/managing ourselves and our relationships,

UNFPA Work Environment:

UNFPA provides a work environment that reflects the values of gender equality, diversity, integrity and healthy work-life balance. We are committed to ensuring gender parity in the organization and therefore encourage women to apply. Individuals from the LGBTQIA+ community, minority ethnic groups, indigenous populations, persons with disabilities, and other underrepresented groups are highly encouraged to apply. UNFPA promotes equal opportunities in terms of appointment, training, compensation and selection for all regardless of personal characteristics and dimensions of diversity. Diversity, Equity and Inclusion is at the heart of UNFPA”s workforce – click here to learn more.

Disclaimer:

Selection and appointment may be subject to background and reference checks, medical clearance, visa issuance and other administrative requirements.

UNFPA does not charge any application, processing, training, interviewing, testing or other fee in connection with the application or recruitment process and does not concern itself with information on applicants” bank accounts.

Applicants for positions in the international Professional and higher categories, who hold permanent resident status in a country other than their country of nationality, may be required to renounce such status upon their appointment.