JOB DESCRIPTION

Description

International Organization for Migration is hiring a Consultant for Mid Term Evaluation of the project “Promoting the Integration of Venezuelan Migrants and Refugees into the Peruvian Host Community through Socioeconomic Interventions to

Evaluation Requester:Korea International Cooperation Agency (KOICA)

Evaluation Manager:IOM Peru Office | Program Support Unit | Senior Monitoring and Evaluation Assistant

IOM are/unit:PSU

Duty Station:Lima, Peru

Consultant type:B

Service duration:15 weeks / 102 days / 3.5 months

Total budget:15,000 USD

Wbl:IS0137 / Task X:1:2:001

1. Evaluation Context

The political, human rights, and socio-economic situation in the Bolivarian Republic of Venezuela (hereinafter Venezuela) has triggered the departure of over 5.1 million Venezuelans to neighboring countries and beyond. Peru is the second largest recipient country and, as of August 2021, hosts around 1.1 million Venezuelan refugees and migrants living in communities with limited capacity to absorb them, particularly in terms of providing basic services and access to livelihoods.

ata collected through the 2021 Joint Needs Assessment (JNA), as well as information compiled and analyzed by the Refugee and Migrant Working Group (GTRM), aligned with the 2021 Refugee and Migrant Response Plan (RMRP), indicate that access to essential goods and services to meet basic immediate needs (Outcome 1.1)1and opportunities for socio-economic integration (Outcome 1.2)2are among the most urgent priority needs for Venezuelans residing in Peru.

The GTRM estimated that around 100,000 Venezuelan refugee and migrant children and adolescents were out of the Peruvian school system in 2020. To ensure school enrollment and prevent dropout, schools in Peru must promote safe environments free from xenophobia and discrimination (Outcome 1.3)3. Despite governmental efforts, a lack of school capacity persists, along with other barriers such as insufficient teacher capacity to provide inclusive education and a shortage of school infrastructure. Since education has been virtual in both Peru and Venezuela since March 2020, there has been no interaction between students from both countries, affecting integration efforts.

Additionally, the GTRM’s Joint Needs Assessment reported that “access to health services, including mental health, is hindered due to the lack of inclusion of refugees and migrants in the national health system.” Improving mental and emotional health plays a key role in fostering integration.

In this context, under Outcome 1.44, the project seeks to address immediate humanitarian needs exacerbated by the long-term impact of the pandemic while promoting self-sufficiency activities aimed at improving livelihoods, employment opportunities, income generation, and access to key services such as education and mental health and psychosocial support (MHPSS).

To support the integration of Venezuelan refugees and migrants into Peruvian host communities through socioeconomic interventions, the project aims to contribute to stable and inclusive community development by promoting peacebuilding between host communities and Venezuelan refugees and migrants. The intervention has a national scope and intends to improve the integration of Venezuelan migrants and refugees with their Peruvian host communities. As such, the project seeks to increase the percentage of refugees and migrants who feel economically and socially integrated.

The main outputs planned in this initiative were:

Vulnerable Migrants and refugees from Venezuela will have work experience implementing sustainable community development initiatives in their host communities through labour market inclusion (LMI) interventions in prioritized areas.

Refugees and migrants from Venezuela and members of the host community have improved entrepreneurship skills and business management knowledge

Refugee, Migrant and host communities have access to education and increased knowledge of migration issues

Vulnerable refugees and migrants have improved socio-emotional stability through inclusion and social development

The project includes two external evaluations: a mid-term evaluation and a final evaluation. The mid-term evaluation aligns with IOM’s Evaluation Policy5and guidelines6and will focus on assessing efficiency, effectiveness, and impact to provide useful recommendations for the remaining implementation period. The final evaluation will also measure relevance, coherence, and sustainability to generate recommendations for future projects.

2. Purpose and objectives of the evaluation

The performance evaluation of the project “Promoting the Integration of Venezuelan Migrants and Refugees into the Peruvian Host Community through Socioeconomic Interventions” has as its main purpose accountability to the donor, and additionally, to support learning. The evaluation will help provide evidence of both planned and unplanned results—what is working, what is not, and why. Likewise, the evaluation will help identify lessons learned and gather evidence of potential good practices, which may be adopted and implemented during the remainder of this project and in future projects, within the framework of IOM’s institutional strategy on migration, integration, and mental health.

Main objective

Evaluate the progress of the project/program concerning the established objectives, identify possible implementation adjustments, and provide recommendations to improve effectiveness and efficiency in the following project phases.

Specific objectives

Analyze the extent to which the project results have been achieved so far and whether key indicators are being met.

Evaluate the effectiveness, efficiency and impact7of the intervention proposed in the project, and assess strategies, approaches, and implementation processes in terms of performance, gender sensitivity, and inclusiveness.

Obtain recommendations, identify implementation gaps, and document independent lessons and good evidence-based practices to enhance strategies and interventions for strengthening project implementation.

Target audience of the evaluation: The implementing team, Government counterparts (e.g., Ministry of Education), Other sector directors participating in workshops, Other relevant actors, The Korea International Cooperation Agency (KOICA)

Expected uses for the evaluation:

IOM Peru Mission leadership and team will use the evaluation as a reflection document on the project and its methodology, develop and follow up on a management response matrix to the evaluation’s recommendations including corresponding updates in theProject Information and Management Application – PRIMA, and formulate and implement a communication plan for the results.8

KOICA will use the conclusions and performance ratings under each criterion to assess the value generated by the resources allocated to fund the set of activities.

Government entities directly involved, implementing partners, and other relevant stakeholders involved in the project implementation process will use the conclusions to improve the quality of their work and institutional relationship with IOM regarding the project objectives.

3. Scope of the evaluation

Temporal Scope: The implementation period of the project runs from August 2023 to December 2026. The mid-term evaluation will cover the design phase of the intervention and its implementation from August 2023 to December 2024.

Thematic -scope: The evaluation will cover all results, outputs, and activities of the project according to the Project Results Matrix (see Annex 1).

All related activities (e.g., training, fairs, campaigns, support services) will be reviewed only in terms of their contribution to achieving outputs.

Output

Key Indicator

1.1– Labour Market Inclusion

% increase in perception of refugees and migrants feeling economically/socially integrated

1.2– Entrepreneurship

% of refugees and migrants supported who initiated a job

1.3– Education and Migration Awareness

% of refugees/migrants declaring improvement in sales due to training/seed capital

1.4– Social and Emotional Stability

% of parents/legal tutors trained who know the school enrollment procedure

Geographic scope: The evaluation will consider national-level actions and complementary actions in the regions of Lima, Callao, Tumbes, and Tacna.

Cross-cutting issues: The evaluation must consider a series of cross-cutting themes relevant to the intervention, in accordance with IOM and United Nations guidelines on the matter: the Rights-Based Approach to Programming9, gender10, and disability11. These cross-cutting issues must be integrated into the evaluation questions, the design and implementation of the evaluation, and the presentation of its results.

In line with its purpose and objectives, the evaluation is expected to identify and document lessons learned and potential good practices, and present action12-oriented recommendations, all of which must be clearly supported by findings and conclusions based on triangulated evidence.

It is also important to consider potential limitations regarding the development of the mid-term evaluation. The timeline could be extended due to delays in the information-gathering process. IOM has the contact information of beneficiaries served; however, there is a 20% non-response rate, based on past experiences. To address this, IOM maintains backup databases to replace non-respondents in the sample until the desired sample size is met.

4. Criteria of Evaluation

This evaluation will apply the standard evaluation criteria for development projects, following the definitions and guidelines of the OECD-DAC (Development Assistance Committee of the Organization for Economic Co-operation and Development) as revised in 201913.

Aligned with the principles of utility of evaluation criteria, the objectives of the evaluation exercise, and the Utility Norm and Evaluability Standard of the United Nations Evaluation Group (UNEG)14, the following three criteria will be considered:

Effectiveness: The extent to which the intervention has achieved, or is expected to achieve, its objectives and results, including differentiated results among groups.

Efficiency: The extent to which the intervention delivers, or is likely to deliver, results in an economic and timely manner.

Impact: The extent to which the intervention has generated, or is expected to generate, significant effects -positive or negative, intended or unintended-at the highest level.

5. Questions of Evaluation

Below is a list of questions derived from the previously mentioned objectives and criteria. This list is not exhaustive and may be adjusted depending on the scope of the evaluation and the available resources and duration. A finalized version will be included in the inception report.

The inception report must include an evaluation matrix in which the evaluator specifies sub-questions, measurement indicators, sources of verification, and data collection and analysis methods aligned with the project performance rating matrix.

Criteria

Evaluation questions

Effectiveness

To what extent has the project achieved the proposed results, and what factors supported or hindered the observed achievement level? What were considered the project’s main achievements to date?

Were the coordination and collaboration mechanisms with national and local authorities and other project partners adequate and sufficient to facilitate the achievement of results?

How effective has the inclusion of Venezuelan migrants and refugees in project activities been in achieving the expected outcomes?

Did the indicators accurately reflect the intended results in the intervention logic? Were any adjustments needed or recommended to ensure the indicators effectively measure impact-level results?

What main challenges has the project faced in achieving its objectives? How have these challenges been addressed?

Were the risks and external factors (e.g. instutional instability) faced by the project monitored and managed throughout project implementation?

Efficiency

Were the human and financial resources invested in the project reasonable concerning the results generated?

To what extent was the project team/implementing partners able to complete activities and deliver results within the expected timeline, and why?

Were the monitoring/donors’ reports prepared relevant, reliable, and complete following donor requirements?

Impact

Is there systematic evidence of improvement in the perception of socioeconomic integration among the Venezuelan migrant population, to which the project has contributed at the national and local levels?

Has a positive change in behaviors and wellbeing been objectively observed among beneficiaries who received MHPSS (Mental Health and Psychosocial Support) assistance?

Did the project produce any unintended results, either positive or negative, in terms of migrant integration or institutional relationships?

Cross-cutting issues

Did the project design include gender-sensitive participation of diverse stakeholders in the holistic management of migrant integration in Peru?

To what extent did the project include considerations of gender equality, rights-based approaches, and disability in its design and implementation, in line with applicable IOM frameworks?

Did the project’s outputs and activities reflect differentiated needs based on age, disability, displacement, gender, nationality, migratory status, and vulnerability conditions?

Were safe and accessible spaces guaranteed for the participation of vulnerable populations?

Did beneficiaries receive clear, accessible, and culturally appropriate information about their rights and the services offered through the project?

6. Methodology and methods

This results-based evaluation will have a non-experimental design and will apply a Utilization-Focused Evaluation (UFE) approach along with a Theory-Driven Evaluation based on the project’s Theory of Change15. The evaluator is expected to propose an approach and data collection and analysis methods that enable credible, valid, and reliable answers to the evaluation questions, generate practical recommendations, and identify lessons and potential good practices from the project’s design, implementation, and monitoring processes—all while considering the cross-cutting issues mentioned earlier. Basic parameters for methodological proposal:

Data collection methods must be carefully selected to produce reasonable empirical evidence in support of the evaluation criteria, questions, and objectives.

During data analysis, disaggregation by gender, age, and disability might be ensured by incorporating specific demographic questions into data collection tools and applying filters during analysis to identify trends and disparities across these groups.

Data triangulation will be used to ensure credibility and validity of findings. Conclusions and recommendations will be validated at key stages of the evaluation process with stakeholders—without compromising the evaluator’s independence.

The proposed methodology must outline a mixed-method approach for data collection and analysis, clearly describing how different sources of evidence will be used to triangulate information.

Data Collection Methods May Include (but are not limited to):

Document review to gain an understanding of the context, as well as the documents that support the planning, implementation, and results of the intervention.

Semi-structured interviews and focus groups with stakeholders, partners, and beneficiaries.

Surveys with participants and stakeholders, if relevant.

Method

Possible Sources

Document

Review

Project documents and annual reports (2023 and 2024)

ToRs for consultancy

Training/event participant lists

Government correspondence

Reports from implementing partners and consultants

Semi-Structured

Interview

Interviews to 10-12 people:

(1-2) Key representatives from the Donor

(1) IOM project manager

(3-4) IOM team members

(5) Subject matter experts from research institutes and implementing partners

Focus Group

A focus group in-person with stakeholders and/or beneficiaries for each output (total of 4) and at least 1 per region:

Teachers from educational institutions where the project was implemented

Migrants and host community members participating in project activities

Survey

Online/in-person surveys targeting migrant and host populations

A stratified sample will be used, considering the regions of Lima and Callao, Tumbes, and Tacna. Additionally, the sample design will consider a 95% confidence level and a 0.05 margin of error.

Sample distribution:

Total sample size : 354

Lima: 266

Tumbes: 66

Tacna: 24

The modules to be considered in the instrument must be aligned with the outcome indicators:

Perception of socioeconomic integration among migrants and refugees (applied to the full sample)

Effective labor market inclusion (applied to 23% of the sample, based on service received)

Income improvements from business training/seed capital (applied to 30% of the sample)

Legal guardians of children/adolescents demonstrating awareness of school enrollment procedures (applied to 34% of the sample)

The evaluation team is encouraged to use innovative approaches for data collection and analysis. Proposals must clearly define the specific role of each methodological approach in answering the evaluation questions.

7. Ethics, Norms and Evaluation Standards

The evaluation must adhere to the standards and norms of the United Nations Evaluation Group (UNEG)16and the United Nations Principles on Data Protection and Privacy17, especially regarding interaction with project beneficiaries. The evaluator must adhere to the ethical guidelines of UNEG18and its Code of Conduct for evaluations within the United Nations system19; in particular, they must sign thePledge on Ethical Conduct in Evaluation20.

At all stages, the evaluation must adhere to the Principles of Data Protection21and the IOM Data Protection Manual22, particularly those related to the legal collection of data, obtaining informed consent from individuals, and the protection and confidentiality of personal data. Obtaining informed consent from data sources is asine qua noncondition for data collection, analysis, and use.

Both in its design, execution, and dissemination of results, the evaluation must clearly integrate the rights-based, gender, and disability approaches in accordance with the UNEG and IOM guidelines. It is expected that methodology, methods, tools, and data analysis techniques be gender sensitive. Additionally, the findings, conclusions, and recommendations should reflect an analysis from a gender perspective.

8. Deliverables

The following are the expected products and deliverables resulting from the evaluation process. All documents must meet publishable quality standards, including linguistic, structural, and semantic aspects. Also, all documents must be in Spanish rather than the final evaluation report, the information sheet and the final presentation.

Inception report, in accordance with the format indicated in Annex 1. This constitutes the detailed proposal of the evaluator and includes the evaluation analysis, the reconstruction of the Theory of Change of the response, the evaluation matrix, and the data collection instruments. Its approval by evaluation management is a requirement to proceed to the data collection stage, and its quality will be assessed using the inception report quality control tool23.

A virtual session for presenting and discussing the preliminary findings, once the data collection stage is completed and before starting the preparation of the evaluation report.

Draft Evaluation Report, in accordance with the format indicated in Annex 3, presenting the findings and conclusions related to the evaluation questions derived from the evidence found, as well as the recommendations derived from the conclusions. Its quality will be assessed using the evaluation report quality control tool24, and it is expected that its maximum length will be twenty-five (25) pages, excluding annexes, cover page, table of contents, and lists (of acronyms, tables, figures, etc.).

Final version of the Evaluation Report, once the observations presented to the draft version by the evaluation management have been considered.

In-person presentationof the evaluation findings addressed to IOM Peru and KOICA. Include a summary slide of the methodology used and the implemented instruments.

Information sheet about the evaluation, prepared according to the format indicated in Annex 3 and completed according to the guidelines in Annex 4. This does not replace the executive summary, which is part of the evaluation report.

Partially completed management response matrix by the evaluator(recommendations section), using the IOM template from Annex 7.

9. Specification and Roles

A Reference Group for the Evaluation (GRE) is established with the purpose of promoting ownership and use of the results. This group:

Will be composed of the project manager, the Regional M&E Officer, a representative of the Fund, the Regional Thematic Specialist, and the coordinator of the thematic area in the country office.

Will review the inception and final reports and provide written comments through the evaluation manager.

Will participate in the presentation of preliminary findings, validation of recommendations, and the final presentation of results by the evaluator.

Will provide information and guidance during the inception phase useful for defining the final design of the evaluation and identifying key data sources.

Evaluation Manager: Once the Terms of Reference (ToR) have been developed, the evaluation manager will coordinate with the evaluator to ensure the timely and proper delivery of expected outputs. They will gather the project information required for the evaluation and make it available to the evaluator in coordination with the Project Manager or their delegate. The manager will coordinate the distribution of evaluation products among GRE members, compile comments, and forward them to the evaluator within the agreed deadlines. They will convene the GRE for meetings to present findings and conclusions according to the established schedule. The manager will send the final versions of the evaluation documentation to the Fund and the Central Evaluation Unit (EVA) for inclusion in the global repository. They will also introduce the evaluator to the individuals participating in the evaluation, both within IOM and among other project stakeholders. The manager will assess the quality of evaluation products based on the criteria outlined in the Guidelines and may consult with the Regional M&E Officer for technical advice, if needed.

Evaluator: In accordance with the ToR, the evaluator must maintain close communication with the evaluation manager to ensure the successful and timely implementation of the evaluation. The evaluator will implement mechanisms to document the level of data triangulation supporting each finding and will clearly state the levels of available evidence in their report. The evaluator will include in the Inception Report a detailed description of the data analysis methods and their associated reliability. The evaluator will independently apply the inception report and evaluation report quality control tools before submitting the draft reports to the evaluation manager.

Donor: The Korea International Cooperation Agency (KOICA) will review the final draft report and the Evaluation Brief submitted by the evaluator to provide feedback and ensure the documents are approved and registered as delivered.

10. Work Plan

The timeframe for implementing this evaluation is a maximum of 14 calendar weeks. The evaluator may propose a shorter duration with proper justification, while ensuring that the deliverables meet the expected content and quality standards.

Below is a summary of the main milestones:

Phase 1: Inception.This phase focuses on the production of the Inception Report, based on document review and interviews with evaluation users. It includes evaluability assessment, the reconstruction of the Theory of Change (ToC), the development of the evaluation matrix, the definition of the sampling strategy, and the creation of data collection and analysis tools. The inception report might be in Spanish. Approval of the Inception Report by the evaluation manager is a prerequisite to proceed to the next phase.

Phase 2: Data Collection and Analysis. This involves the implementation of data collection activities from primary and secondary sources, according to the design and tools included in the approved inception report. Once data collection is completed, the evaluation team presents preliminary findings to the evaluation users. Then, the team applies the quantitative and qualitative analysis methods to analyze and triangulate the data collected, and derive evidence-based findings, conclusions, recommendations, lessons learned, and good practices. The preliminary findings of the data collection and analysis may be presented in a report written in Spanish. This will involve a meeting with the project team to review the findings and gather feedback for the final report.

Phase 3: Report Production and Dissemination. The evaluation team produces the first draft of the final evaluation report for review and comments by the users in Spanish. A workshop is being held to discuss the feedback on the report and to validate the recommendations. Based on this feedback, the team develops the final version of the report and an information sheet in Spanish and English. A final presentation of the results marks the conclusion of the evaluation team’s activities.

Activity

Responsible Party

Duration (Days)

Schedule

Definition of the Evaluation Reference Group

Evaluation Manager

1.00

Week 1

Kick-off meeting

Evaluation Manager / Project Manager / ROMEO

1.00

Week 1

</t

Level of Education: Bachelor Degree

Work Hours: 8

Experience in Months: No requirements