UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.

Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.

And we never give up.

For every child, HOPE

Vacancy Announcement: Consultant

 Consultancy Title: ConsultantDiagnostic review on the use of methodological approaches and evaluation methods in UN evaluations – Team Member

Section/Division/Duty Station: Evaluation, NYHQ

Duration:1 November  2024 to 31July 2025

Home/ Office Based: NYHQ / Remote

 About UNICEF – DO NOT EDIT

If you are a committed, creative professional and are passionate about making a lasting difference for children, the world’s leading children’s rights organization would like to hear from you. For 70 years, UNICEF has been working on the ground in 190 countries and territories to promote children’s survival, protection and development. The world’s largest provider of vaccines for developing countries, UNICEF supports child health and nutrition, good water and sanitation, quality basic education for all boys and girls, and the protection of children from violence, exploitation, and AIDS. UNICEF is funded entirely by the voluntary contributions of individuals, businesses, foundations and governments. UNICEF has over 12,000 staff in more than 145 countries.

BACKGROUND

Purpose of Activity/ Assignment:

The mission of the UNICEF evaluation function is to help drive results for children by fostering evidenceinformed decision[1]making. Evaluation in UNICEF supports both learning and decisionmaking, which in turn support better results for children. Evaluation also helps to hold UNICEF accountable for contributing to results for children, or for failing to do so. UNICEF’S evaluation function, along with the evaluation units of many other UN agencies, are a part of the United Nations Evaluation Group (UNE, an interagency professional network that aims to promote, strengthen and advocate for a robust, influential, independent and credible evaluation function throughout the UN system. In the past few years, as a part of these efforts to understand and promote the use of evaluative methods, the UNEG Methods Working Group has developed a compendium of methods (Issue I and Iwith illustrative examples and organized dedicated learning seminars. These endeavors received very positive informal feedback, but they were not necessarily based on systematic assessment of methods currently used or misused by the UN evaluation community. We know relatively little of the extent to which standard social science methods and specific evaluative approaches were applied in UN evaluation practicewhat are the common patterns in approaches used? what are some of the systematic issues related to the methodological approaches of UN evaluations?

In an effort to answer these questions, the UNEG Methods group seeks to hire consultant(who will carry out a diagnostic review of the different evaluation methods and approaches used across UN evaluation exercises. The overarching objective of the review will be to assess the prevalence, common patterns and trends in the use of standard social science methods (e.g., quantitative and qualitativand specific evaluation approaches. This will provide ground for stimulating a dialogue on systemlevel changes required to improve quality, credibility, and utilization within individual UN agencies, funds and programmes.

UNICEF Evaluation office is represented in the UNEG Methods Working Group through a cochair of the WG. The 2024 Work Plan for the Group has been approved by the UNEG Heads and issued by the UNEG Secretariat. This consultancy will be cochaired by the Senior Evaluation Specialist, Methods, Impact, Innovation and Learning, who is a cochair of the WG

Defining Evaluation Methods and Approaches

Evaluation methods are rooted in social science disciplines and their approaches to a scientific inquiry (e.g., psychology, economics, public polic.

The planned review will start by establishing and delineating a universe of various methods and approaches most commonly used in the global evaluation practice. To do so, it will draw on the evaluation methods typologies from available literature. It will cover scientific multimethod and mixed method concepts, and more recent evaluation methods typologies, approaches and techniques. This will be agreed upon and finalized in collaboration with the UNEG Methods group members. Once the complete list of evaluation methods and approaches is finalized, the review will assess their application across all UN evaluation reports.

 Scope of Work:

The review will aim to answer the following two broad questions:

1. What are the evaluation designs, methods, and approaches most commonly used in the UN evaluations?
2.  Have there been any patterns in the use of methods over the years, thematic areas, and the UN agencies, funds, and programmes?

The review will cover all publicly available evaluation reports produced by UN agencies within the last five years(20192023). This includes all evaluative exercises, including evaluability assessments and synthese. Methodology Machine learning approaches of Natural Language Processing (NLwill be used in this review. The exercise will be carried out in two phases: data curation and extraction, and analysis of trends in use of methods and approaches using NLP. As noted above, this will however be preceded by the finalization of the list of the universe of various methods and approaches most commonly used in the global evaluation practice.

1. Data curation and extraction: The review will use text data from evaluation reports of various UN organizations to extract information about methodologies. The UNEG Working Group Members conducted a preliminary review and identified that evaluation reports of several UN Agencies, Funds and Programmes are available online.

Analysis of trends in use of methods and approaches using NLP: After completion of text extraction and processing, NLP methods such as sentiment analysis, and dictionarybased search methods will be used to search for the different types of evaluation approaches and methods used across the UN evaluation reports. Trend analysis will be carried out to identify any insights by time, geography, agencies etc.

Deliverables shall be submitted to the WG CoChairs at agreed deadlines. Failure to submit outputs in a timely manner will result in the nullification of this agreement.

Disturbed effort between team Leads and Team Member:

Team Lead Expected time effort required: 25

Team Member Expected time effort required :

Terms of Reference / Key Deliverables:

Work Assignment Overview Deliverables/ Outputs Delivery deadline.

1. A note with the detailed methodology (to be finalized in discussions with UNEG Co[1]Chairs/member, and timelines.

• inception note with defined methodology (not more than 57 page– By 15 November 2024

2. Powerpoint with appropriate data visualizations

• Presentation – By 31 December 2024

3. Powerpoint presentation and a report (max 2530 page

• Presentation +Report – By 15 June 2025

4. Metadata and code used for the final analysis; word dictionary used for the search of different methods and approaches.

• Submission of Metadata and code – By 31 July 2025

Qualifications

Education:

Advanced university degree in evaluation, public policy, international development, social sciences or related field;

 Work experience:

Knowledge/Expertise/Skills required *:

• At least 5 years of experience in evaluation or in evaluative studies

Expertise in using Natural Language Processing for research

. • Experience in analyzing qualitative data.

• Strong analytical and writing skills.

• Excellent written and oral English.

• Working knowledge of Spanish and French is desirable

Requirements:

Completed profile in UNICEF’s eRecruitment system and

Upload copy of academic credentials

Financial proposal that will include/ reflect :

• • the costs per each deliverable and the total lumpsum for the whole assignment (in US$) to undertake the terms of reference.
  • travel costs and daily subsistence allowance, if internationally recruited or travel is required as per TOR.
  • Any other estimated costs: visa, health insurance, and living costs as applicable.
  • Indicate your availability

Any emergent / unforeseen duty travel and related expenses will be covered by UNICEF.

At the time the contract is awarded, the selected candidate must have in place current health insurance coverage.

Payment of professional fees will be based on submission of agreed satisfactory deliverables. UNICEF reserves the right to withhold payment in case the deliverables submitted are not up to the required standard or in case of delays in submitting the deliverables on the part of the consultant.

U.S. Visa information:

With the exception of the US Citizens, G4 Visa and Green Card holders, should the selected candidate and his/her household members reside in the United States under a different visa, the consultant and his/her household members are required to change their visa status to G4, and the consultant’s household members (spouswill require an Employment Authorization Card (EAto be able to work, even if he/she was authorized to work under the visa held prior to switching to G

Only shortlisted candidates will be contacted and advance to the next stage of the selection process

For every Child, you demonstrate…

UNICEF’s core values of Commitment, Diversity and Integrity and core competencies in Communication, Working with People and Drive for Results. View our competency framework at: Here

UNICEF offers reasonable accommodation for consultants/individual contractors with disabilities. This may include, for example, accessible software, travel assistance for missions or personal attendants. We encourage you to disclose your disability during your application in case you need reasonable accommodation during the selection process and afterwards in your assignment.

UNICEF has a zerotolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(and employment history. Selected candidates may be required to provide additional information to conduct a background check.

Remarks: 

Individuals engaged under a consultancy will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverag. Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants. Consultants are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws.

The selected candidate is solely responsible to ensure that the visa (applicabland health insurance required to perform the duties of the contract are valid for the entire period of the contract. Selected candidates are subject to confirmation of fullyvaccinated status against SARSCoV2 (Covid19) with a World Health Organization (WHendorsed vaccine, which must be met prior to taking up the assignment. It does not apply to consultants who will work remotely and are not expected to work on or visit UNICEF premises, programme delivery locations or directly interact with communities UNICEF works with, nor to travel to perform functions for UNICEF for the duration of their consultancy contracts.