Colombia: Final Evaluation Consultant

TERMS OF REFERENCE – EXTERNAL EVALUATION INDEPENDENT CONTRACTOR

Organizational Background

Heartland Alliance International (HAI) is the youngest and fastest growing part of Heartland Alliance for Human Needs & Human Rights, a family of organizations that has been leading anti-poverty and social justice work in Chicago for more than 125 years. HAI is comprised of nearly a dozen country offices implementing programs on a broad range of human rights issues globally, as well as the Chicago-based Marjorie Kovler Center for the Treatment of Survivors of Torture, which serves individuals from more than 50 countries. HAI has significant expertise in the fields of trauma-informed mental health care and access to justice for survivors of rights abuses. It is also an industry leader in access to high-quality and stigma-free health care. Across all of its programs, HAI promotes progressive, innovative approaches to human rights protections and gender equality.

Project Background

The Improving Health and Building Resilience (Mejorando Aceso a la Salud y Resiliencia, MAS) Colombia project is a two-year project funded by the United States Department of State (DoS) Bureau of Population, Refugees and Migration (PRM) implemented by Heartland Alliance International (HAI) together with local partner, Asociación Nacional de Afrocolombianos Desplazados (AFRODES) in areas of the Pacific Coast of Colombia that have high concentrations of Internally Displaced People (IDPs) and humanitarian emergencies. The project aims to address gaps in mental health and psychosocial support (MHPSS) services for IDPs, strengthen institutional capacity to address the needs of these individuals, and build community resilience through frontline MHPSS services and referral to other critical services including for Gender-Based Violence (GBV) survivors for the Pacific Coast (Cauca, Valle del Cauca, Chocó, Nariño). This program also seeks to increase the capacity of government institutions to respond to the MHPSS and sexual and reproductive health (SRH) needs of IDPs and strengthen the engagement of IDPs and civil society organizations (CSOs) in humanitarian coordination structures and government support mechanisms for victims. It achieves these goals through three self-reinforcing strategic objectives (SO):

SO 1: IDPs and conflict-affected communities become more resilient with increased access to mental health and psychosocial support services (MHPSS).

SO 2: Public institutions are better able to respond to MHPSS needs of IDPs and conflict victims.

SO 3: IDPs and communities impacted by internal displacement are more engaged in defending and promoting their rights.

Evaluation Rationale

HAI seeks a Contractor to conduct a final evaluation of MAS over the life of the project. The evaluation will measure the project’s effectiveness, efficiency, relevance, potential for impact, and potential for sustainability. HAI and the Contractor will collaboratively design the evaluation methodology to investigate research questions that include but are not limited to:

1. How did services provided by the Community Psychosocial Agents (CPAs) and psychologists impact the psychosocial wellbeing of beneficiaries?

   • Differentiate between individual and group interventions and office- and field-based services.

   • Explore differences based on sociodemographic variables

   • Use both qualitative and quantitative data (i.e., quantitative improvement in symptoms)

2. How effectively has the project engaged stakeholders in project implementation, including populations with special needs such as women and LGBTQ participants?

3. How effective have trainings and capacity building by project staff been at the individual and organizational/institutional level? How have individual trainees, organizations, or institutions used knowledge or skills through the life of the project?

   • Include trainings and capacity building activities with different target groups, such as external trainings as well as trainings with CPAs

4. How effective was the project in leading to improved access to and quality of care for IDPs both in terms of offering services and in building capacity/raising awareness in the public sector, including with public entities and civil society organizations? What internal and external factors contributed to this impact?

5. What are the key project achievements that will be sustained beyond the life and the context of the project at the community, organizational, and participant level?

The evaluation will be shared with both HAI and the donor and will help inform the implementation of HAI’s humanitarian response and mental health programming both locally and worldwide. The external evaluation will complement HAI’s internal assessment of the project, and HAI will share the evaluation with staff, partners, and other stakeholders as necessary.

1. Evaluation Design & Methodology

The design and methodology applied to this evaluation should generate the highest quality and most credible evidence to answer the evaluation questions listed above, and should primarily focus on assessing project activities implemented under MAS. Prior to the start of data collection, the Contractor will develop and present, for HAI review and approval, a written methodology that details the evaluation activities, evaluation tools, and data analysis plan for this evaluation. The written methodology should be guided by a participatory approach to evaluation, and will be comprised of a mix of quantitative and qualitative tools appropriate to the evaluation’s research questions. HAI expects the methodology to include, but not be limited to, the following activities:

• A desk review of relevant information on the status of MHPSS services for IDPs in Colombia, the social and political context of the target population, and strategies to measure community-based MHPSS interventions in both stable and emergency settings;
• A review of all relevant project documentation (in English and Spanish);
• A review of project data;
• Site visits;
• Data collection methods and tools, including a pre-selection of key informant interviews;
• Data analysis plan. Evaluation questions should be answered with information that is disaggregated by sociodemographic characteristics, location, and different types of services.

HAI will provide the evaluation team access to all relevant project documents, including but not limited to:

• The original grant agreement and subsequent modifications to the grant agreement;
• Quarterly reports, work plans and management reviews developed as part of routine project monitoring; and

• All data, including data collection tools, previously collected from routine project monitoring.

• Evaluation Deliverables

• Evaluation Methodology & Work Plan: A written, draft methodology for the evaluation will be submitted to HAI for review and approval no later than the 7^th day following the signing of this agreement. The methodology plan should include the evaluation design, data collection plan, instruments, tools, and an operational work plan, including a timeline. The methodology plan will be discussed with HAI prior to any data collection activities. The Contractor may change the methodology but must submit any changes for approval to HAI prior to implementation.

• Evaluation Data: All raw data used in the evaluation analysis should be submitted no later than two weeks after data collection activities are completed. The Contractor will provide data in an electronic file in an easy to read format; organized and fully documented for use by those not familiar with the project or evaluation.

• Draft Evaluation Report: The Contractor will submit a draft report of the findings and recommendations from the evaluation no later than August 9, 2019. Prior approval by the Company will be required should the Contractor want to revise this deadline. The format of the evaluation report should follow the guidelines provided by HAI. The report must be submitted in English, electronically.

• Debriefing with HAI and partner staff: The Contractor will present major accomplishments and findings from the evaluation to HAI and partner staff through PowerPoint presentation after completion of field-based data collection. The debriefing will include a discussion of achievement and issues as well as any recommendation the team has for possible modifications to project approaches, results, or activities. The Contractor will consider HAI project staff comments and revise the evaluation report accordingly, as appropriate.

• Final Report: HAI will provide the Contractor with written comments on the Draft Evaluation Report no later than August 23, 2019. The Contractor will submit a Final Evaluation Report that incorporates HAI’s responses to the Draft Evaluation Report no later than September 6, 2019. The format of the evaluation report should follow the guidelines provided to the Contractor by HAI. The final version of the Final Evaluation Report will be submitted in English, electronically no later than September 13, 2019. Please note that these dates are non-negotiable.

• Evaluation Team Composition & Desired Qualifications

The Contractor will receive support from HAI’s project team, as well as HAI’s Monitoring & Evaluation team. The Contractor should possess the following qualifications:

• Fluency in Spanish and English preferred. Applicants who are fluent in Spanish only or English only will also be considered. Applicants who are fluent in English only must include costs for translation services by a certified translator for all data collection activities in the evaluation budget. Applicants who are fluent in Spanish only must include costs for translation of all deliverables into English by a certified translator in the evaluation budget;
• Outstanding oral and written communication skills;
• At least four years demonstrated experience in conducting evaluations and writing reports;
• Demonstrated experience evaluating MHPSS programming;
• Demonstrated experience evaluating humanitarian programming and/or demonstrated knowledge of the key challenges and actors associated with providing services for Internally Displaced People (IDPs) in Colombia;

• Experience evaluating US government-funded programs preferred.

• Evaluation Management

HAI will provide overall supervision to the Contractor during contract period. HAI will assist the Contractor with finalizing the evaluation methodology and work plan, and assist in identifying and providing key project documents for the Contractor to review during the contract period. HAI will provide overall logistical support and assistance to the Contractor, as needed and when possible. HAI’s project-based staff will be made available to the Contractor for consultations during the evaluation process.

The Contractor is responsible for arranging and completing all evaluation-related activities listed in the approved methodology and work plan, in coordination with HAI’s local project and logistics staff. The Contractor will advise HAI prior to each of those activities. The Contractor will be responsible for completing and submitting to the Company the evaluation products listed in Section E of this document. The Contractor will be responsible for procuring his/her own work/office space, computers, Internet access, printing, and photocopying throughout the evaluation. The Contractor will be required to make his/her own payments for such services. The Contractor will be responsible for any expenses beyond the fee provided by the Company as determined in this document.

1. Payment Schedule

   Payment will be tied to deliverables. A final payment schedule will be specified in the final signed Contractor Agreement.

I.**Submission Process**

To be considered for this contract, please submit the following to alblack@heartlandalliance.org:

1. A letter of interest, indicating how your skills and experience meet the qualifications listed above;

2. A résumé or CV (no more than four pages);

3. A writing sample (no more than three pages);

4. A timeline of activities to be completed during the evaluation, including the dates and deliverables listed in Section E of this TOR;

5. A draft budget, including consultation fees and all other evaluation-related expenses, including travel, with a maximum amount of $15,000 USD.

Do NOT submit an application through the Heartland Alliance career website. Such submissions will not be considered.

If the applicant is fluent in English and Spanish, the submission will be expected to be in English. If the applicant is not fluent in English, the application may be submitted in Spanish, given that a budget for translation is included within the $15,000 budget. Incomplete submission packages and submission packages that do not adhere to the page limits stated above will not be considered. Present employees of HAI or HAI’s implementing partners need not apply.

US Based:

https://chu.tbe.taleo.net/chu01/ats/careers/requisition.jsp?org=HEARTLANDALLIANCE&cws=1&rid=5930

International:https://chu.tbe.taleo.net/chu01/ats/careers/requisition.jsp?org=HEARTLANDALLIANCE&cws=38&rid=5930