Mid-Term Review of Integrated Ecosystem Management and Restoration of South East Coast Forests

Department of Sustainable Development, Saint Lucia

Terms of Reference

Mid-Term Review of the UNEP/GEF6-Intergrated Ecosystem Management and Restoration of The Forests on the South East Coast

Job Title	Mid-Term Review of the UNEP/GEF-6 project- Integrated Ecosystem Management and Restoration of The Forests of the South East Coast
Contracting Authority	Department of Sustainable Development, Ministry of Education, Sustainable Development, Science, Technology and Vocational Training, Saint Lucia
Date of Issue	August 2023
Deadline of Issue	September 2023
Duration	4 months
To Apply	Interested persons are invited to apply for this opportunity. Please email the Chairperson of the Department Public Procurement Committee at [email protected] the following: Submit the following: 1. Cover Letter 2. Curriculum Vitae 3. Technical Proposal on how the objectives outlined are to be achieved 4. A work plan showing the timeline for expected deliverables 5. Writing sample (copy of a published report authored by the applicant) 6. Financial Proposal 7. Contact information for three (3) references Please use the e-mail subject line: “Application for Mid-Term Review of the UNEP/GEF- 6 Integrated Ecosystem Management and Restoration of Forests in the South East Coast”
EQUAL EMPLOYMENT OPPORTUNITY (EEO)	The Department of Sustainable Development (DSD) – Saint Lucia provides equal opportunity and fair and equitable treatment in employment to all people without regard to race, colour, religion, sex, national origin, age, disability, political affiliation, marital status, or sexual orientation. The DSD also strives to achieve equal employment opportunity in all personnel operations through continuing diversity enhancement programs.
LATE BIDS	Late bids will not be acknowledged.

Terms of Reference

Mid-Term Review of the Integrated Ecosystem Management and Restoration of The Forests on the South East Coast

1. Project General Information

Table 1. Project summary

UNEP Sub- programme:	Healthy and Productive Ecosystems	UNEP Division/Branch:	Ecosystems Division
Expected Accomplishment(s):	3(a) The health and productivity of marine, freshwater and terrestrial ecosystems are institutionalized in education, monitoring frameworks at the national and international levels EA3(b) Policymakers in the public and private sectors test the inclusion of the health and productivity of ecosystems in economic decision-making	Programme of Work Output(s):	(i) Increase in the number of countries that have made progress to monitor and maintain the health and productivity of marine and terrestrial ecosystems (ii) Increase in the number of countries that demonstrate enhanced knowledge of the value and role of ecosystem services (i) Increase in the number of public sector institutions that test the incorporation of the health and productivity of marine and terrestrial ecosystems in economic decision-making
SDG(s) and indicator(s)	SDG 6; Targets 6.3, 6.6 SDG 7; Target 7.b SDG 14; Targets 14.1, 14.2, 14.5. 14.7 SDG 15; Targets 15.1, 15.2, 15.3, 15.5
GEF Core Indicator Targets (identify these for projects approved prior to GEF-7)	• 1&2. Terrestrial/marine protected areas created or under improved management for conservation and sustainable use (Hectares): 4,000 ha (new land and sea protected areas) • 3. Area of land restored (Hectares): 2,500 ha • 4. Area of landscapes under improved practices (excluding protected areas)(Hectares): 5,000 ha • 5. Area of marine habitat under improved practices (excluding protected areas) (Hectares): 500 ha rehabilitated • 6. Greenhouse Gas Emissions Mitigated (metric tons of CO2e): 1,458,871 metric tons • 11. Number of direct beneficiaries disaggregated by gender as co-benefit of GEF investment: Not defined
Dates of previous project phases:		Status of future project phases:

FROM THE PROJECT‘S PIR REPORT (use latest version) :

Project Title:

Integrated Ecosystem Management and Restoration of Forests on the South East Coast

Executing Agency:

Department of Sustainable Development, Ministry of Education, Sustainable Development, Innovation, Science, Technology and Vocational Training

Project partners:

Ministry of Agriculture, Ministry of Tourism, Department of Physical Planning, Ministry of Social Equity, Energy?

Geographical Scope:

National

Participating Countries:

Saint Lucia

GEF project ID:	9406	IMIS number*1:	GFL/11207-14AC0003-SB- 006383
Focal Area(s):	Biodiversity	GEF OP #:
GEF Strategic Priority/Objective:	CCM1 Programme-1; CM2 Programme-4; BD1 Programme-2; LD2 Programme-3; SFM-3	GEF approval date*:	27 September 2018
UNEP approval date:	27 September 2018	Date of first disbursement*:	30 July 2019
Actual start date2:	February 2019	Planned duration:	60 months
Intended completion date*:	31 December 2023	Actual or Expected completion date:	31 December 2023
Project Type:	Full Size Project	GEF Allocation*:	USD 4,428, 145
PPG GEF cost*:	USD	PPG co-financing*:
Expected MSP/FSP Co- financing*:	USD	Total Cost*:	USD 4, 428,145
Mid-term Review/eval. (planned date):	August 2021	Terminal Evaluation (actual date):	N/A
Mid-term Review/eval. (actual date):		No. of revisions*:	3
Date of last Steering Committee meeting:	26 April 2023	Date of last Revision*:	01 January 2022
Disbursement as of 31 December 2022*:	US 1,258,024.50	Date of financial closure*:	31 December 2024
Date of Completion3*:	31 December 2023	Actual expenditures reported as of 31 December 20224:	USD1,047,383.51
Total co-financing realized as of 31December 2022	USD 64,044.16	Actual expenditures entered in UMOJA as of 30 September 2023*:	N/A
Leveraged financing:5

_________________
¹ Fields with an * sign (in yellow) should be filled by the Fund Management Officer
² Only if different from first disbursement date, e.g., in cases were a long time elapsed between first disbursement and recruitment of project manager.
³ If there was a “Completion Revision” please use the date of the revision.
⁴ Information to be provided by Executing Agency/Project Manager
⁵ See above note on co-financing

2. Project Rationale

The Integrated Ecosystem Management and Restoration of Forests in the South East Coast Project is a five- year project that seeks to address the lack of integrated protection and sustainable management of ecosystems (forests, mangroves, seagrass beds) in the South East coastal areas, which provide livelihoods, ecosystem services, buffers against climate change and extreme events, and sources of economic growth. Without sound management, the economic development of the region could lead to the irreversible degradation of ecological services, which in turn will undermine efforts to achieve growth and could lead to a gradual impoverishment of local communities. The project will use innovative financing to enable the private sector and NGOs to participate in the management and sustainable use of protected areas.

3. Project Results Framework

Project Objectives and Components

i. Component 1 Ecosystem Management Strengthening: This is to establish effective ecosystems management mechanisms. For this to occur, the project must facilitate information-generation on ecosystems, species, and ecological services that exist in the region, so that economic decision-making is informed by a consideration of real environmental costs.

ii. Component 2 Rehabilitated Landscapes: The project will seek to rehabilitate and further protect degraded landscapes, based on improved mechanisms for land use planning and collaborative investment decision-making.

iii. Component 3 Sustainable Livelihoods: the project design acknowledges that no intervention will be successful unless it creates economic opportunities, incentives and livelihoods for the primary natural resources users. The project will therefore invest in working with communities to support sustainable natural- resource based livelihoods and will support communities to access resources, including the introduction of renewable energy technologies at the community level, capacity building, and inputs to successfully engage in such activities. Activities will be tailored to render tourism and agricultural activities, as the two major sectors contributing to the use of natural resources, more sustainable, and therefore contribute to the maintenance of ecological integrity of the area.

4. Executing Arrangements:

 This project is being executed by the Department of Sustainable Development on behalf of the Government of Saint Lucia, with UNEP as Implementing Agency (IA). The Department of Sustainable Development (DSD) within the Ministry of Education, Innovation, Sustainable Development, Science, Technology and Vocational Training acts as the project`s Executing Agency (EA), with responsibility for project execution at the national level. Overall project supervision is the responsibility of UNEP as Implementing Agency (IA), and UNEP`s Task Manager (TM) and Programme Assistant (PA) provide support and works closely with DSD personnel, who carry out all project management related issues.

5. Project Cost and Financing

The project falls under the full size project (FSP) category, with an overall project budget of USD 19,053,136 million comprised of a GEF allocation of USD 4,428,145 and co-financing support of USD 14,624,991 from various partners, both in cash and in-kind. The table below shows the itemised budget by component and funding source.

Project Components	(in $)
	GEF Project Financing	Co-financing
Component 1: Ecosystems Management	1,267,000	3,441,622
Component 2: Restored/Rehabilitated Productive Landscapes	1,686,000	7,600,000
Component 3: Sustainable Livelihood	1,141,545	3,029,045
PMC	170,000	248,000
M&E	163,100	86,000
Total Project Cost	4,428,145	14,624.991

6. Implementation Issues

The project has faced challenges during the first phase of implementation, some of which have been actioned, where the Department of Sustainable Development continues to monitor and address as per below:

Problems identified	Actions taken
The initial Covid 19 spikes in Saint Lucia caused several activities such as meetings with collaborating agencies, and stakeholders to be postponed.	The PIU was able to organise virtual meetings with several collaborating agencies and some stakeholders.
The Covid 19 situation also caused delays in receiving responses from officers in collaborating agencies as some were quarantined and other officers were ill.	Work plans reorganized and deliverables deadlines adjusted. The PIU also used virtual meeting to hold meetings with stakeholders.
The procurement of consultants took much longer than envisaged due to the lengthy process involved in contracting consultants.	More direct communication established with Central Public Procurement Board and Attorney General’s Chambers.
The procurement of supplies also took longer than anticiptes as the collaborating agencies were not familiar with the procurement process of the PIU. Delays were also experienced as suppliers were slow to respond to requests and the suppliers in some cases did not provide the correct documentation or documentation provided did not have the correct information or was missing information.	The PIU held a meeting with the collaborating agencies where a presentation on the steps involved in the process was discussed. Documentation on these steps was also provided to assist the agencies in becoming more efficient in using the process. Closer collaboration with the Accounts section. The PIU was able to meet with the suppliers and identified where information needed was missing or erroneous on documents to be submitted. The PIU also informed the collaborating agencies on common errors made by suppliers when providing documentation so that in reviewing the documents they would be able to spot errors and provide advice on the correct information to be supplied.

Section 2. OBJECTIVE AND SCOPE OF THE MID-TERM REVIEW

7. Objective of the Review

In line with the UNEP Evaluation Policy6 and the UNEP Programme Manual7, the Mid-Term Review is undertaken approximately half-way through project implementation to analyze whether the project is on-track, what problems or challenges the project is encountering, and what corrective actions are required. The MTR will assess project performance to date (in terms of relevance, effectiveness and efficiency), and determine the likelihood of the project achieving its intended outcomes and supporting their sustainability.

8. Key Review Principles

Mid-Term Review findings and judgements will be based on sound evidence and analysis, clearly documented in the Review Report. Information will be triangulated (i.e. verified from different sources) as far as possible, and when verification is not possible, the single source will be mentioned (whilst anonymity is still protected). Analysis leading to evaluative judgements should always be clearly spelled out.

_______________
⁶ https://www.unenvironment.org/about-un-environment/evaluation-office/policies-and-strategies
⁷ https://wecollaborate.unep.org

As this Review is being undertaken at the mid-point of project implementation, particular attention will be given to identifying implementation challenges and risks to achieving the expected project objectives and sustainability, which will support potential course correction. Possible questions to be considered include:

• Does the Theory of Change (TOC) properly reflect the project’s intended change process?

• Is the stakeholder analysis still appropriate and adequate to support the project’s ambitions?

• Are results statements in keeping with both UNEP and GEF definitions (e.g. outcomes are expressed as the uptake or use of outputs)

• Are roles and responsibilities commonly understood and playing out effectively?

• Is there an effective monitoring mechanism for the project’s implementation (this is separate from, and supports, reporting in the annual PIR)?

• Is the rate of expenditure appropriate for the mid-point?

• Have plans for inclusivity (human rights, gender considerations, disability inclusion etc) been implemented as planned, or does more need to be done?

• Are safeguard identification and mitigation plans being monitored and steps taken to minimize negative effects?

• Is there an exit strategy in place and are the elements needed for the project’s benefits to be sustained after the project end, being incorporated in the project implementation?

• Have recommendations from previous performance assessments (where they exist) been appropriately addressed?

• (Where relevant) What changes were made to adapt to the effects of COVID-19 and how might any changes affect the project’s performance?

A Mid-Term Review is a formative assessment, which requires that the consultants go beyond the assessment of “what” the project performance is and make a serious effort to provide a deeper understanding of “why” the performance is as it is. (i.e. what is contributing to the achievement of the project’s results). This should provide the basis for the lessons that can be drawn from the project at the mid-point and the recommendations that support adaptive management for the remainder of the project.

Attribution, Contribution and Credible Association: In order to attribute any outcomes and impacts to a project intervention, one needs to consider the difference between what has happened with, and what would have happened without, the project (i.e. take account of changes over time and between contexts in order to isolate the effects of an intervention). This requires appropriate baseline data and the identification of a relevant counterfactual, both of which are frequently not available for evaluations. Establishing the contribution made by a project in a complex change process relies heavily on prior intentionality (e.g. approved project design documentation, logical framework) and the articulation of causality (e.g. narrative and/or illustration of the Theory of Change). Robust evidence that a project was delivered as designed and that the expected causal pathways developed supports claims of contribution and this is strengthened where an alternative theory of change can be excluded. A credible association between the implementation of a project and observed positive effects can be made where a strong causal narrative, although not explicitly articulated, can be inferred by the chronological sequence of events, active involvement of key actors and engagement in critical processes.

A key aim of the Mid-Term Review is to encourage reflection and learning by UNEP staff, the Executing Agency and key project stakeholders. The Review Consultant should consider how reflection and learning can be promoted, both through the review process and in the communication of review findings and key lessons. Clear and concise writing is required on all review deliverables. There may be several intended audiences, each with different interests and needs regarding the report. The Task Manager will plan with the Review Consultant which audiences to target and the easiest and most effective way to communicate the key review findings and lessons to them. This may include some or all of the following: a webinar, conference calls with relevant stakeholders, the preparation of a review brief or interactive presentation. Draft and final versions of the Main Review Report will be shared with key stakeholders by the Task Manager and a copy of the final version will be submitted to the UNEP Evaluation Office, who will provide an assessment of the quality of the Review Report based on a standard UNEP template.

9.  Key Strategic Questions

In addition to the evaluation criteria outlined in Section 10 below, the Review will address the strategic questions listed below. These are questions of interest to UNEP and to which the project is believed to be able to make a substantive contribution. Also included are three questions that are required when reporting in the GEF Portal and these must be addressed in the MTR:

• To what extent is the project following a robust theory of change and capacity building with appropriately formulated outputs, direct outcomes, intermediate states and long-term results?

• What revisions are required to ensure that implementation can be effectively evaluated at the end the project? This includes consideration of whether the outcome indicators are verifiable and appropriate for recording progress towards the achievement of the development objectives.

• The extent to which the design and implementation of the models that are to demonstrate viable forest management (SFM) livelihoods under Component 3 are meeting the intended impact with the likelihood of sustainability and scaling-up.

Address the questions required for the GEF Portal in the appropriate parts of the report and provide a summary of the findings in the Conclusions section of the report:

a) Under Monitoring and Reporting/Monitoring of Project Implementation:
What is the performance at the project’s mid-point against Core Indicator Targets? (For projects approved prior to GEF-7, these indicators will be identified retrospectively and comments on performance provided).

b) Under Factors Affecting Performance/Stakeholder Participation and Cooperation:
What has been the progress, challenges and outcomes regarding engagement of stakeholders in the project/program? (This should be based on the description included in the Stakeholder Engagement Plan or equivalent documentation submitted at CEO Endorsement/Approval)

c) Under Factors Affecting Performance/Responsiveness to Human Rights and Gender Equality:
What has been the progress, challenges and outcomes regarding gender-responsive measures and any intermediate gender result areas? (This should be based on the documentation at CEO Endorsement/Approval, including gender-sensitive indicators contained in the project results framework or gender action plan or equivalent)

d) Under Factors Affecting Performance/Environmental and Social Safeguards:
What has been the experience at the project’s mid-point against the Safeguards Plan submitted at CEO Approval? The risk classifications reported in the latest PIR report should be verified and any measures taken to address identified risks assessed. (Any supporting documents gathered by the Consultant during this review should be shared with the Task Manager for uploading in the GEF Portal)

e) Under Factors Affecting Performance/Communication and Public Awareness:
What has been the progress, challenges and outcomes regarding the implementation of the project’s Knowledge Management Approach, including: Knowledge and Learning Deliverables (e.g. website/platform development); Knowledge Products/Events; Communication Strategy; Lessons Learned and Good Practice; Adaptive Management Actions. (This should be based on the documentation approved at CEO Endorsement/Approval)

10. Evaluation Criteria

All evaluation criteria will be rated on a six-point scale. Sections A-G below, outline the scope of the criteria and a link to a table for recording the ratings is provided in Annex 1. A weightings table will be provided in excel format (see notes in Annex 1) to support the determination of an overall project rating.

A.  Strategic Relevance

The Review will assess the extent to which the activity is suited to the priorities and policies of the target group, recipient and donor. The Review will include an assessment of the project’s relevance in relation to UNEP’s mandate and its alignment with UNEP’s policies and strategies at the time of project approval, as well as each country’s UNDAF. Under strategic relevance an assessment of the complementarity of the project with other interventions addressing the needs of the same target groups will be made. This criterion comprises four elements:

i. Alignment to the UNEP Medium Term Strategy8 (MTS), Programme of Work (POW) and Strategic Priorities

The Review should assess the project’s alignment with the MTS and POW under which the project was approved and include, in its narrative, reflections on the scale and scope of any contributions made to the planned results reflected in the relevant MTS and POW. UNEP strategic priorities include the Bali Strategic Plan for Technology Support and Capacity Building9 (BSP) and South-South Cooperation (S-SC). The BSP relates to the capacity of governments to: comply with international agreements and obligations at the national level; promote, facilitate and finance environmentally sound technologies and to strengthen frameworks for developing coherent international environmental policies. S-SC is regarded as the exchange of resources, technology and knowledge between developing countries.

ii. Alignment to Donor/Partner Strategic Priorities

Donor strategic priorities will vary across interventions. The Review will assess the extent to which the project is suited to, or responding to, donor priorities. In some cases, alignment with donor priorities may be a fundamental part of project design and grant approval processes while in others, for example, instances of ‘softly-earmarked’ funding, such alignment may be more of an assumption that should be assessed.

iii. Relevance to Global, Regional, Sub-regional and National Environmental Priorities

The Review will assess the alignment of the project with global priorities such as the SDGs and Agenda 2030. The extent to which the intervention is suited, or responding to, the stated environmental concerns and needs of the countries, sub-regions or regions where it is being implemented will be considered. Examples may include: UN Development Assistance Frameworks (UNDAF) or, national or sub-national development plans, poverty reduction strategies or Nationally Appropriate Mitigation Action (NAMA) plans or regional agreements etc. Within this section consideration will be given to whether the needs of all beneficiary groups are being met and reflects the current policy priority to leave no-one behind.

iv. Complementarity with Existing Interventions/Coherence¹⁰

An assessment will be made of how well the project, either at design stage or during the project inception or mobilization11, took account of ongoing and planned initiatives (under the same sub-programme, other UNEP-programmes, or being implemented by other agencies within the same country, sector or institution) that address similar needs of the same target groups. The Review will consider if the project team, in collaboration with Regional Offices and Sub-Programme Coordinators, made efforts to ensure their own intervention was complementary to other interventions, optimized any synergies and avoided duplication of effort. Examples may include work within UNDAFs or One UN programming. Linkages with other interventions should be described and instances where UNEP’s comparative advantage has been particularly well applied should be highlighted.

B. Effectiveness

The Review will assess effectiveness across three dimensions: delivery of outputs, achievement of project outcomes and, where appropriate and feasible, likelihood of impact. At the mid-point more emphasis is placed on performance at the output and outcome levels, but observations about likelihood of impact may be helpful for course correction or adjusting the emphasis of the project’s efforts.

___________________
⁸ UN Environment’s Medium Term Strategy (MTS) is a document that guides UN Environment’s programme planning over a four-year period. It identifies UN Environment’s thematic priorities, known as Sub-programmes (SP), and sets out the desired outcomes, known as Expected Accomplishments (EAs), of the Sub-programmes. https://www.unenvironment.org/about-un-environment/evaluation-office/our-evaluation- approach/un-environment-documents
⁹ http://www.unep.fr/ozonaction/about/bsp.htm
¹⁰ This sub-category is consistent with the new criterion of ‘Coherence’ introduced by the OECD-DAC in 2019.
¹¹ A project’s inception or mobilization period is understood as the time between project approval and first disbursement. Complementarity during project implementation is considered under Efficiency, see below.

i. Availability of Outputs¹²
The Review will assess the project’s success in producing the programmed outputs and achieving targets and milestones as per the project design document (ProDoc). Any formal modifications/revisions made during project implementation will be considered part of the project design. Where the project outputs are inappropriately or inaccurately stated in the ProDoc, a table should be provided showing the original formulation and the amended version for transparency. The delivery of outputs will be assessed in terms of both quantity and quality, and the assessment will consider their usefulness and the timeliness of their provision. It is noted that emphasis is placed on the performance of those outputs that are most important to achieve outcomes. The Review will briefly explain the reasons behind the success or shortcomings of the project in delivering its programmed outputs and meeting expected quality standards.

ii. Achievement of Project Outcomes¹³
The achievement of project outcomes is assessed as performance against the project outcomes defined in the Project Results Framework14. These are outcomes that are intended to be achieved by the end of the project timeframe and within the project’s resource envelope. Emphasis is placed on the achievement of project outcomes that are most important for attaining intermediate states. As with outputs, a table can be used where substantive amendments to the formulation of project outcomes is necessary to make them consistent with UNEP guidelines. Where possible, the Review should report evidence of attribution, contribution or credible association between UNEP’s intervention and the project outcomes.

iii. Likelihood of Impact
Based on the articulation of longer-term effects as defined in the project objective or stated intentions, the Review will, where possible, assess the likelihood of the intended, positive impacts becoming a reality.

The Review will also consider the likelihood that the intervention may lead, or contribute, to unintended negative effects (e.g. will vulnerable groups such as those living with disabilities and/or women and children, be disproportionally affected by the project?). Some of these potential negative effects may have been identified in the project design as risks or as part of the analysis of Environmental, Social and Economic Safeguards¹⁵. The Review will consider the extent to which the project is playing a catalytic role or is promoting longer-term scaling up and/or replication¹⁶.

C. Financial Management

Under financial management the Mid-Term Review will assess: a) whether the rate of spend is consistent with the project’s length of implementation to-date, the agreed workplan and the delivery of outputs and b) whether financial reporting and/or auditing requirements are being met consistently and to adequate standards by all parties. This includes an assessment of whether UNEP’s financial management policies and the GEF’s fiduciary standards are being met. Any financial management issues that are affecting the timely delivery of the project or the quality of its performance will be highlighted.

¹² Outputs are the availability (for intended beneficiaries/users) of new products and services and/or gains in knowledge, abilities and awareness of individuals or within institutions (UNEP, 2019)
¹³ Outcomes are the use (i.e. uptake, adoption, application) of an output by intended beneficiaries, observed as changes in institutions or behavior, attitude or condition (UNEP, 2019)
¹⁴ UNEP staff are currently required to submit a Theory of Change with all submitted project designs. The level of ‘reconstruction’ needed during an evaluation will depend on the quality of this initial TOC, the time that has lapsed between project design and implementation (which may be related to securing and disbursing funds) and the level of any changes made to the project design. In the case of projects pre-dating 2013 the intervention logic is often represented in a logical framework and a TOC will need to be constructed in the inception stage of the evaluation.
¹⁵ Further information on Environmental, Social and Economic Safeguards (ESES) can be found at http://wedocs.unep.org/handle/20.500.11822/8718http://www.unep.org/about/eses/
¹⁶ Scaling up refers to approaches being adopted on a much larger scale, but in a very similar context. Scaling up is often the longer- term objective of pilot initiatives. Replication refers to approaches being repeated or lessons being explicitly applied in new/different contexts e.g. other geographic areas, different target group etc. Effective replication typically requires some form of revision or adaptation to the new context. It is possible to replicate at either the same or a different scale.

D.  Efficiency

The Review will assess the cost-effectiveness and timeliness of project execution. Focusing on the translation of inputs into outputs, cost-effectiveness is the extent to which an intervention has achieved, or is expected to achieve, its results at the lowest possible cost. Timeliness refers to whether planned activities were delivered according to expected timeframes as well as whether events were sequenced efficiently. The Review will describe any cost or time-saving measures put in place to maximize results within the secured budget and agreed project timeframe and consider whether the project is being implemented in the most efficient way compared to alternative interventions or approaches. The Review will also assess ways in which potential project extensions can be avoided through stronger project management.

E.  Monitoring and Reporting

The Review will assess monitoring and reporting across two sub-categories: monitoring of project implementation, and project reporting.

i. Monitoring of Project Implementation
Each project should be supported by a sound monitoring plan that is designed to track progress against SMART17 results towards the achievement of the project’s outputs and outcomes, including at a level disaggregated by gender, marginalisation or vulnerability, including those living with disabilities. The Review will assess the use and quality of the monitoring plan. In particular, the evaluation will assess the relevance and appropriateness of the project indicators as well as the methods used for tracking progress against them as part of conscious results-based management. This assessment will include consideration of whether the project gathered relevant and good quality baseline data that is accurately and appropriately documented. The Review will assess whether the monitoring system is operational and facilitates the timely tracking of results and progress towards project milestones and targets throughout the project implementation period. It will also consider the quality of the information generated by the monitoring system during project implementation and how it was used to adapt and improve project execution, achievement of outcomes and ensure sustainability. The Review should confirm that funds allocated for monitoring are being used to support this activity.

The performance at the project’s mid-point against Core Indicator Targets should be reviewed. For projects approved prior to GEF-7, these indicators will be identified retrospectively and comments on performance provided.

ii. Project Reporting
Projects funded by GEF have requirements with regard to verifying documentation and reporting (i.e., the Project Implementation Reviews, Tracking Tool and CEO Endorsement template18), which will be made available by the Task Manager. The Review will assess the extent to which both UNEP and GEF reporting commitments have been fulfilled. Where corrective action is indicated in the annual Project Implementation Review reports (e.g. as an identified risk), the Review Consultant will record whether this action has been taken.

F. Sustainability

Sustainability¹⁹ is understood as the probability of the benefits associated with the project outcomes being maintained and developed after the close of the intervention. The Review will identify and assess the key conditions or factors that are likely to undermine or contribute to the persistence of benefits at the outcome level. Some factors of sustainability may be embedded in the project design and implementation approaches while others may be contextual circumstances or conditions that evolve over the life of the intervention. Where applicable an assessment of bio-physical factors that may affect the sustainability of project outcomes may also be included.

¹⁷ SMART refers to results that are specific, measurable, achievable, relevant and time-oriented. Indicators help to make results measurable.
¹⁸ The Consultant(s) should verify that the annual Project Implementation Reviews have been submitted, that the Tracking Tool is being kept up-to- date and that in the CEO Endorsement template Table A and Section E have been completed.
¹⁹ As used here, ‘sustainability’ means the long-term maintenance of outcomes and consequent impacts, whether environmental or not. This is distinct from the concept of sustainability in the terms ‘environmental sustainability’ or ‘sustainable development’, which imply ‘not living beyond our means’ or ‘not diminishing global environmental benefits’ (GEF STAP Paper, 2019, Achieving More Enduring Outcomes from GEF Investment)

The Review will ascertain that the project has put in place an appropriate exit strategy and measures to mitigate risks to sustainability. The Review Consultant will consider: a) the level of ownership, interest and commitment among government and other stakeholders to take the project achievements forwards, b) the extent to which project outcomes are dependent on future funding for the benefits they bring to be sustained and c) the extent to which the sustainability of project outcomes is dependent on issues relating to institutional frameworks and governance. It will consider whether institutional achievements such as governance structures and processes, policies, sub-regional agreements, legal and accountability frameworks etc. are robust enough to continue delivering the benefits associated with the project outcomes after project closure.

G. Factors Affecting Project Performance and Cross-Cutting Issues

These factors are rated in the ratings table but can be discussed as cross-cutting themes as appropriate under the other evaluation criteria, above. Where the issues have not been addressed under other evaluation criteria, the consultant(s) will provide summary sections under the following headings)

i. Preparation and Readiness
This criterion focuses on the inception or mobilisation stage of the project. The Review will assess whether appropriate measures were taken to either address weaknesses in the project design or respond to changes that took place between project approval, the securing of funds and project mobilisation. In particular the Review will consider the nature and quality of engagement with stakeholder groups by the project team, the confirmation of partner capacity and development of partnership agreements as well as initial staffing and financing arrangements.

ii. Quality of Project Implementation and Execution
Specifically, for GEF funded projects, this factor refers separately to the performance of the Executing Agency and the technical backstopping and supervision provided by UNEP, as the Implementing Agency.

The Review will assess the effectiveness of project management with regard to: providing leadership towards achieving the planned outcomes; managing team structures; maintaining productive partner relationships (including Steering Groups etc.); maintaining project relevance within changing external and strategic contexts; communication and collaboration with UNEP colleagues; risk management; use of problem-solving; project adaptation and overall project execution. Evidence of adaptive project management should be highlighted.

iii. Stakeholder Participation and Cooperation
Here the term ‘stakeholder’ should be considered in a broad sense, encompassing all project partners, duty bearers with a role in delivering project outputs and target users of project outputs and any other collaborating agents external to UNEP. The assessment will consider the quality and effectiveness of all forms of communication and consultation with stakeholders throughout the project life and the support given to maximise collaboration and coherence between various stakeholders, including sharing plans, pooling resources and exchanging learning and expertise. The inclusion and participation of all differentiated groups, including gender groups, should be considered.

The progress, challenges and outcomes regarding engagement of stakeholders in the project/program should be reviewed. This should be based on the description included in the Stakeholder Engagement Plan or equivalent documentation submitted at CEO Endorsement/Approval.

iv. Responsiveness to Human Rights and Gender Equity
The Review will ascertain to what extent the project has applied the UN Common Understanding on the human rights-based approach (HRBA) and the UN Declaration on the Rights of Indigenous People. Within this human rights context the Review will assess to what extent the intervention adheres to UNEP’s Policy and Strategy for Gender Equality and the Environment²⁰.

The report should present the extent to which the intervention, following an adequate gender analysis at design stage, has implemented the identified actions and/or applied adaptive management to ensure that Gender Equity and Human Rights are adequately taken into account. In particular, the Review will consider to what extent to which project design, the implementation that underpins effectiveness and monitoring have taken into consideration: (i) possible gender inequalities in access to, and the control over, natural resources; (ii) specific vulnerabilities of disadvantaged groups (especially women, youth and children and those living with disabilities) to environmental degradation or disasters; (iii) the role of women in mitigating or adapting to environmental changes and engaging in environmental protection and rehabilitation.

The progress, challenges and outcomes regarding gender-responsive measures and any intermediate gender result areas should be reviewed. This should be based on the documentation at CEO Endorsement/Approval, including gender-sensitive indicators contained in the project results framework or gender action plan or equivalent,

v. Environmental and Social Safeguards
UNEP projects address environmental and social safeguards primarily through the process of environmental and social screening, risk assessment and management (avoidance or mitigation) of potential environmental and social risks and impacts associated with project and programme activities. The evaluation will confirm whether UNEP requirements21 were met to: screen proposed projects for any safeguarding issues; conduct sound environmental and social risk assessments; identify and avoid, or where avoidance is not possible, mitigate, environmental, social and economic risks; apply appropriate environmental and social measures to minimize any potential risks and harm to intended beneficiaries and report on the implementation of safeguard management measures taken.

The evaluation will also consider the extent to which the management of the project is minimising UNEP’s environmental footprint.

The Safeguards Plan submitted at CEO Approval should be reviewed, the risk classifications verified and any measures taken to address identified risks assessed. Any supporting documents gathered by the Consultant should be shared with the Task Manager.

vi. Country Ownership and Driven-ness
The Review will assess the quality and degree of engagement of government / public sector agencies in the project. The Review will consider the involvement not only of those directly involved in project execution and those participating in technical or leadership groups, but also those official representatives whose cooperation is needed for change to be embedded in their respective institutions and offices. This factor is concerned with the level of ownership generated by the project over outputs and outcomes and that is necessary for long term impact to be realised. Ownership should extend to all gender and marginalised groups.

vii. Communication and Public Awareness
The Review will assess the effectiveness of: a) communication of learning and experience sharing between project partners and interested groups arising from the project during its life and b) public awareness activities that were undertaken during the implementation of the project to influence attitudes or shape behaviour among wider communities and civil society at large. The Review should consider whether existing communication channels and networks were used effectively, including meeting the differentiated needs of gender or marginalised groups, and whether any feedback channels were established. Where knowledge sharing platforms have been established under a project the Review will comment on the sustainability of the communication channel under either socio-political, institutional or financial sustainability, as appropriate.

The progress, challenges and outcomes regarding the implementation of the project’s Knowledge Management Approach, including: Knowledge and Learning Deliverables (e.g. website/platform development); Knowledge Products/Events; Communication Strategy; Lessons Learned and Good Practice; Adaptive Management Actions should be reviewed. This should be based on the documentation approved at CEO Endorsement/Approval.

²⁰The Evaluation Office notes that Gender Equality was first introduced in the UNEP Project Review Committee Checklist in 2010 and, therefore, provides a criterion rating on gender for projects approved from 2010 onwards. Equally, it is noted that policy documents, operational guidelines and other capacity building efforts have only been developed since then and have evolved over time. https://wedocs.unep.org/bitstream/handle/20.500.11822/7655/-Gender_equality_and_the_environment_Policy_and_strategy- 2015Gender_equality_and_the_environment_policy_and_strategy.pdf.pdf?sequence=3&isAllowed=y
²¹For the review of project concepts and proposals, the Safeguard Risk Identification Form (SRIF) was introduced in 2019 and replaced the Environmental, Social and Economic Review note (ESERN), which had been in place since 2016. In GEF projects safeguards have been considered in project designs since 2011.

Section 3. REVIEW APPROACH, METHODS AND DELIVERABLES

The Mid-Term Review will use a participatory approach whereby key stakeholders are kept informed and consulted throughout the review process. Both quantitative and qualitative evaluation methods will be used as appropriate to determine project achievements against the expected outputs, outcomes and impacts. It is highly recommended that the Review Consultant maintains close communication with the project team and promotes information exchange throughout the review implementation phase in order to increase their (and other stakeholder) ownership of the review findings.

Where applicable, the Review Consultant should provide a geo-referenced map that demarcates the area covered by the project and, where possible, provide geo-reference photographs of key intervention sites (e.g. sites of habitat rehabilitation and protection, pollution treatment infrastructure, etc.)

The findings of the Review will be based on the following:

(a) A desk review of:
• Relevant background documentation,

• Project Document and Appendices

• Project design documents (including minutes of the project design review meeting at approval); Annual Work Plans and Budgets or equivalent, revisions to the project (Project Document Supplement), the logical framework and its budget;

• Half-Year Progress Reports (HYPR), Project Implementation Review (PIR) reports, and financial reports (in the UNEP Anubis data management system), progress reports from collaborating partners, meeting minutes, relevant correspondence and including the Tracking Tool etc.;

• Communication strategy

• Evaluations/Reviews of similar projects.

(b) Interviews (individual or in group) with:
• UNEP Task Manager (TM) and UNEP Programme Assistant;

• Project Director (PD), National Project Co-ordinator (NPC), Monitoring and Evaluation Officer (M&E) and other team members; Officers from the Department of Forestry (DOF), the Agriculture Extension Services (AES), The Department of Fisheries (DOF), The Department of Renewable Energy, The Water Resource Management Agency (WRMA), The Livestock Department (LD), Department of Physical Planning (DOPP), The Ministry of Tourism (MOT),

• UNEP Fund Management Officer (FMO);

• Representatives from civil society and specialist groups such as The Saint Lucia National Trust, The Saint Lucia National Conservation Fund, The Anse Ger Rural Women’s Group, the Laborie Development Foundation.
(c) Field visits: One
(d) Other data collection tools: If needed, to be decided by the Review Consultant at the inception phase

11. Review Deliverables and Review Procedures

The Review Consultant will prepare:

• Inception Report: (see Annex 3 for guidance on structure and content) containing confirmation of the results framework and Theory of Change of the project, project stakeholder analysis, review framework and a tentative review schedule.

• Preliminary Findings Note: typically, in the form of a PowerPoint presentation, the sharing of preliminary findings is intended to support the participation of the project team, act as a means to ensure all information sources have been accessed and provide an opportunity to verify emerging findings.

• Draft and Final Review Reports: (see Annex 4 for guidance on structure and content) containing an Executive Summary that can act as a stand-alone document; detailed analysis of the review findings organised by evaluation criteria and supported with evidence; lessons learned and recommendations and an annotated ratings table.

Review of the draft review report. The Review Consultant will submit a draft report to the Project Director, The National Project Co-ordinator and Task Manager and revise the draft in response to their comments and suggestions. Once a draft of adequate quality has been peer-reviewed and accepted, the Project Director with concurrence from the Task Manager, will share the cleared draft report with key project stakeholders for their review and comments. Stakeholders may provide feedback on any errors of fact and may highlight the significance of such errors in any conclusions as well as providing feedback on the proposed recommendations and lessons. Any comments or responses to draft reports will be sent to the Project Co-ordinator for consolidation. The Project Co-ordinator will provide all comments to the Review Consultant for consideration in preparing the final report, along with guidance on areas of contradiction or issues requiring an institutional response. The Task Manager will support as appropriate.

At the end of the review process and based on the findings in the Review Report, the Task Manager will prepare a Recommendations Implementation Plan in the format of a table, to be completed and updated at regular intervals, and circulate Lessons Learned.

12. The Review Consultant

The Review Consultant who will work under the overall responsibility of the Project Director Ms. Annette Rattigan-Leo the Chief Sustainable Development and Environment Officer (or her designate) of the Department of Sustainable Development in consultation with the Task Manager Christopher Cox and Programme Assistant Gloritzel Frangakis, the Portfolio Manager (OiC) Ersin Esin and the Fund Management Officer, George Saddimbah. The consultant will liaise with the Task Manager on any procedural and methodological matters related to the Review. It is, however, the consultants’ individual responsibility to arrange for their travel, visa, plan meetings with stakeholders (with assistance from the DSD), organize online surveys, and any other logistical matters related to the assignment. The Project Team, supported by the Task Manager will, where possible, provide logistical support (introductions, meetings etc.) allowing the Review Consultant to conduct the Review as efficiently and independently as possible.

The Review Consultant will be hired over a period of 4 months [ October to January 2024 ] and should have the following: a university degree in environmental sciences, international development or other related fields; a minimum of 10 years of technical evaluation experience is required, preferably to include elaboration and design of projects, evaluating large, regional or global programmes and using a Theory of Change approach; a broad understanding of multi-sectorial projects or initiatives analysis and evaluation, including multilateral funding or support agencies. English and French are the working languages of the United Nations Secretariat. For this consultancy, fluency in oral and written English is a requirement.
Working knowledge of the UN system and specifically the work of UNEP is an added advantage. The work will be home-based and expected to be facilitated by telecommuting (considering any continued COVID -19 protocols) with at least 1 mission to the country.

The Review Consultant will be responsible, in close consultation with the Project Director and supported by the National Project Co-ordinator supported by the Task Manager, for overall management of the review and timely delivery of its outputs, described above in Section 11 Evaluation Deliverables, above. The Review Consultant will ensure that all evaluation criteria and questions are adequately covered.

13. Schedule of the Review

The table below presents the tentative schedule for the Review.

Table 3. Tentative schedule for the Review

Milestone	Indicative Time frame
Kick-off meeting – hybrid (via Skype, Zoom, etc.)	October 2023
Inception Report	October2023
Data collection and analysis, desk-based interviews and surveys	November to December2023
PowerPoint/presentation on preliminary findings and recommendations	December 2023
Draft Report to Project Co-ordinator	Deember 2023
Draft Report shared with the wider group of stakeholders	December 2023
Final Main Review Report	January 2024
Final Main Review Report shared with all respondents	January 2024

14. Contractual Arrangements

The Review Consultant will be selected and recruited by the Department of Sustainable Development (DSD) under a service Contract for approval by the Government of Saint Lucia through the Ministry of Education, Sustainable Development, Innovation, Science Technology and Vocational Training on a “fees only” basis (see below). By signing the service contract with the DSD, the consultant certifies that they have not been associated with the design and implementation of the project in any way which may jeopardize their independence and impartiality towards project achievements and project partner performance. In addition, they will not have any future interests (within six months after completion of the contract) with the project’s executing or implementing units.

Fees will be paid on an instalment basis, paid on acceptance by the Project Director and Task Manager of expected key deliverables. The schedule of payment is as follows:

Schedule of Payment for the Consultant:

Deliverable	Percentage Payment
Approved Inception Report (as per annex document 3)	20%
Approved Draft Main Evaluation Report (as per annex document 4)	40%
Approved Final Main Evaluation Report	40%

Fees only contracts:
The consultant may be provided with access to UNEP’s Anubis information management system and if such access is granted, the consultant agrees not to disclose information from that system to third parties beyond information required for, and included in, the Review report.
In case the consultant is not able to provide the deliverables in accordance with these guidelines, and in line with the expected quality standards by the Project Director in consultation with the Task Manager, payment may be withheld at the discretion of the Permanent Secretary of the DSD until the consultants have improved the deliverables to meet the DOE and UNEP’s quality standards.
If the consultant(s) fail to submit a satisfactory final product to the Project Director in a timely manner, i.e. before the end date of their contract, the DSD reserves the right to employ additional human resources to finalize the report, and to reduce the consultants’ fees by an amount equal to the additional costs borne by the DSD to bring the report up to standard or completion.

Department of Sustainable Development Evaluation Criteria

The evaluation criteria and weightings that will be applied to this TOR are as follows:

Category	Description	Weighting
1	Qualifications of consultant and availability of named individuals including national experts	25
2	Adherence to TOR specifications and related requirements: Clear understanding of required deliverables	35
3	Experience with similar work	25
4	Demonstrated track record of success, supported by references	15
	Total	100

Annex 1: Tools, Templates and Guidance Notes for use in the Review

The tools, templates and guidance notes listed in the table below are available from the UNEP Evaluation Office ([email protected]) and are intended to help Task Managers and Review Consultants to produce review products that are consistent with each other and which contribute to UNEP results reporting. (Three key templates are also attached below). This suite of documents is also intended to make the review process as transparent as possible so that all those involved in the process can participate on an informed basis. It is recognised that the review needs of projects and portfolio vary and adjustments may be necessary so that the purpose of the review process (broadly, accountability and lesson learning), can be met. Such adjustments should be decided between the Task Manager and the Review Consultant in order to produce review reports that are both useful to project implementers and that produce credible findings.

Document Name

Evaluation Ratings Table (see below)

Weighting of Ratings (excel sheet)

Matrix Describing Ratings by Criteria

Structure and Contents of the Inception Report (see below)

Guidance on Stakeholder Analysis

Use of Theory of Change in Project Evaluations

Possible Evaluation Questions

Structure and Contents of the Main Review Report (see below)

Cover Page, Prelims and Style Sheet for Main Review Report

Financial Tables

Template for the Assessment of the Quality of the Review Report (this will be completed by the UNEP Evaluation Office and annexed to the Review Report)

The review will provide individual ratings for the evaluation criteria described in the table below. The Evaluation Office website (https://www.unenvironment.org/about-un-environment/evaluation/our- evaluation-approach) holds all support tools, templates and guidance notes mentioned below.

Most criteria will be rated on a six-point scale as follows: Highly Satisfactory (HS); Satisfactory (S); Moderately Satisfactory (MS); Moderately Unsatisfactory (MU); Unsatisfactory (U); Highly Unsatisfactory (HU). Sustainability and Likelihood of Impact are rated from Highly Likely (HL) down to Highly Unlikely (HU) and Nature of External Context is rated from Highly Favourable (HF) to Highly Unfavourable (HU). A Ratings Matrix is available to support a common interpretation of points on the scale for each evaluation criterion. These ratings are ‘weighted’ to derive the Overall Project Rating (see ‘Weighting of Ratings’ on the Evaluation Office website).

In the conclusions section of the Main Mid Term Review Report, ratings will be presented together in a table, with a brief justification for each rating, cross-referenced to findings in the main body of the report.

Criterion (Enter each rating into the Weighting of Ratings table to arrive at the rating for each criterion and the overall project rating)	Summary Assessment	Rating
A. Strategic Relevance		HS → HU
1. Alignment to MTS and POW		HS → HU
2. Alignment to UNEP / Donor/GEF strategic priorities		HS → HU
3. Relevance to regional, sub-regional and national environmental priorities		HS → HU
4. Complementarity with existing interventions		HS → HU
B. Effectiveness22		HS → HU
1. Availability of outputs		HS → HU
2. Achievement of project outcomes		HS → HU
3. Likelihood of impact		HL → HU
C. Financial Management		HS → HU
1. Adherence to UNEP’s policies and procedures		HS → HU
2. Completeness of project financial information		HS → HU
3. Communication between finance and project management staff		HS → HU
D. Efficiency		HS → HU
E. Monitoring and Reporting		HS → HU
2. Monitoring of project implementation		HS → HU
3. Project reporting
F. Sustainability (the overall rating for Sustainability will be the lowest rating among the three sub-categories)		HL → HU
1. Socio-political sustainability		HL → HU
2. Financial sustainability		HL → HU
3. Institutional sustainability		HL → HU
G. Factors Affecting Performance and Cross-Cutting Issues23		HS → HU
1. Preparation and readiness		HS → HU
2. Quality of project management and supervision24		HS → HU
3. Stakeholders participation and cooperation		HS → HU
4. Responsiveness to human rights and gender equity		HS → HU
5. Environmental, social and economic safeguards		HS → HU
6. Country ownership and driven-ness		HS → HU
7. Communication and public awareness		HS → HU
Overall Project Rating		HS → HU

^{____________________
22} Where a project is rated, through the assessment of Project Design Quality template during the review inception stage as facing either an Unfavourable or Highly Unfavourable external operating context, ratings for Effectiveness, Efficiency and/or Sustainability may be increased at the discretion of the Review Consultant and Project Manager together. Any adjustments must be fully justified.

Annex 3: Guidance on the Structure and Contents of the Mid Term Review Inception Report

Guidance on the Structure and Contents of the Review Inception Report
(NOTE: This report should be written as original text and not copied from the TOR)

Section	Notes	Data Sources	Recommended no. pages
Preliminary pages	Review and complete (where necessary) the Project Identification Table that was in the Terms of Reference.	TOR, ProDoc, PM	1
1. Introduction	Summarise: Purpose and scope of the review (eg learning/accountability and the project boundaries the review covers) Project problem statement and justification for the intervention. Institutional context of the project (MTS, POW, Division, umbrella etc) Target audience for the review findings.	TOR and ProDoc	1
2. Project outputs and outcomes	Confirm the formulation of planned project outputs and expected outcomes. The project should be assessed against its intended results, but these may need to be rephrased, re-aligned etc. Where the articulation of the project’s results framework, including outputs, outcomes, long term impacts and objectives/goals, needs to be revised, a table should be provided showing the original version and the revisions proposed for use in the review. SPECIFY WHICH GEF CORE INDICATOR TARGETS WERE IDENTIFIED AT CEO ENDORSEMENT/APPROVAL (For projects approved prior to GEF-7 these will be identified retrospectively and progress against them assessed).	ProDoc, Revision documents, consultation with Project Manager (PM)	1 /2
3. Review of project design	Complete the template for assessment of Project Design Quality, including ratings, and present as an annex (template available) Summarise the project design strengths and weaknesses within the body of the inception report.	Project document and revisions, MTE/MTR if any	1 page narrative and completed assessment of PDQ template
4. Stakeholder analysis25	Identify key stakeholder groups and provide an analysis of the levels of influence and interest each stakeholder group has over the project outcomes. Give due attention to gender and under-represented/marginalised groups. (guidance note available)	Project document Project preparation phase. PM	1
5. Theory of Change	The Project Design document should have a Theory of Change. Review and revise (or reconstruct) the Theory of Change at Review Inception26 (TOC at Review Inception) based on project documentation, formal revisions, annual reports etc. Present this TOC as a one- page diagram, where possible, and explain it with a narrative, including a discussion of the assumptions and drivers (guidance note and samples available). Identify aspects of the TOC at Review Inception that need to be explored further during the review process with the project team and stakeholders. Note if the needs of different groups (vulnerable, gender groups etc) need to be reflected in the TOC Identify any key literature/seminal texts that establish cause and effect relationships for this kind of intervention at higher results levels (eg benefits of introducing unleaded fuel)	Project document narrative, logical framework and budget tables. Other project related documents.	Diagram and up to 2 pages of narrative
6. Review methods	Describe all review methods (especially how sites/countries will be selected for field visits or case studies; how any surveys will be administered; how findings will be analysed etc) Methods to ensure that potentially excluded groups (excluded by gender, vulnerability or marginalisation) are reached and their experiences captured effectively, should be made explicit in this section. Summarise date sources/groups of respondents and method of data collection to be used with each (e.g skype, survey, site visit etc) Create a review framework that includes detailed review questions linked to data sources. Note that the Evaluation Office provides a matrix for rating each of its evaluation criteria. Include any new questions raised by review of Project Design Quality and TOC analysis. Present this as a table/matrix in the annex (samples available) Design draft data collection tools and present in the annex (eg interview schedules, questionnaires etc)	Review of all project documents.	1 page narrative. The review framework as a matrix and draft data collection tools as annexes.
7. Team roles and responsibilities	Describe the roles and responsibilities among the review team, where appropriate		½
8. Review schedule	Provide a revised timeline for the overall review (dates of travel and key review milestones) Tentative programme for site/country visits	Discussion with PM on logistics	½ (table)
9. Learning, communication and outreach	Describe the approach and methods that will be used to promote reflection and learning through the review process (eg opportunities for feedback to stakeholders; translation needs etc) See EOU website (https://www.unenvironment.org/about-un- environment/evaluation) for examples of Evaluation Briefs - 2-page summaries of main findings.	Discussions with the PM	½
TOTAL NARRATIVE PAGES			8-12 pages, plus annexes
Annexes	A - Review Framework B - Draft data collection tools C - Completed assessment of the Project Design Quality D - List of documents and individuals to be consulted during the main review phase E - List of individuals and documents consulted for the inception report

²³ While ratings are required for each of these factors individually, they should be discussed within the Main Reivew Report as cross- cutting issues as they relate to other criteria. Note that catalytic role, replication and scaling up are expected to be discussed under effectiveness if they are a relevant part of the TOC.
²⁴ In some cases ‘project management and supervision’ will refer to the supervision and guidance provided by UNEP to implementing partners and national governments while in others, specifically for GEF funded projects, it will refer to the project management performance of the Executing Agency and the technical backstopping provided by UNEP, as the Implementing Agency.
²⁵ Evaluation Office of UNEP identifies stakeholders broadly as all those who are affected by, or who could affect (positively or negatively) the project’s results. At a disaggregated level key groups should be identified, such as: implementing partners; government officials and duty bearers (e.g. national focal points, coordinators); civil society leaders (e.g. associations and networks) and beneficiaries (e.g. households, trade speople, disadvantaged groups, members of civil society etc).
²⁶The project’s TOC at Evaluation Inception is prepared during the inception phase of the evaluation and refined during the evaluation process to become the TOC at Evaluation. For the TOC at Evaluation Inception the evaluation team will need to examine the result statements and their causal logic from the project logframe and the drivers and assumptions from the narrative sections from the ProDoc (in particular from the critical success factors and risks sections). Stakeholder roles may be available from the description of the project intervention and the stakeholder and partner analysis sections.

Annex 4: Guidance on the Structure and Contents of the Main Mid Term Review Report

NOTE: Review Consultants are kindly advised to refer the reader to paragraphs in different parts of the report instead of repeating material.

Preliminaries	Title page – Name and ID number of the evaluated project, type of evaluation (Mid- Term Review), month/year evaluation report completed, UNEP logo. Include an appropriate cover page image. Disclaimer text- ‘This report has been prepared by an independent Review Consultant and the findings and conclusions expressed herein do not necessarily reflect the views of UNEP or its staff. Acknowledgements – This is a maximum of two paragraphs. Short biography of the consultant(s) – giving relevant detail of experience and qualifications that make the consultant a suitable candidate for having undertaken the work. (Max 1 paragraph) Contents page – including chapters, tables and annexes Abbreviations table – only use abbreviations for an item that occurs more than 3 times within the report. Introduce each abbreviation where it appears first in the report and ensure it is in the abbreviations table. Where an abbreviation has not been used recently in the text, provide its full version again. The Executive Summary should be written with no abbreviations. Paragraph numbering – All paragraphs should be numbered, starting from the Executive Summary Header/footer – Name of evaluated project, type of evaluation and month/year evaluation report completed. Page numbers, header and footer do not appear on the title page
Project Identification Table	An updated version of the Project Identification Table (i.e., the table at the beginning of each Project Implementation Review Report).
Executive Summary (Kindly avoid all abbreviations in the Executive Summary) Start numbering paragraphs from the Executive Summary.	The Executive Summary should be able to stand alone as an accurate summary of the main review product. It should include a concise overview of the review object; clear summary of the review objectives and scope; overall evaluation rating of the project and key features of performance (strengths and weaknesses) against exceptional criteria (plus reference to where the evaluation ratings table can be found within the report); summary of the main findings of the exercise, including a synthesis of main conclusions (which include a summary response to key strategic evaluation questions) and selected lessons learned and recommendations. (Max 4 pages)
I. Project Overview (describe the Evaluand)
	Provide an overview of the project, covering, inter alia: • its institutional context within UNEP (where managed from etc) • implementation structure (with diagram) • the problem/issue the project aims to address • project parameters for the review (start and end date; geographic reach; total budget etc) • project results framework - Theory of Change diagram to be included under Review findings below (justify any revisions to the formulation of results statements to conform to UNEP definitions and/or international standards) • description of targeted groups/stakeholders and their relationship with the project (including, stakeholder analysis diagram) • any major and agreed changes to the project (e.g. formal revisions, additional funding etc) • any external challenges faced by the project (eg conflict, natural disaster, political upheaval etc • financial tables ((a) budget at design and expenditure by components (b) planned and actual sources of funding/co-financing (Max 3 pages)
II. Review Methods	This section is the foundation for the Review’s credibility, which underpins the validity of all its findings. The section should include: a description of review methods and information sources used, including the number and type of respondents; justification for methods used (e.g. qualitative/quantitative; electronic/face-to-face); any selection criteria used to identify respondents, case studies or sites/countries visited; strategies used to increase stakeholder engagement and consultation, including different gender groups; details of how data were verified (e.g. triangulation, review by stakeholders etc). The methods used to analyse data (e.g. scoring; coding; thematic analysis etc) should be described. It should also address limitations to the Review such as: low or imbalanced response rates across different groups; extent to which findings can be either generalised to wider review questions or constraints on aggregation/disaggregation; any potential or apparent biases; language barriers and ways they were overcome. Ethics and human rights issues should be highlighted including: how anonymity and confidentiality were protected and strategies used to include the views of marginalised or potentially disadvantaged groups and/or divergent views. (Max 2 pages)
III. Review Findings
**Refer to the TOR for descriptions of the nature and scope of each criterion**	This chapter is organized according to the evaluation criteria presented in the TORs and reflected in the evaluation ratings table. The Review Findings section provides a summative analysis of all triangulated data relevant to the parameters of the criteria. Review findings should be objective, relate to the review objectives/questions, be easily identifiable and clearly stated and supported by sufficient evidence. This is the main substantive section of the report and incorporates indicative evidence27 as appropriate. “Factors Affecting Performance” should be discussed as appropriate in each of the evaluation criteria as cross-cutting issues. Ratings are provided at the end of the assessment of each evaluation criterion and the complete ratings table is included under the conclusions section, below. Evaluation Criteria: A. Strategic Relevance B. Effectiveness (includes delivery of outputs and achievement of outcomes within the context of the Theory of Change - include TOC diagram) C. Financial Management D. Efficiency E. Monitoring and Reporting F. Sustainability (Max 15 pages)
I. Factors Affecting Performance	These factors are not discussed in stand-alone sections but are integrated in criteria A- G as appropriate. A rating is given for each of these factors in the Evaluation Ratings Table.
IV. Conclusions and Recommendations
A. Conclusions	This section should summarize the main conclusions of the review following a logical sequence from cause to effect. The conclusions should highlight the main strengths and weaknesses of the project, preferably starting with the positive achievements and a short explanation of how these were achieved, and then moving to the less successful aspects of the project and explanations as to why they occurred. Answers to the key strategic evaluation questions, including an answer to the questions on Core Indicator Targets, stakeholder engagement, gender responsiveness, safeguards and knowledge management, required for the GEF portal, should be provided. All conclusions should be supported with evidence that has been presented in the evaluation report and can be cross-referenced to the main text using paragraph numbering. The conclusions section should end with the overall assessment of the project, followed by the ratings table. Conclusion section should have a table summarizing the findings of the following questions: a) What is the performance at the project’s mid-point against Core Indicator Targets? b) What has been the progress, challenges and outcomes regarding engagement of stakeholders in the project/program? c) What has been the progress, challenges and outcomes regarding gender-responsive measures and any intermediate gender result areas? d)What has been the experience at the project’s mid-point against the Safeguards Plan submitted at CEO Approval? e)What has been the progress, challenges and outcomes regarding the implementation of the project's Knowledge Management Approach, including: Knowledge and Learning Deliverables The conclusions section should not be a repeat of the Executive Summary but focuses on the main findings in a compelling story line that provides both evidence and explanations of the project’s results and impact. (Max 2 pages)
B. Lessons Learned	Lessons learned should be anchored in the conclusions of the Review, with cross- referencing to appropriate paragraphs in the review report where possible. Lessons learned are rooted in real project experiences, i.e. based on good practices and successes which could be replicated in similar contexts. Alternatively, they can be derived from problems encountered and mistakes made which should be avoided in the future. Lessons learned must have the potential for wider application and use and should briefly describe the context from which they are derived and those contexts in which they may be useful. Specific lessons on how human rights and gender equity issues have been successfully integrated into project delivery and/or how they could have could have been taken into consideration, should be highlighted.
C. Recommendations	All recommendations should be anchored in the conclusions of the report, with paragraph cross-referencing where possible. Recommendations are proposals for specific actions to be taken by identified people/position-holders to resolve concrete problems affecting the project or the sustainability of its results. They should be feasible to implement within the timeframe and resources available (including local capacities), specific in terms of who would do what and when, and set a measurable performance target in order that the project team/Head of Branch/Unit can monitor and assess compliance with the recommendations. It is suggested that a succinct and actionable recommendation is stated first and is followed by a summary of the finding which supports it. In some cases, it might be useful to propose options, and briefly analyse the pros and cons of each option. Specific recommendations on actions that could be taken within the available time and resources to ensure the delivery of results relevant to human rights and gender equity should be highlighted. In cases where the recommendation is addressed to a third party, compliance can only be monitored and assessed where a contractual/legal agreement remains in place. Without such an agreement, the recommendation should be formulated to say that UNEP project staff should pass on the recommendation to the relevant third party in an effective or substantive manner. The effective transmission by UNEP of the recommendation will then be monitored for compliance. Where a new project phase is already under discussion or in preparation with the same third party, a recommendation can be made to address the issue in the next phase.
Annexes	These may include additional material deemed relevant by the Review Consultant but must include: 2. List of respondents 3. List of documents consulted 4. Review itinerary, containing the names of locations visited and the names (or functions) and of people met/interviewed. (A list of names and contact details of all respondents should be given to the Project Manager for dissemination of the report to stakeholders but contact details should not appear in the report). 4. Summary of co-finance information and a statement of project expenditure by activity or component 5. Any communication and outreach tools used to disseminate results (e.g. power point presentations, charts, graphs, videos, case studies, etc.) 6. Any documents provided to implement the Safeguards Plan 7. Brief CVs of the consultants 8. Mid Term Review TORs (without annexes).

_______________
²⁷ This may include brief quotations, anecdotal experiences, project events or descriptive statistics from surveys etc. The anonymity of all respondents should be protected.