17-January-2023
What do you like most about field missions?
I like the opportunity to directly contribute and add value to the work and livelihoods of the Bank’s partners and program beneficiaries. I believe evaluation should be valuable to all respondents even before the evaluation report is published and shared with respondents. Before an interview, I often ask myself: What additional information, a linkage/connection with stakeholders, awareness of opportunity, etc., give the respondent to improve their work/life? I find that this increases the usefulness of the discussion and generates unexpected insights. This approach also allows the respondent to share their valuable time with the next evaluator, in the future.
What are some of the most important challenges faced during a field mission in collecting data for evaluation? And how do you manage them?
The first challenge I face when preparing for a field mission is methodological, that is, preparing the questions and aligning them with the evaluation questions and the broad objectives of the evaluation. To manage this challenge, I usually develop the evaluation matrix . In some cases, a case study template that further develops the questions in the evaluation matrix into sub-questions, granular enough to capture the overarching question in each OECD/DAC* criteria (relevance, coherence, effectiveness, efficiency, impact, and sustainability).
Once in the field, I see five main challenges:
o First, with the limited time for interviews, it is difficult for respondents to provide detailed answers that align with each evaluation question. The chances of missing out on relevant questions are high. The solution I recommend is to prioritize questions and always request written reports that can shed light on the responses provided by the respondent.
o Second, ensure the survey respondent/key informant remains focused (and does not deviate) from the evaluation questions. In managing this challenge, do not hesitate to ask for clarifications where necessary, including in instances where the evaluator does not understand the technical details of the information.
o Third, logistical challenges can derail a field data collection mission, especially when the evaluator is making field trips away from the capital. For instance, during a mission to Tanzania, the team’s evening return flight to Dar es Salaam from Dodoma was canceled. This implied that we could miss our flight to other regions in the Mainland for data collection. To avoid that, arrange transport beforehand, get the required approvals before arrival, and provide some vacant space between trips to manage unavoidable delays. Be ready to adapt to real-time changes in the mission plan.
o Fourth, language can become a barrier when engaging with final beneficiaries who are often unable to communicate in English or French. Many interesting and important conversations can be lost in translation, and it can become difficult to ask follow-questions. Hiring a local translator and properly integrating them into the evaluation team can address that challenge.
o Fifth, complexities in engaging with high-level government officials and program/project beneficiaries. It is important to have situational awareness to reframe questions appropriately, depending on whether a respondent is a cabinet minister or a farmer. Be open and compassionate with beneficiaries to get the highest quality and truthful information from them.
What will be your key advice to a young evaluator going on his/her first field mission?
I have four tips for young evaluators going on their first field mission:
o It is important to have an adequate understanding of the program/policy/strategy being evaluated.
o Prioritize the evaluation questions and ask the most critical/important ones at the earliest stages of an interview.
o Flexibility to quickly adapt to unexpected changes in the mission plan.
o The importance of having a positive disposition and mindset cannot be overemphasized. This is important because respondents and partners tend to share their valuable time responding to the request of the evaluator.
As a young evaluator, where do you see yourself in 2 or 5 years?
My only hope is to continue doing fascinating evaluations that will be useful for stakeholders and also allow me to learn and grow.
*OECD/DAC : Organisation for Economic Co-operation and Development/ Development Assistance Committee