30-July-2020
- Tell me about one of your recent memorable evaluation experiences:
My recent memorable evaluation experience relates to a situation where I was able to turn around a conflict situation into a real engagement success story. In an evaluation I led, even though the evaluation design stage clearly developed the consultation process with stakeholders in its approach paper, in practice, the implementation of the engagement process was less effective and the process resulted in a conflict over the data used and the resulting evaluation ratings, for different reasons. This situation required us to discuss carefully all the points of difference with the relevant Operations departments, which cost additional financial resources and was highly time-consuming. But more importantly, it allowed both IDEV and Operations to engage and learn on a deeper level, and to better understand each other’s viewpoint. Lessons learned from this experience were successfully applied in subsequent evaluations, making now the conflict situation as water under the bridge.
- What do you do before any evaluation?
For all evaluations, my first priority is to understand the logic behind the interventions in the domain covered by the evaluation. Thanks to google and other platforms, I search, download, read and summarize accordingly. I use this approach since each new evaluation comes with different challenges and contexts.
- What do you bring with you on your field missions in conducting an evaluation?
During field missions, I normally have a couple of things : (i) a camera, as there are a lot of things on the ground that are not written on paper and to take photos for our evaluation publications and website; (ii) special shoes adaptable to any kind of terrain; (iii) a notepad used only for personal observations and insightful quotations; (iv) an audio recorder to use if necessary and allowed; and (v) an interview guide.
- What is the most important lesson you learned from the recent evaluations you have carried out?
Indeed, as much as we advocate for accountability and learning, we as evaluators learn a lot from our experiences. At IDEV, we work to support the work of the Bank’s operations team and so engaging them throughout the evaluation cycle pays off. This consistent engagement process ensures that each stakeholder stays on the same page and even where disagreements arise regarding an evaluation, they should be treated fairly and firmly. Secondly, I have learned that an evaluator should always be ready to be confronted with a conflict situation because of his/her independence and his/her focus on evidence, mainly in the context where M&E is not as strong as expected. Also, I have learned that an evaluator should be open to changing his/her point of view when substantial additional evidence is presented to him/her. Finally, the evaluator should use any potential conflict situation as an opportunity to build greater trust with stakeholders as it is not about personal issues but the end beneficiaries of the Bank’s interventions.
- What should an evaluator avoid doing during an evaluation?
Like any other profession, there are key values that underpin the evaluation profession and for that matter evaluators. And for me as an independent evaluator, integrity is so critical. Thus, an evaluator should avoid manipulating or skewing evaluation results. Going this path defeats the very purpose of evaluation.
Also, when conducting an evaluation, the role of the evaluator is to gather the maximum of evidence from different sources to triangulate. This should be done by applying robust methods with a high sense of ethics and by treating all stakeholders with respect. I think that the evaluator should have more skills than only technical, including being a communicator, a diplomat, a philosopher, a psychologist, a teacher or a coach, …
Most critical for an evaluation is to better prepare his/her work and delineate the scope of the evaluation. The focus should be on the main evaluation questions, as evaluation cannot respond to all questions. Like for development projects, poor preparation of an evaluation could lead to time and cost over-runs and eventually result in an evaluation with limited utility.
- How can we strengthen the evaluation practice?
While accountability is critical, it is now time to foster the learning aspect of the evaluation through better engagement, as it seems that we are not learning as much as we should from experiences. Providing concrete guidance to operations on M&E tools such as the logic model, theory of change, evaluability assessment, and results framework, will not threaten the independence of evaluators. Besides, in the majority of African countries where the Bank works, we will need to do more to build a strong culture of evaluation in the public sector. This is crucial to the collective success of the Bank’s work in Regional Member Countries.
- How do you think evaluation will evolve?
The evaluator will play a more advisory role in addition to that of analyst due to the development of new technologies such as Artificial Intelligence (AI) and Big data which can facilitate evaluations. The more the evaluator is helping development actors while keeping his/her independence, to better design and implement development interventions, the more we will all impact the lives of people. My experience in helping operational colleagues in areas such as clarifying the theories of change of their interventions shows a dramatic change in the way they are thinking about development.
- What changes will we see?
Well, I’m currently reading the book “Homo Deus: The brief history of tomorrow” by Israeli author Yuval Noah Harari which analyses the future ahead and underscores how algorithms (e.g., artificial intelligence) will revolutionize the world of work. The evaluation field will be no different especially with the advent of Big Data technology for instance. These technologies will greatly influence how we evaluate interventions in the not-too-distant future. Also, as the situation of the COVID-19 outbreak has shown, real-time evaluation will gain momentum.
- What do you like the most in your job?
I like engaging with people on the ground as it is the only way you can better appreciate the impact of our interventions and discover the untold stories and peoples’ real feelings. Also, the fact that my role as an independent evaluator helps the Bank to learn and improve its operations gives me great joy, as it impacts the lives of people.