Terms of Reference: lessons on their structure and interpretation of results

Lire en francais | Leer en español

Autor: Dr. Issa Sombié/From Burkina Faso. He is a researcher at the National Center for Scientific and Technological Research and a university professor. Doctor in Sociology and holder of a master’s degree in Evaluation

The Terms of Reference (TdR) are a framework that allows to describe the purpose, scope, and process–which includes the management and technical aspects–of an evaluation according to the needs of those who request an evaluation of a program or project.

This guide is of utmost importance in the evaluation processes, since it is where the expectations of those who request an evaluation are placed.  In addition, they serve as a frame of reference for the evaluator so their clarity and precision is essential. 

Often, the relationships between evaluators and the person requesting the evaluation are affected by missing, misleading, or ambiguous elements in the Terms of Reference.  It is common that, during this development, the real needs of the program are not adequately reflected.

Next, we will look at the most accepted basic elements as a structure to use the Terms of Reference(ToR) and applicable to the evaluation process, according to the United Nations Children’s Fund (UNICEF). 

Table 1: Basic elements of the structure of the UNICEF Terms of Reference (ToR) 

In one exercise, we will evaluate three evaluation processes where the use of the Terms of Reference (ToR), in various programs carried out in Burkina Faso, in Africa, led to mixed results.

The first concerns an evaluation of a control program for the elimination of lymphatic filariasis.  

The second is the evaluation of an HIV/AIDS program implemented by a  local organization.

Finally, the evaluation process of a malaria control program of an international NGO is shown.

Reading the tables shows us that the different elements proposed by the analytical framework  are not always considered for the drafting of the Terms of Reference.  We find that the  “Background” was not presented in the three examples of ToR as suggested by the UNICEF analysis framework.  

The “objectives of the evaluation”, as well as the “composition of the team”, were also considered in the three examples, although not exactly as the analytical framework suggests.  As for the elements:   “existing data sources”, “involvement of the players involved”, “procedures and logistics”, and “budget”, they were not developed in any of the three examples. 

Finally, the “methodology”, “evaluation questions”, “responsibilities and commitments of the players involved” were considered only in one of the three examples.  We note that certain elements that are essential have not always been addressed. 

With this brief analysis of the Terms of Reference of an evaluation process, we learned a few  things:

First lesson: 

It is necessary to clarify the purpose and objectives of the evaluation.  A precise and detailed description of the objectives of the evaluation will allow the evaluation team to propose appropriate methods to meet the stated needs.  To do this, whoever requests an evaluation of a program or project must strive to establish their priorities in terms of expected information and not according to the evaluation criteria.  Above all, program managers should avoid standard Terms of Reference formulas in which the project name is simply changed.

Second lesson:

It is essential to create evaluation questions which, although not exhaustive or correctly drafted, always allow the evaluation team to better understand the expectations and information needs of those who request an evaluation.  From  the evaluator’s point of view, we tend to say that there is no wrong question and that what matters most is knowing what you want to find.  Many times, the lack of evaluation skills hides a real problem in establishing needs, even the choice of evaluation questions is left in the hands of the evaluator.

Third lesson:

Although it is difficult to demand a standard structure of the Terms of Reference, their wording must include a number of essential elements.  We believe that the elements relating to the context, purpose, and objectives of the evaluation, questions, criteria, and methodology are fundamental  for the evaluators to prepare their technical proposal.  It is recommended that program managers study the various guidelines in this area.  Before establishing the purpose and objectives of the evaluation, it is necessary to anticipate the possible use of the results of the evaluation.

Conclusion: 

To a large extent, the quality of assessments depends on the well-developed structure of the Terms of Reference.  In fact, it is difficult to provide an adequate answer to a poorly formulated question or poorly proposed problem.

This analysis exercise allowed us to identify certain deficiencies that may affect the Terms of Reference.     Therefore, we have found that the objectives and evaluation questions are often little or poorly defined, which results in a lack of skills when preparing the ToR.  The most serious is that, based on the results, an inadequate focus is placed on the importance given to an evaluation process. 

Why do we carry out an evaluation?  It should be recalled that the first purpose of an evaluation is to support decision-making.  This means that, before developing it, the players have already taken inventory of the expected information and know what it will be used for. 

However, in the context of program and project management in Africa, we often observe that it merely seeks to satisfy the provisions set out in the program or project formulation document.  An evaluation is commissioned because it was already planned at the time of developing the project, so it must be done.  This would also explain the scarce use of evaluation results.

In these circumstances, the problem of formulating needs for evaluation, as well as of the quality of its results, is likely to persist.