Evaluating the Effectiveness of Interventions: Practices and Outcomes from Rural Development to Educational Support

[2J201] Developing Analytical Framework of the External Validity of Evidence: Case Studies of Interventions for Educational Development

*Takao MARUYAMA¹ (1. Japan International Cooperation Agency (JICA))

Keywords:Educational Development, Impact Evaluation, Evidence

The number of impact evaluations on basic education in low- and lower-middle-income countries has dramatically increased since the 2000s in response to the global awareness of the importance of evidence. While the impact evaluation reveals whether interventions work, it does not tell the effectiveness beyond the sample or the sampling frame of the evaluation, as the context differs. This study develops the analytical framework of the external validity of evidence for policymakers and practitioners in educational development. The framework consists of several simple and structured steps to analyze the applicability of interventions for educational development in low- and lower-middle-income countries: (1) laying out a causal chain of the interventions; (2) identifying the underlying theories or concepts of the causal chain; (3) checking whether there is a gap between the description in the causal chain (i.e., intended outputs/outcomes) and the local situation; and (4) listing the contextual factors that could hamper the causal chain and examining whether the interventions are designed to address them. This study develops the framework through the two case studies of interventions in school management.

This study is a joint work with Akiko Kageyama, Kosho Yamagata, and Nobuhiro Kunieda.