What's in the box? Issues in evaluating interventions to develop strong and open communities

The complex range and scope of interventions put in place to address the needs of socially disadvantaged communities present a number of challenges to service providers and funders who need to be convinced that their investment is making a difference. Our experience as evaluators of complex sets of interventions in two socially disadvantaged communities has raised four questions that we feel require thoughtful reflection by those intervening and those evaluating the impact of the interventions. Are we doing what we say we are doing? Are we measuring what we say we are doing? Are we measuring what we say we are measuring? Are we introducing evidence-based practice for which there is no evidence? Using examples from our work we explore a number of theoretical, practical and logistical issues in evaluating interventions to build strong and open communities. This paper is intended to open debate rather than provide solutions to these questions.