Contamination in trials of educational interventions

Abstract

Objectives: To consider the effects of contamination on the magnitude and statistical significance (or precision) of the estimated effect of an educational intervention, to investigate the mechanisms of contamination, and to consider how contamination can be avoided. Data sources: Major electronic databases were searched up to May 2005. Methods: An exploratory literature search was conducted. The results of trials included in previous relevant systematic reviews were then analysed to see whether studies that avoided contamination resulted in larger effect estimates than those that did not. Experts’ opinions were elicited about factors more or less likely to lead to contamination. We simulated contamination processes to compare contamination biases between cluster and individually randomised trials. Statistical adjustment was made for contamination using Complier Average Causal Effect analytic methods, using published and simulated data. The bias and power of cluster and individually randomised trials were compared, as were Complier Average Causal Effect, intention-to-treat and per protocol methods of analysis. Results: Few relevant studies quantified contamination. Experts largely agreed on where contamination was more or less likely. Simulation of contamination processes showed that, with various combinations of timing, intensity and baseline dependence of contamination, cluster randomised trials might produce biases greater than or similar to those of individually randomised trials. Complier Average Causal Effect analyses produced results that were less biased than intention-to-treat or per protocol analyses. They also showed that individually randomised trials would in most situations be more powerful than cluster randomised trials despite contamination. Conclusions: The probability, nature and process of contamination should be considered when designing and analysing controlled trials of educational interventions in health. Cluster randomisation may or may not be appropriate and should not be uncritically assumed always to be a solution. Complier Average Causal Effect models are an appropriate way to adjust for contamination if it can be measured. When conducting such trials in future, it is a priority to report the extent, nature and effects of contamination.