Criminology and Criminal Justice Research: Methods - Threats To Validity

Validity refers to the accuracy of measurement or whether the instrument is in fact measuring what it is suppose to measure (Hagan). While quantitative research methods have permeated criminological and criminal justice research, they are not without problems. Threats to validity are perhaps the most profound and should be acknowledged. Some of these threats are internal and are concerned with whether the observational process itself produced the findings, while external threats are concerned with whether the results were unique and applicable only to the group or target studied (Hagan).

Internal threats. According to Campbell and Stanley, a number of internal threats need to be considered, including: (1) history, (2) maturation, (3) testing, (4) instrumentation, (5) statistical regression, (6) selection bias, (7) experimental mortality, and (8) selectionmaturation interaction. In determining whether a particular design rules out threats to internal validity, Cook and Campbell suggest that "estimating the internal validity of a relationship is a deductive process in which the investigator has to systematically think through how each of the internal validity threats can be ruled out" (p. 55).

External threats. Campbell and Stanley also identify several threats to external validity, including: (1) testing effects, (2) selection bias, (3) reactivity or awareness of being studied, and (4) multiple-treatment interference. These threats are greater for experiments conducted under more carefully controlled conditions (Maxfield and Babbie). Perhaps one of the best methods for assessing threats to external validity is replication, or the repetition of experiments or studies utilizing the same methodology. By replication of key findings, researchers can gain confidence that the results observed in one study may not be due to external validity threats. One of the key examples of replication occurred in the late 1980s when the Minneapolis Domestic Violence Experiment was replicated in six cities throughout the United States (Sherman). Importantly, these replications yielded both similar and contradictory conclusions to those observed in the initial experiment.