Criminology and Criminal Justice Research: Methods

Quantitative research methods are typically concerned with measuring criminological or criminal justice reality. To understand this process several terms must first be identified. Concepts are abstract tags placed on reality that are assigned numerical values, thus making them variables. Variables are then studied to examine patterns of relation, covariation, and cause and effect. At the most basic level, there exists at least one dependent variable and one independent variable. The dependent variable is commonly referred to as the outcome variable. This is what the researcher is attempting to predict. The independent variable is commonly referred to as the predictor variable, and it is the variable that causes, determines, or precedes in time the dependent variable (Hagan). Consider the following examples.

Criminological theorists may be interested in studying the relationship between impulsivity (independent variable) and criminal behavior (dependent variable). In studying such a relationship, scholars create a summated scale of items that is designed to indirectly measure the concept of impulsivity. Then, this impulsivity scale is used to predict involvement in criminal behavior. Criminal justice scholars may be interested in studying the effects of a mandatory arrest policy (independent variable) on future patterns of domestic violence (dependent variable). In studying such a question, scholars typically evaluate the effect of an arrest, compared to some other sanction, on the future criminal behavior of the arrestee. Thus, quantitative research methods involve a pattern of studying the relationship(s) between sets of variables to determine cause and effect.

Three criteria are needed to establish causality. The first is association. That is, the independent and dependent variables must be related to one another. The second is time order; the independent variable must precede the dependent variable in time. Finally, there is the issue of nonspuriousness. This occurs if the relationship between the independent and dependent variables is not due to variation in some unobserved third variable.

There are a number of different quantitative research methods available to researchers, most of which fall under the rubric of a research design, which loosely can be defined as the plan or blueprint for a study that includes the who, what, where, when, why and how of an investigation (Hagan). These research methods include: survey research, experimental and quasi-experimental research, cross-sectional research, longitudinal research, time series research, and meta-analysis.

Survey research. Serving as the most frequently used mode of observation within the social sciences, including criminology (Maxfield and Babbie), survey research involves the collection of information from a sample of individuals through their responses to questions (Schutt). Survey research is generally carried out via mail, telephone, computer, or in person.

Typically, surveys contain a combination of open- and closed-ended questions. Open-ended questions ask the respondent to provide an answer to a particular question. For example, the respondent may be asked: "What do you think is the most important problem facing residents in your neighborhood today?" Then in their own words, the respondent would provide his or her answer. On the other hand, closed-ended questions ask the respondents to select an answer from a list of choices provided. For example, the question asked above would read exactly the same only now respondents are provided with a list of options to choose from: "What do you think is the most important problem facing residents in your neighborhood today? (a) crime, (b) drugs, (c) education, (d) employment, (e) family structure, (f ) poverty, (g) health care, (h) child care, (i) extracurricular activities, ( j) other."

Surveys offer a number of attractive features that make them a popular method of doing research. They are versatile, efficient, inexpensive, and generalizable. At the same time, survey methods may be limited due to problems in sampling, measurement, and overall survey design. When creating a survey, researchers should take care in making sure that the items in the survey are clear and to the point.

Experimental and quasi-experimental research. Some scholars believe that experimental research is the best type of research to assess cause and effect (Sherman; Weisburd). True experiments must have at least three features: (1) two comparison groups (i.e., an experimental group and a control group); (2) variation in the independent variable before assessment of change in the dependent variable; and (3) random assignment to the two (or more) comparison groups (Schutt).

Many experiments contain both a pre-test and a post-test. The former test measures the dependent variable prior to the experimental intervention while the latter test measures the outcome variable after the experimental group has received the treatment. Randomization is what makes the comparison group in a true experiment a powerful approach for identifying the effects of the treatment (Schutt). Assigning groups randomly to the experimental and comparison groups ensures that systematic bias does not affect the assignment of subjects to groups. This is important if researchers wish to generalize their findings regarding cause and effect among key variables within and across groups.

The classic experimental design is one in which there is a pre-test for both groups, an intervention for one group (i.e., the experimental group), and then a post-test for both groups. Consider the following criminal justice example. Two police precincts alike in all possible respects are chosen to participate in a study that examines fear of crime in neighborhoods. Both precincts would be pre-tested to obtain information on crime rates and citizen perceptions of crime. The experimental precinct would receive a treatment (i.e., increase in police patrols), while the comparison precinct would not receive a treatment. Then, twelve months later, both precincts would be post-tested to determine changes in crime rates and citizen perceptions.

There have been several experimental designs in criminology and criminal justice including the Domestic Violence Experiment (Sherman), where offenders were randomly assigned to one of three interventions (arrest, mediation, separation). The Jersey City Police Department's Program to Control Violent Places also utilized an experimental design (Braga et al.). For this study, twenty-four high-activity, violent crime places were matched into twelve pairs and one member of each pair was allocated to treatment conditions in a randomized block field experiment.

On the other hand, quasi-experimental research lacks the random assignment to experimental and control groups, but can be approximated by close and careful matching of subjects across the two groups on several key variables. The two major types of quasi-experimental designs are: (1) nonequivalent control group designs, which have experimental and comparison groups that are designated before the treatment occurs and are not created by random assignment; and (2) before-and-after designs, which have both a pre- and post-test but no comparison group (Schutt).

An example of a nonequivalent control group design is a study of the effect of police actions on seat-belt law violations. For example, Watson selected two communities of comparable size where police enforcement of the law was low. In the experimental community, Watson instituted a media campaign to increase seat-belt usage, followed by increased police enforcement of the seat-belt law. Watson found that the percentage of drivers using seat belts increased in the experimental community but remained stable or declined slightly in the comparison community.

An example of the before-and-after design is the Pierce and Bowers analysis of the impact of the Massachusetts Bartley-Fox gun law. This law carried a one-year minimum prison sentence for the unlicensed carrying of firearms. Their early evaluation showed a decrease in gun-related assaults, robberies, and homicides, but was offset by increases in nongun assaults and robberies using other weapons.

Cross-sectional research. Cross-sectional designs involve studies of one group at one point in time. Therefore, they offer a quick glimpse or snapshot of the phenomena being studied. Typically, they refer to a representative sample of the group and thus allow researchers to generalize their findings (Hagan). Cross-sectional research designs permeate criminology and criminal justice research. Hirschi's famous study of causes of delinquency utilized a cross-sectional design in which he asked male respondents a series of questions related to involvement in delinquent activities and emotional ties to social bonds.

Longitudinal research. There are two commonly used longitudinal research designs, panel and cohort studies. Both study the same group over a period of time and are generally concerned with assessing within- and between-group change. Panel studies follow the same group or sample over time, while cohort studies examine more specific populations (i.e., cohorts) as they change over time. Panel studies typically interview the same set of people at two or more periods of time. For example, the National Crime Victimization Survey (NCVS) randomly selects a certain number of households from across the United States and interviews a member from each a series of seven times at six-month intervals. Cohort studies follow individuals or specific cohorts as they change over time. One classic example of a cohort study was conducted by Marvin Wolfgang and his colleagues in Philadelphia. The authors traced the criminal records of all boys born in Philadelphia in 1945 through the age of eighteen. Similarly, Tracy, Wolfgang and Figlio tracked the criminal history of males and females born in Philadelphia in 1958.

Time-series designs. Time-series designs typically involve variations of multiple observations of the same group (i.e., person, city, area, etc.) over time or at successive points in time. Typically, they analyze a single variable (such as the crime rate) at successive time periods, and are especially useful for studies of the impact of new laws or social programs (Schutt). An example of a time-series design would be to examine the murder rate in the United States over the last twenty years or to compare the murder rate of the United States and Canada over the same period of time.

An interrupted time-series design analyzes a single variable at successive time periods with measures taken prior to some form of interruption (i.e., intervention) and other observations taken after the intervention. An example of an interrupted time-series design may be found in Spelman and Eck (1987). These authors studied the number of larcenies from automobiles in Newport News, Virginia. The intervention in this study was a problem-oriented policing program that consisted of special tracking and investigation of crime incidents. The results showed that the number of larcenies dropped significantly immediately after the intervention took place and remained significantly small for over one year after the intervention. In another interrupted time series study, D'Alessio and Stolzenberg investigated the impact of Minnesota sentencing guidelines on jail incarceration. They found that the onset of the sentencing guidelines increased judicial use of the jail sanction beyond the effect of preexisting trends.

Although time-series designs are especially useful in studying trends over time and how such trends are influenced by some sort of intervention, researchers should be aware of one key feature of time-series designs: the inability to control for all potential spurious effects. Consider the following example. Suppose that a researcher is studying the effect on robberies of a mandatory convenience store law that requires stores to have at least two clerks working during hours of operation. After examining the number of robberies before and after the law took effect, the researcher observed that the number of robberies significantly decreased after the law was instituted. Therefore, the researcher claimed that the law led to the decrease in the number of robberies committed and concluded that the law should be generalized to other locales. However, what the researcher may have failed to consider was the recent capture of two offenders who were committing 75 percent of all convenience store robberies, and who just happened to be captured about the time the law took effect. In sum, researchers need to be careful in making sure that their interpretations of interrupted time-series analyses take into consideration as much information, both empirical and nonempirical, as possible.

Meta-analysis. A recent advent in research methodology is the use of meta-analysis. This research approach is the quantitative analysis of findings from multiple studies. At its core, meta-analysis involves researchers pulling together the results of several studies and making summary, empirical statements about some cause and effect relationship. A classic example of meta-analysis in criminology was performed by Wells and Rankin and concerned the relationship between broken homes and delinquency.

After observing a series of findings showing that the broken-homes-causes-delinquency hypothesis was inconclusive, Wells and Rankin identified fifty studies that tested this hypothesis. After coding the key characteristics of the studies, such as the population sampled, age range, measures (both independent and dependent) used, the authors found that the average effect of broken homes across the studies was to increase the probability of delinquency by about 10 to 15 percent. Perhaps more importantly, they found that the different methods used across the studies accounted for much of the variation in estimating the effect of broken homes. For example, the effect of broken homes on delinquency tended to be greater in studies using official records rather than self-report surveys.

Although the research community has not spoken with one voice regarding the usefulness of meta-analysis, one thing is clear: meta-analysis makes the research community aware that it is inappropriate to base conclusions on the findings of one study. It is because of this important lesson that meta-analysis has become a popular technique in criminological and criminal justice research (Lipsey and Wilson).