The traditional “broken windows theory” goes that acts of public disorder in neighborhoods—such as graffiti, litter, and abandoned homes—can encourage future crime there. But now research led by Northeastern University assistant professor Daniel T. O’Brien has leveraged Big Data to shed new light on the factors that predict crime in urban neighborhoods.

The researchers found that, in fact, private conflict may be a stronger predictor of crime in a community.

“Our research suggests that the ‘broken windows model’ doesn’t effectively capture the origins of crime in a neighborhood,” O’Brien said. “What’s happening is that violent crime is bubbling out from the social dynamics of the community, out from these private conflicts that already exist, and then is escalating and spilling into public spaces.”

O’Brien holds joint appointments in the School of Public Policy and Urban Affairs and the School of Criminology and Criminal Justice, and his research uses Big Data—most often in the form of large administrative data sets generated by city government—in conjunction with traditional methodologies to explore the behavioral and social dynamics of urban neighborhoods. This work is emblematic of his classroom teaching, as students in his Big Data for Cities course are immersed in the emerging field of “urban informatics.”

O’Brien is also the research director of the Boston Area Research Initiative, which undertakes and supports cutting-edge urban research at the intersection of social science and public policy.

The findings, O’Brien said, show how Big Data can be used to advance the study of neighborhood dynamics and to assess and mitigate crime. His collaborators were Harvard University professors Robert J. Sampson and Christopher Winship, who are director and co-director, respectively, of the Boston Area Research Initiative.

In a study with Sampson and Winship, O’Brien developed an ecometric methodology that translated more than 300,000 non-emergency calls to 311 in the city of Boston during 2011 and 2012 into measures of physical disorder, such as graffiti and the accumulation of litter. The purpose of this methodology, which was published this summer in the journal Sociological Methodology, was to create a detailed set of metrics for a city to use in assessing neighborhood dynamics several times a year.

Then, O’Brien and Sampson put the broken windows theory to the test by applying this methodology to more than 1 million 311 and 911 calls in that same time period. The goal was to not only measure social disorder and crime, but to examine how it changed over time. The results, published this summer in the Journal of Research in Crime and Delinquency, pointed to a “social escalation model” in which future disorder and crime emerge not from public cues but from private disorder within the community.

Using the 911 and 311 data the researchers developed six measures—public social disorder, public violence not involving guns; domestic violence and other private conflicts; gun violence, and private neglect in neighborhoods, and public denigration in neighborhoods. Upon examining the connections between these six factors, here’s what they found:

•    Private conflict was the strongest leading indicator of crime in the model, predicting increases in social disorder, public violence, guns, and even physical disorder in privately owned spaces.
•    Physical and social forms of public disorder were weakly predictive of future violence and disorder, if at all; public denigration had no predictive power, and the link from public social disorder to later public violence was half the magnitude of the reverse pathway from violence to social disorder.

The researchers noted that the study did not demonstrate causality; in other words, why private conflict was such a stronger indicator of crime than private neglect and public denigration.

O’Brien and his colleagues theorized that a stressful social ecology like that reflected in private conflict among residents can drive behavior that leads to multiple consequences for a neighborhood as a whole. For example, friendship disputes or domestic violence can spill out into the public space, and those incidents are likely to increase in severity over time—a progression that until now has been largely invisible to researchers because traditional forms of assessment such as neighborhood surveys or observations only capture what’s visible in the public space and not what’s behind closed doors.

“With these data sets, the methodology becomes a tool for reliably and continuously tracking and analyzing the conditions of the city,” O’Brien said.

From the city to the classroom
O’Brien said the type of large administrative data sets that proved to be crucial in these studies will be the focus of his Big Data for Cities course this fall. In this course, undergraduate and graduate students learn how to manage and analyze large data sets—such as restaurant inspection violation records and the tax assessor’s database for the city of Boston—by pursuing group research projects focused on specific city resources and services.

The course, which launched in fall 2014, dovetails with the university’s new M.S. in Urban Informatics, which couples comprehensive data analytics skills with an understanding of the big questions cities face in the 21st century.

“What’s special about these Big Data is that we’re all still trying to figure out what to do with them, in terms of translating them into societal impact and improving municipal services,” he said. “The students [last fall] were excited to take on this challenge, and it’s been inspiring to see them draw out their ideas and conclusions.”