Earlier this month the Affective Science Institute hosted another great event called Reading the Face: Translating Science to Security. Three emotion scientists, including Northeastern’s Lisa Feldman Barrett, discussed their unique research approaches to studying the human ability to detect a person’s emotion by looking at his or her face.

In the wake of 9/11, large-scale security programs were rolled out at airports across the country in which security personnel were trained to stop individuals that looked suspicious. In addition to the researchers, the event brought Peter DiDomenica, former Director of Security Policy at Boston Logan International airport who developed the TSA’s behavior-based screening program. Speaking from experience, DiDomenica said that security guards have used the technique to arrest criminals trying to fly for various reasons, from drug smuggling to skipping the country to launching a terrorist attack on-board.

But the science, said Barrett and her colleagues Jon Freeman of Dartmouth College and Philippe Schyns of the University of Glasgow, is unable to support the hypothesis that we can determine who is a threat by simply looking at peoples’ faces.

Freeman uses mouse tracking to monitor how quickly research subjects are able to categorize faces. When a gender ambiguous face is presented on a computer screen for instance, Freeman’s group looks at the actual trajectory of the mouse toward its ultimate destination: a button with either male or female written in the right or left corner of the screen. He has also looked at how our categorical decisions change based on context. In one experiment, he presented study subjects with similar faces across a range of skin-colors. When the faces belonged to men wearing “high-status” clothing, such as a business suit, subjects categorized darker faces as Caucasian whereas the opposite was true when those same men wore “low-status” clothing, such as a janitorial uniform.

Schyns uses brain imaging to understand the neurological underpinnings to these categorization decisions, which happen in an instant but actually require very complex machinery to take place. He’s shown that there are very subtle but important differences across cultures for what signals different emotions like happiness or anger. For example, individuals from eastern cultures transmit more emotional information with their eyes whereas western individuals use their mouth more.

Barrett brought home the message that there is no single, universal facial expression for any single emotion, a hypothesis that has long dominated psychology teaching without much data to support it. She talked about research in which members of her lab looked at the differences between Americans and people of a remote Namibian tribe who have had very little exposure to western culture. They were able to distinguish smiling from neutral and wide-eyed fear faces, but not able to pick up on more subtle expressions, regardless of the face’s ethnic background. Americans were only to do a little better at the task.

So, to recap: we’re not terribly good at distinguishing individuals from one another. Different cultures demonstrate (and pick up on) emotion using different parts of the face.  And, finally, we’re not all that susceptible to facially transmitted emotional information to begin with. If all of this is true, then how can DiDomenico’s behavior-based screening program be even remotely successful?

Emotion detection is an important part of the evaluation, he said, but it’s not everything. The screening is designed to look at dozens of factors, from baseline attributes like the mood of the crowd, to how an individual answers benign questions like “how’s it going today?” to where the person is traveling to and from. Ultimately, threat detection experts use a range of information to do their jobs and only some of it comes from the face.

“Much of how we perceive each other is based on knowledge we have from our culture,” says Barrett. “This presents certain challenges for threat detection. The science of how we perceive each other gives us some clues, however, to what experts are doing, and how to train people to become experts.”

Apparently only one in 173 million passengers will be a terrorist. “We’re not necessarily going to catch him, but it puts one more obstacle in the way,” said DiDomenico. “It’s hard to measure the impact when something doesn’t happen. You don’t have to catch terrorists for it to be successful.”