Facial Expressions: How Brains Process Emotion

Have you ever thought someone was angry at you, but it turned out you were just misreading their facial expression? Caltech researchers have now discovered that one specific region of the brain, called the amygdala, is involved in making these (sometimes inaccurate) judgments about ambiguous or intense emotions. Identifying the amygdala's role in social cognition suggests insights into the neurological mechanisms behind autism and anxiety.

The research was done in the laboratories of Ralph Adolphs, Bren Professor of Psychology and Neuroscience and professor of biology, and collaborator Ueli Rutishauser (PhD '08) of Cedars-Sinai Medical Center in Los Angeles and a visiting associate in biology and biological engineering at Caltech. It appears in the April 21 issue of Nature Communications.

"We have long known that the amygdala is important in processing emotion from faces," says Adolphs. "But now we are starting to understand that it incorporates a lot of complex information to make fairly sophisticated decisions that culminate in our judgments."

When looking at a face, brain cells in the amygdala fire electrical impulses or "spikes" in response. However, the role of such face cells in social cognition remains unclear. Adolphs and his group measured the activity of these cells, or neurons, in patients while they were shown images of faces expressing different degrees of happiness or fear. The subjects were also shown images of faces with more ambiguous or neutral emotions, such as moderate displeasure or muted happiness. For each type of image, subjects were asked to decide whether the face looked fearful or happy. The researchers then investigated how neurons reacted to different aspects of emotions, and how the activity of the face cells related to the decision made by the subjects.

The researchers found that there are two groups of neurons in the amygdala that respond to facial emotions.

One group, the emotion-tracking neurons, detects the intensity of a single specific emotion, such as happiness or fear. For example, a happiness-signaling neuron would fire more spikes if the emotion was extreme happiness, and fewer spikes if the emotion was mild happiness. Separate groups of neurons within the emotion-tracking neurons code specifically for fear or for happiness.

The other group, the ambiguity-coding neurons, indicates the ambiguity of the perceived emotion, irrespective of the nature of that emotion.

Showing patients images of emotionally ambiguous faces was the key to understanding how the specialized neurons in the amygdala contribute to decision making, the researchers say. The faces were so ambiguous that a patient would sometimes judge the same image to be fearful at times and happy at other times. The emotion-coding neurons indicated the subjective decision the patient made about the face.

"Most people are familiar with feeling that a face just looks too ambiguous to really decide what emotion the person is having," says first author and visitor in neuroscience Shuo Wang (PhD '14). "The fact that amygdala neurons signal a decision made about a face, such as which emotion it shows, gives us important insight because it shows that the amygdala is involved in making decisions rather than simply representing sensory input."

In addition to recording single cells from the amygdala, the researchers also carried out a neuroimaging study using fMRI (in a separate group of participants), and additionally studied the emotion judgments of three rare subjects with lesions of the amygdala. The lesion subjects showed an abnormally low threshold for deciding when a face was fearful, and the fMRI study also showed the specific effect of emotion intensity and ambiguity in the amygdala. The study is the first to combine so many different sources of data.

These findings also suggest a mechanistic basis for potential treatments involving the painless electrical stimulation of the amygdala, which are currently being studied in ongoing clinical trials. "Researchers at multiple institutions are currently evaluating whether deep-brain stimulation of the amygdala is effective in treating severe cases of autism or post-traumatic stress disorder," says Rutishauser. "Patients with severe PTSD are thought to have a hyperactive amygdala, which electrical stimulation might be able to inhibit. Our findings that amygdala neurons carry signals about the subjective percept of emotions indicates a more specific reason for why such electrical stimulation might be beneficial."

The paper is titled "The human amygdala parametrically encodes the intensity of specific facial emotions and their categorical ambiguity." Michael Tyszka, associate director of the Caltech Brain Imaging Center, is an additional Caltech coauthor. The Caltech Brain Imaging Center is one of the affiliated research centers in the Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech. The work was funded by the Autism Science Foundation, the Simons Foundation, the National Institute of Mental Health Conte Center, and the National Science Foundation.