Specific types of serotonin neurons modulate aggression

How does a small cluster of fairly homogeneous brain cells that make the neurotransmitter serotonin regulate so many functions, from breathing to mood to appetite?

The answer is that they're not so homogeneous after all. Over the past few years, researchers, including Harvard Medical School geneticist Susan Dymecki, have begun uncovering distinct subtypes of serotonin neurons that appear to have specific responsibilities.

Now, Dymecki and colleagues have identified two groups of serotonin neurons that help suppress aggression in male mice.

The discovery refines scientists' understanding of the complex neural circuitry underlying aggression and reveals a concrete connection between dopamine-producing neurons and the two specific subtypes of serotonin neurons in the mouse brain.

"These particular serotonin neurons are sensitive to dopamine in distinct and very specific ways," said Dymecki, senior author of the study published Nov. 15 in Cell Reports.

"There has been evidence of a general interplay between serotonin and dopamine," she said, "but here we've linked the two in a specific group of cells involved in modulating a specific behavior."

The authors propose that dopamine escalates aggression through a double whammy: both on its own, as has been shown by decades of research, and by suppressing the serotonin neurons that normally soothe aggression, according to the new study.

"A lot of times, you study neurotransmitters like dopamine or serotonin or norepinephrine but you don't necessarily understand how they're all working together," said Tedi Asher, a graduate student in the Dymecki lab and co-first author of the study. "Our work begins to make connections between these systems."

"It's particularly exciting here," Asher added, "where dopamine is thought to increase aggression and serotonin is thought to decrease it, to see that those two systems are talking to each other."

Although the research was limited to mice, Dymecki believes her team's observations will also hold true for humans, in part because the relevant area of the hindbrain is similar across mammals.

If confirmed in humans, the findings could lead to more precise treatments for the excess aggression that can arise in conditions such as schizophrenia, intermittent explosive disorder, autism and dementia.

Soothing serotonin

Serotonin is one of many neurotransmitters known to affect aggressive behavior in people and animals. Recent research has linked low serotonin with a higher likelihood of pathological aggression, and doctors sometimes prescribe selective serotonin reuptake inhibitors, or SSRIs, to boost serotonin levels in patients with impulsive aggression.

However, SSRIs are blunt instruments that strike all serotonin neurons. Trying to adjust one serotonin-regulated behavior can affect others, including breathing rate, mood, sex drive and appetite.

Dymecki realized that if she could find out whether only some of the neurons influence aggression, then it might be possible to develop treatments that target those cells alone and reduce unwanted side effects.

Dymecki's team began by silencing—that is, blocking neurotransmitter release from—the majority of serotonin neurons in a group of lab mice. As expected, those mice became more aggressive, biting, threatening and rattling their tails at other mice more often than did mice with intact serotonin systems.

In earlier studies, Dymecki and others showed that serotonin neurons can be subdivided based on patterns of genes that are turned on or off. Building on that work, the team took a new group of mice and silenced many subtypes one at a time, watching for any that significantly changed the rodents' aggression levels.

They found two: Drd1a/Pet1 neurons and Drd2/Pet1 neurons.

Dopamine surprise

Mice in which either neuron subtype was silenced were more aggressive. Silencing other subtypes of serotonergic neurons had no effect.

What makes those two special? For one thing, they have dopamine receptors on their surfaces—hence the "Dr" that begins their names. Drd1a/Pet1 cells are studded with dopamine receptor type 1, Drd2/Pet1 cells with dopamine receptor type 2.

Dopamine has a range of roles in the body, from motor control to reward motivation. The researchers discovered that dopamine suppressed Drd2/Pet1 neurons' ability to fire. Muffling the same neurons using genetic tools similarly dampened aggression in the mice.

The researchers also found that the majority of serotonin cells nestled right alongside Drd2/Pet1 neurons respond to dopamine in the opposite way, ramping up their activity. Dymecki's team didn't detect any dopamine receptors on their surfaces, leaving it a mystery what those cells are doing and how dopamine acts on them.

Further experiments revealed that Drd2/Pet1 cells' dopamine receptors activate during adolescence. By contrast, Drd1a/Pet1 cells express their dopamine receptors before and briefly after birth.

More to puzzle out

Both serotonin neuron subtypes differed from their brethren in other significant and sometimes unexpected ways.

It made sense to the researchers when they found that Drd1a/Pet1 cells communicate with regions of the brain known to influence aggression, but they were surprised to discover that the Drd2/Pet1 cells connect to regions involved in sensory processing.

"These differences in circuitry may indicate distinct means of impacting the same behavior: aggression," said Asher.

It's not yet clear why the mouse brain contains multiple types of dopamine-responsive serotonin neurons, each active at different stages of development. Dymecki's group intends to find out.

The present findings demonstrate that "a small population of serotonin neurons can have a profound effect on social interaction," said Dymecki. "We have a precedent now to deconstruct the serotonin dependency of, and the circuitry underlying, distinct aspects of social behavior."