Our brains process irony in emojis, words in the same way

That winky-face emoji that you use at the end of a text isn't just a fun picture added to your sentence. It can convey linguistic meaning that changes the interpretation of the sentence, a new study finds.

University of Illinois researchers studied the brain wave patterns of people reading sentences paired with emojis. Sometimes the emojis matched the literal meaning of the sentence, but in other cases they were construed as indicating irony. Researchers found the participants' brains processed the ironic emojis in the same way they process ironic language.

The results of the research by Benjamin Weissman, a U. of I. doctoral student in linguistics, and Darren Tanner, until recently a U. of I. linguistics professor, appear in the journal PLOS ONE.

"Emojis are ubiquitous. They are seemingly everywhere, and people use them a lot in text messages and online along with language," Weissman said. "But nobody in linguistics or psychology had looked at them using the experimental methods we have at our disposal, like looking at the electrical activity generated by the brain in real time. Looking at brain activity to understand how we process language is pretty common. But people hadn't studied emojis paired with language like this."

The researchers looked at brain-wave patterns associated with language processing, including a pattern called the P600, a kind of "error signal" that occurs when a person encounters something linguistically unexpected. The signal indicates the brain is reprocessing or reanalyzing the content of the sentence.

This signal was traditionally associated with grammatical errors, Tanner said, but previous studies have shown verbal irony also triggers this type of brain wave. Weissman and Tanner hypothesized, if emojis can convey linguistic meaning in the same way as words, they would see similar brain-wave patterns with people reading sentences that include an ironic emoji.

"Irony or sarcasm is a really conventionalized use of emojis," Tanner said. "People have trouble understanding what some emojis mean, even in context. The exception is with sarcasm or irony. The winky-face emoji conveys a certain meaning that people understand."

In three experiments, a total of 106 participants in the study read sentences followed by an emoji – either a smile, frown or wink – while their brain waves were recorded. The participants then answered comprehension questions about how they ultimately interpreted the meaning of the sentences. The questions probed whether the participants had interpreted the sentence literally, based on only the words, or whether the wink emoji had changed their interpretation to a nonliteral (ironic) reading of the sentence.

When comparing the brain responses elicited by the ironic (wink) emojis with the nonironic emojis (smile or frown), Weissman and Tanner found the brain-wave patterns elicited by the ironic emojis looked just like those seen in previous studies with ironic verbal language. More interestingly, they said, the size of each participant's brain response to the ironic emojis reflected how willing that participant was to let the wink emoji override the literal meaning of the sentence. This indicates that this brain response pattern is a reflection of the "work" that the readers' brains are doing to reinterpret the sentence in light of the wink emoji, just like when a person reinterprets the meaning of a sentence when someone is being sarcastic.

Weissman and Tanner said their research shows words combined with emojis can be considered another form a multimodal communication, similar to words plus gestures or words plus facial expressions.

"Spoken and signed languages evolved long before humans developed written representations of language, and spoken and signed languages have ways of enhancing or changing meaning through mechanisms like intonation or gesture," Weissman said. "Written language largely strips those extra modes for conveying meaning away, but emojis seem to be an important recent development that adds some of these mechanisms back into written forms of language. Emojis can convey irony or sarcasm in a written format in the same way we might use intonation to convey the same thing when speaking.

"I wouldn't go so far as saying emojis are words, but they can be used linguistically," he said. "And what this study shows is that for things like sarcasm, our brains can use information from emojis almost immediately, and in the same way as language."

"This helps broaden our view of communication. It can be words, or words plus pictures, or words plus gesture, and it can be words plus emojis," Tanner said. "You can't just use a string of emojis, but when you pair emojis with words, they can enhance each other. The combined effect of written words plus emojis is bigger than words or emojis alone."