A team of researchers from the University of Cambridge and Dartmouth College has developed a new device that uses infrared light to monitor how body language impacts social interactions.

Because body language influences many aspects of people’s lives, including job interviews, doctor-patient conversations and group projects, many people have tried to study them through video sessions, audio recordings and paper questionnaires.

However, these approaches can be biased, inaccurate and burdensome on users for most of them require obtrusive devices, such as invasive cameras.

To address this, the researchers developed a device, called a Protractor, that uses infrared light to measure non-verbal behaviors that can potentially show how people interact in social settings.

“Our system is a key departure from existing approaches,” Xia Zhou, assistant professor of computer science at Dartmouth and co-lead author of the study, said in a statement. “The ability to sense both body distance and relative angle with fine accuracy using only infrared light offers huge advantages and can deepen the understanding on how body language plays a role in social interactions.”

The study is published in the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, and will be presented at the UbiComp 2018 conference to be held Oct. 8-12.

The Protractor

Resembling a wearable access badge worn with a lanyard or clip, a Protractor uses near-infrared light, operating at a wavelength commonly used in TV remote controls and invisible to human eyes. It measures body language, such as body angles and distances between people.

“The ability to use invisible light to determine someone’s role and attitude in social settings has powerful implications for individuals and organizations that are concerned about how they communicate,” Cecilia Mascolo, professor of mobile systems at the University of Cambridge, said in a statement.

In order to correct for when a user’s hand or clothing could temporarily block the infrared light channel, the researchers designed algorithms that exploit inertial sensors to work around the absence of light tracking results.

In addition, the researchers modulated the light from each protractor tag to encode each light with a specific tag ID so they could identify individuals wearing Protractor tags, according to Zhao Tian, a doctoral candidate at Dartmouth. They also adapted the frequency of emitting light signals based on the specific context to limit power consumption.

The Marshmallow Challenge

To test its effectiveness, the researchers used the Protractor tags to track body language during a problem-solving group task known as “The Marshmallow Challenge.”

For this task, teams of four members were given 18 minutes to build a structure that could support a marshmallow using tape, string and a handful of spaghetti.

“We focused on two types of body language: the distances between users, and their relative body orientation,” said Zhou. “These pairwise features can be aggregated as features to infer the instant task role of each team member, and the timeline, or stages, of the building process of the Marshmallow challenge.”

In the study of 64 participants, Protractor tags achieved 1 to 2-inch mean error in estimating distances between users and less than 6 degrees error 95 percent of the time for measuring relative body orientation.

Using these measurements, the researchers assessed each individual’s task role with close to 85 percent accuracy and identified stages in the building process with over 93 percent accuracy.

Future Uses

Protractor tags will not only be used in social researches, but also in many other important real-life settings.

They can provide real-time feedback during interviews, understand team dynamics to achieve higher creativity, and study body language impacted by cultures in today’s increasingly internationalized workplaces.  “They (companies or institutions) can use Protractor results to study the correlation of our social interaction patterns with our personality, mental states, productivity, and cultural norms,” said Zhou.

“They can use the results to possibly assess the effectiveness of team collaboration, and to train one’s engagement skills, which are essential for job interviews, doctor-patient interactions, sales training/orientation and more.”