Can Machine Translation Facilitate Better Global Meetings in a Common Language?

A group of researchers set out to prove that team members who are globally distributed can better prepare for general, common-language meetings by sharing information ahead of time. Specifically, the researchers hypothesized that exchanging logs using machine translation (MT) could benefit teamwork by enabling comprehension.

In a research article published in the journal of the Association for Computing Machinery in April 2022, Yongle Zhang, Dennis Asamoah Owusu, Marine Carpuat, and Ge Gao, from the University of Maryland in the US, detailed an experiment conducted with native speakers of English and Mandarin to prove their theory.

A general premise behind the theory is that “communication at team meetings is often less effective than in subgroup conversations.” Subgroup conversations are held in the native language of the participants, who are then required to use a second language, often English, to communicate with other subgroups about work matters.

Prior research supports this premise with plenty of qualitative data, as can be seen in the article’s extensive reference section. This study adds quantitative data that corroborates anecdotal evidence, albeit in a controlled experiment with only one language pair.

MT and Other Interventions Render Mixed Results

Some distinct challenges of having multiple language teams have been documented in previous research. These challenges include the assumption that information exchanged in subgroups is common knowledge for the whole team, the existence of diverging mental models when giving and receiving information, and the use of ineffective synchronous and asynchronous interventions.

Interventions most recently examined and reported in the cited literature include MT (synchronous) for single sentences, automatic speech recognition (ASR) for oral conversation scripts, and image retrieval to illustrate the meaning of text messages.

These interventions are useful, but do not work across all settings. The University of Maryland researchers point out that a reason for this might be that such interventions are linguistically (rather than contextually) driven.

Task Experiments in Two Languages

The study involved 20 teams of 4 people each: 2 native English speakers and 2 native speakers of Mandarin. All 80 participants were given the same task, which consisted in selecting, as a group, the best candidate out of four for a research-assistant job. The task had to be completed under the same conditions and the outcome was measured using uniform criteria.

Each person in a subgroup had different information about the job candidates, a situation that created interdependencies and the need for discussion. Participants received materials and discussed the task particulars in their native language over instant messaging.

The groups were then randomly assigned an MT intervention (translating discussion logs) or no intervention (no translation). And then they joined a meeting conducted in English. Task completion results for both groups were examined in detail using various statistical analyses.

MT Logs Added Clarity, Depth, Comfort

The analyses measuring communication quality showed a higher level of perceived clarity and comfort among the groups that exchanged machine-translated logs with their peers, regardless of native language.

The data showed that subgroups in the MT-intervention branch performed the task better as a team. They also held deeper discussions, demonstrated better comprehension, and showed higher confidence levels in the task outcome — even if they had actually chosen the wrong candidate against a single, possibly correct choice.

The data collected validated the hypothesis that machine-translated logs prepare team members for better meetings in a common language.

The process also offers opportunities to further improve translation quality from a lexical perspective. An MT engine could be further trained to improve context-dependent consistency in conversation logs.