To address this challenge, the researchers explored more efficient approaches to collect high-quality translation data. They compared 11 different approaches — including human-only, machine-only, and hybrid methods — to determine the most effective and cost-efficient one.
Slator Translation as a Feature (TaaF) Report
The Slator Translation as a Feature (TaaF) Report is a vital and concise guide on how AI translation is becoming an integral feature in enterprise technology.
Human-only workflows involved either a single human translation step or included an additional one or two human review steps. Machine-only workflows ranged from single-step AI translations using top AI systems — MT systems or LLMs — to more complex workflows, where AI translations were refined by an LLM. Hybrid workflows combined human expertise and AI efficiency; in some cases, AI translations were refined by humans (i.e., post-editors), while in others, human translations were refined by LLMs.
“Human-machine collaboration can be a faster, more cost-efficient alternative to traditional collection of translations from humans”. — Liu et al.
They found that combining human expertise and AI efficiency can achieve translation quality comparable to, or even better than, traditional human-only workflows — all while significantly reducing costs. “Our findings demonstrate that human-machine collaboration can match or even exceed human-only translation quality while being more cost-efficient,” the researchers said.
The best combination of quality and cost appears to be human post-editing of AI translations. This approach delivered top-tier quality at only 60% of the cost of traditional human-only methods, while maintaining the same level of quality.
“This indicates that human-machine collaboration can be a faster, more cost-efficient alternative to traditional collection of translations from humans, optimizing both quality and resource allocation by leveraging the strengths of both humans and machines,” they noted.
The researchers emphasized that the quality improvements stem from the complementary strengths of human and AI collaboration, rather than from the superior capability of either the AI or the human (post-editor) alone, underscoring the importance of leveraging both human and AI strengths to achieve optimal translation quality.
2024 Slator Pro Guide: Translation AI
The 2024 Slator Pro Guide presents 20 new and impactful ways that LLMs can be used to enhance translation workflows.
They noted that LLMs were less effective than human post-editors at identifying and correcting errors in AI-generated translations. On the other hand, human reviewers tended to make fewer changes when reviewing human-generated translations, possibly overlooking certain errors. Interestingly, even additional rounds of human review did not substantially improve the quality. This observation supports the argument for human-machine collaboration, where each component helps address the other’s blind spots, according to the researchers.
“These findings highlight the complementary strengths of human and machine post-editing methods, indicating that a hybrid method is likely the most effective strategy,” they said.
Authors: Zhongtao Liu, Parker Riley, Daniel Deutsch, Alison Lui, Mengmeng Niu, Apu Shah, and Markus Freitag
