Facebook Tests Post-Editing for Neural MT Through “Very Simple Interactions”

New research posted to Arxiv by the Facebook AI Research team shows that the social networking giant is experimenting with a new system called QuickEdit, which is meant to improve its neural machine translation (NMT) output through post-editing.

In their paper, the Facebook team explained that the system “relies on very simple interactions: a human editor modifies a sentence by selecting tokens they would like the system to replace and no other feedback. Our system then generates a new sentence which reformulates the initial sentence by avoiding to use the words from the selected tokens.”

The Facebook team performed tests on the widely used WMT-14 English-German translation task and used BLEU as a standard measure for quality.

Rejected Tokens

They essentially placed an additional layer on top of an existing neural MT stack.

First, neural MT translated a source sentence into the preferred language. A program that simulates a human post-editor then compared that output to the most accurate translation and marked poorly translated parts.

This is where QuickEdit comes in: “This model takes as input a source sentence and an initial guess target sentence annotated with rejected tokens. It can then decode a new target sentence taking the rejection labels into account.”

QuickEdit basically took the source sentence and the post-edited sentence with parts marked as poor translations, and used these to come up with an improved translation that the Facebook team says is much closer to the most accurate translation.

According to the paper, QuickEdit “[shows] +11.4 BLEU with limited post-editing effort… which represents +5.9 BLEU over the post-editing baseline.”

The post-editing effort is limited to marking parts that do not work; no actual translation suggestions are offered. QuickEdit takes the suggestions and decides on its own.

Analogous to a Hypothetical Online Translation Service

The paper went on to say “this is analogous to a hypothetical online translation service which offers a feature enabling the user to progressively mark the parts of a translation which needs to [be] improved.”

To speculate about a potential future use-case for the research: when applied to Facebook’s automated translation of user-generated content, QuickEdit may enable users to simply tag which parts of a translation they think are improper. Facebook’s system will then automatically replace those parts with better options going forward. This would be easier from a user perspective than engaging in actual post-editing as required by Facebook’s current “I have a better translation” option displayed on posts.

Slator first reported on Facebook’s neural machine translation (MT) ambitions in June 2016, when they first said statistical MT has reached its end of life. Good call. The social network switched quickly and completed the transition to neural a little over a year later in August 2017.

Download the Slator 2019 Neural Machine Translation Report for the latest insights on the state-of-the art in neural machine translation and its deployment.