Amazon Prime Video Launches AI Dubbing Pilot

Amazon has announced that it has launched an AI dubbing pilot for licensed movies and series on its entertainment streaming platform Prime Video.

The pilot involves a “hybrid approach to dubbing in which localization professionals collaborate with AI to ensure quality control,” making its vast streaming library accessible to more of its existing customer base of 200 million customers worldwide.

Raf Soltanovich, VP of Technology at Prime Video and Amazon MGM Studios said, “AI-aided dubbing is only available on titles that do not have dubbing support, and we are eager to explore a new way to make series and movies more accessible and enjoyable.”

The initial pilot covers 12 licensed movies and series in English and Latin American Spanish that “would not have been dubbed otherwise”, and are now available to users on the platform. The launch includes titles such as El Cid: La Leyenda, Mi Mamá Lora, and Long Lost.

The global e-commerce giant has recently released details of how its researchers are improving lip-sync output in AI dubbing through a visual speech-to-speech translation (AVS2S) framework, without altering the original visual content.

“AI-aided dubbing is only available on titles that do not have dubbing support” — Raf Soltanovich, VP of Technology, Prime Video and Amazon MGM Studios

The improvements focus on preserving the original videos and voice characteristics of the original speakers, integrating duration and synchronization loss into the training process.

Amazon’s Prime Video AI pilot is the most recent development in the company’s rollout of its AI dubbing technology. The company published a step-by-step AI dubbing tutorial in July last year, instructing users on how to use auto-dubbing using Amazon Translate, Amazon Bedrock, and Amazon Polly.