“We will continue enhancing our automated dubbing solution to deliver high-fidelity, out-of-the-box results, complemented by an efficient human-in-the-loop (HITL) tool. Additionally, we’ll continue advancing our lip-sync technology to handle more complex scenarios and improve audio performance to unlock a broader range of use cases,” he added.
In parallel with Panjaya’s funding is the startup’s launch of BodyTalk — an AI dubbing platform that synchronizes lips and body language across languages.
Slator Pro Guide: Audiovisual Translation
The Slator Pro Guide: Audiovisual Translation is a concise guide to audiovisual translation, including dubbing, subtitling, access services, AI dubbing, AI captions, and more.
According to the company’s website, BodyTalk “automatically adjusts translations and aligns the rhythm and tone of the dubbed audio with the pauses, breaths, gestures and other body movements in the original video.”
Piekarz told Slator that the technology is powered by “a set of proprietary algorithms to analyze videos, and then adapts outputs from best of breed translation and text-to-speech services [using] home-grown AI models for lip syncing.”
Early adopters of the technology include Helena Batt, Director of Localization at TED Conferences, who recently told a SlatorCon audience that Panjaya helped TED take a “huge leap forward” in synthetic dubbing.
Now that the technology has officially launched, Piekarz told Slator that the company is targeting “professional training, marketing, and media-related use cases,” having recently experimented with a full-length documentary.
“This initiative is helping us explore the boundaries of AI and synthetic voice applications and understand how far we can push our current tech. […] We’re advancing our lip-sync technology to handle complex real-world scenarios, such as dynamic camera angles, speaker occlusions, and interactions between multiple speakers,” he concluded.
