Google’s Latest Phone, Pixel 6, to Feature Live Translated Captions for Audio and Video

Google has unveiled details around the latest in its line of smartphones, Pixel 6, slated for fall release 2021. In a departure from the smartphone tech status quo, Pixel 6 and its higher-end version, Pixel 6 Pro, will feature Google’s own system on a chip, Tensor.

Google SVP of Devices and Services, Rick Osterloh, wrote in an August 2, 2021 blog post, “Tensor was built for how people use their phones today and how people will use them in the future. As more and more features are powered by AI and ML […] it’s about using that ML to unlock specific experiences for our Pixel users.”

Communicating in different languages is among those experiences, Osterloh added. Google’s Tensor processing unit (TPU) powers Pixel 6 in generating live translated captions for audio or video content in another language.

Although Google Translate and similar services already offer offline text translation, offline voice translation, which requires the extra step of automatic speech recognition (ASR), is the more challenging proposition — as revealed by Google’s own efforts in rolling out Translatotron.

With Tensor, Pixel 6 can operate Google’s Live Caption and Interpreter Mode features simultaneously, something not possible with previous chips.

Slator 2021 Language Industry Market Report

80-pages. Market Size by Vertical, Geo, Intention. Expert-in-Loop Model. M&A. Frontier Tech. Hybrid Future. Outlook 2021-2025.

$680 BUY NOWIncluded in our Pro and Enterprise plan.
Subscribe now!

“Speech recognition is another foundational technology where you will see a huge improvement in #Pixel6,” according to an August 2, 2021 tweet from Google’s product-focused Twitter account, @MadebyGoogle. “Google Tensor allows us to make big leaps in: voice commands, translation, captioning, & dictation,” claimed a follow-on tweet.

Osterloh has already demonstrated Pixel 6’s “nearly instantaneous” talk-to-type function. Users can continue to speak while editing the resulting text by using their keyboard. In another demo, Osterloh turned on both live captions and live translation, converting a French recording into English.

Fans from various countries are already clamoring for the Pixel 6 and Pixel 6 Pro. Playing coy, Google said they will share more details in the fall. The tech behemoth has also yet to specify which languages (or even how many) will be covered by the live-translated-caption feature.

Pixel 6’s capabilities represent just one possible productization of Google’s recent research into speech-to-speech translation. It will be interesting to see how far Google can go after quintessential smartphone maker Apple struggled to gain a following for the iPhone’s Translate app since its September 2020 release.