How Good is ChatGPT o1 at Translation?

ChatGPT, the large language model (LLM) that arguably ushered in the modern AI era, and all its accompanying hype, has emerged in a new iteration: ChatGPT o1. How does this version compare to its predecessors — and more specifically, how well does ChatGPT o1 translate?

When Slator explored ChatGPT’s Q&A capabilities in October 2024, pitting OpenAI’s LLM against Google’s Gemini and Anthropic’s Claude, ChatGPT stood out for giving lengthy answers and daring to commit to its estimated figures (versus the more vague ranges provided by competitors).

Translation has already figured prominently in ChatGPT’s status and soaring celebrity. By June 2024, American doctors were investigating its use in medical cases, alongside the longstanding machine translation (MT) tool Google Translate. Also stateside, Minnesota’s Enterprise Translation Office selected ChatGPT for translation — with an eye toward possible future use for AI interpretation.

Indeed, for any LLM, mastering text translation is considered a prerequisite in the pursuit of any “downstream” tasks; namely, speech translation. OpenAI unveiled ChatGPT-4o’s real-time speech translation capabilities in May 2024, a testament to the model’s rapid advancement. It stands to reason that the versions of ChatGPT that follow will incorporate further progress.

Is that the case? OpenAI released ChatGPT o1 in December 2024 to much acclaim, following a September 2024 preview.

“I am obviously biased, but whatever, after literally years of consideration I think ChatGPT (especially o1 pro) is the most impressive thing since the iPhone,” wrote Ludwig Pettersson, the founder of Quill and former OpenAI staff member.

Slator’s informal survey of the model’s translation capabilities, tested primarily with European languages, found accurate and even nuanced translations, even among language pairs not including English.

Within the ChatGPT o1 interface, a menu tracks the user’s previous inquiries, by year, the past 30 days, and the present day. From here, users can also “explore other GPTs” and visit Sora, a video generation model.

Bundled with the launch of ChatGPT o1 announcement was OpenAI’s new subscription plan, which offers access to ChatGPT Pro, for USD 200 a month.

Alternatively, ChatGPT o1 can be used for free, with some restrictions — for example, users are alerted when they have “25 responses from o1 remaining.” According to the warning, “[i]f you hit the limit, responses will switch to another model until it resets” in a week’s time. (A drop-down menu also allows users to select their preferred model, whether ChatGPT-4o, o1-mini, or another.)

One striking detail of ChatGPT o1 is the description that appears as the model “reasons” its way to an answer, reportedly checking its work before delivering an answer. 

In response to a prompt requesting a translation of a given text, ChatGPT o1 might inform users that it is “thinking about translation,” “grasping the narrative,” “crafting an accurate translation,” or “choosing the right tone.” Such messages might even reference the source text itself, or try to add a “human touch” by, for example, “responding with empathy.”

Compared to its predecessor, ChatGPT 4, and its contemporary, o1 mini, described as the “smaller, cheaper version optimized for faster responses,” ChatGPT o1’s translations are better overall. They are crisper, read more naturally in the target language, and take more liberties in conveying idioms across languages. 

All three models, however, struggled with displaying right-to-left script, particularly with regard to punctuation.

ChatGPT 4 did score one interesting win by translating a slang acronym into multiple languages, and elaborating, when prompted, on the meaning of the translation and why it was selected. ChatGPT o1 and o1mini, meanwhile, simply copy-pasted the letters, not even transliterating them for non-Latin writing systems.

Ultimately, how well ChatGPT o1 can translate is a subjective matter, and depends on a number of factors (as the model itself can explain), including whether the source and target languages are high- or low-resource, the domain expertise required, and the context in which the translation will be used.