Gemini vs. ChatGPT for Translation: What Language Teams Should Test

Which is better for translation, Gemini or ChatGPT? In November 2026 the accurate answer is that it depends on the language pair, the content and the brief, and that any answer expires with the next release. Google and OpenAI both shipped a new model generation this autumn.

This article covers what is current, what the public evaluations show, which errors survive every release and how to compare the two on your own content.

The Models as of November 2026

OpenAI. The current generation is GPT-6, the family behind ChatGPT. GPT-6 Astra is the flagship for complex reasoning. GPT-6.1 Sol and GPT-6.1 Terra are the middle tiers and GPT-6 Luna is the low-cost model. OpenAI's API documentation lists a context window of about one million tokens for each of them.

Google. Gemini 4 Argon was released on 13 October 2026 with a one-million-token context window. Gemini 4 Carbon followed on 10 November for the hardest tasks, and Google describes Argon as the right choice for most knowledge work. Gemini 4 Hydrogen is the tier built for speed and efficiency.

Two things follow for translation work:

• Context is no longer the limit. Each of these models can take a full document, the glossary and the style guide in one request. Whether it then follows them is a separate question, and the one worth testing.
• You compare tiers, not brands. "Gemini vs. ChatGPT" in practice means one tier of each at a given cost. Smaller tiers trade quality for speed and price, so test the tier you will actually run.

What the Public Evaluations Show

The most careful public comparison is the general translation task of WMT, the annual Conference on Machine Translation, where professional evaluators score the output.

WMT25 compared 60 systems, including 24 large language models that the organizers added for comparison. In the human evaluation Gemini 2.5 Pro was the best-performing system and sat in the top group for nearly all of the language pairs that people evaluated. The human reference translation was in the top group for fewer than half of them. The organizers called their report "Time to Stop Evaluating on Easy Test Sets".

WMT26, published in October 2026, made human evaluation the main measure for every submitted system. It also added translation instructions for individual texts, so systems are now judged on whether they follow a brief as well as on the translation itself.

These results deserve attention, with three limits in mind:

1. The models are already old. WMT25 evaluated models such as Gemini 2.5 Pro and GPT-4.1. Both have since been replaced, and an evaluation takes months to run and publish.
2. The content is general. The test sets are news, social media and speech. They say little about your product strings, your contracts or your terminology.
3. The language pairs are limited. A result for English into Czech tells you little about English into Kurdish. The reference is also one professional translation made under the task's conditions, not a reviewed and approved translation of your content.

A leaderboard is a reason to test a model. It does not replace the test.

Which Errors Survive Every Release

A new release changes how often these errors happen, not whether they can happen. A qualified reviewer checks for them whichever model produced the text:

• Terminology drift. A term translated one way on page two appears differently on page forty, usually after the surrounding context changes.
• Omissions and additions. A dropped clause or list item, or a qualifying phrase that is not in the source.
• Numbers and names. Amounts, dates, units, product names and people's names that change between source and target.
• Register and address. Turkish output that moves between formal siz and informal sen. Arabic output that reads as Modern Standard Arabic when the brief asked for a Gulf or Egyptian voice, or that addresses the user with the wrong gender.
• Placeholders and tags. In Turkish the suffix after a variable depends on the value that fills it, so a string that is right for one name is wrong for the next.
• Fluent text that is wrong. The hardest case to catch, because nothing in the target looks off until someone reads it against the source.

Our article on hallucinations in large language models covers the last point, and the MTPE guide for Turkish and Arabic shows where post-editing is enough and where it is not.

How to Compare Them on Your Own Content

A fair comparison takes about a week and a few thousand words.

1. Use real content. Take samples from each content type you publish and include the hard parts: interface strings with placeholders, legal text, marketing copy.
2. Fix the brief. Give both models the same glossary, style guide and instructions, and name the target variety, for example Gulf Arabic and not just Arabic. Record the exact model, tier and date.
3. Review blind. A native linguist scores the outputs without knowing which model produced which, by error category and severity on one scale. This is the method of a Turkish LQA or an Arabic LQA.
4. Measure the editing, not only the errors. Time how long a post-editor needs to bring each output to publishable quality. A model with fewer errors that are harder to find can cost more than one with more errors that are obvious.
5. Check the data terms first. Before any client content goes into a model, confirm what the plan you use does with it and what your contracts and NDAs allow.
6. Keep the test set. When the next version ships, run it again. The comparison is only useful while it is current.

Where the Model Matters Less Than It Seems

• Your inputs decide more than the engine. Translation memories, approved glossaries and a clear style guide improve the output of either family. They are also the assets you keep when you change models.
• Build the workflow so the engine can change. A setup that depends on one vendor's model has to be rebuilt at every release. One that treats the engine as a setting does not.
• Post-editing remains the safeguard. ISO 18587 is the standard for post-editing machine translation output. At El Turco the engine is set up per project for the language pair and content and tested on a sample, and native post-editors review the output under ISO 18587. Turkish MTPE and Arabic MTPE describe how that works for each language.

Sources

• Findings of the WMT25 General Machine Translation Shared Task
• Findings of the WMT26 General Machine Translation Shared Task
• OpenAI API documentation: models
• Google: Gemini 4 Argon and Gemini 4 Carbon

First published in October 2024 and rewritten in November 2026. Model names, release dates and evaluation results were checked on 13 November 2026 against the sources above. We do not publish a ranking of the two model families, because the result depends on the language pair, the content and the model version. To compare engines on your own content or to set up post-editing, send us a short brief.