Analysis

Gemini 4 Carbon Is #1 Because the Index Pays for Saying "I Don't Know"

Gemini 4 Carbon tops the AI index by about a point, and a 5% term that pays for saying "I don't know" decides it. So pick by what a mistake costs you.

MesmerTools•November 9, 2026•3 min

On October 28, Google announced Gemini 4 Carbon, and it landed at #1 on the Intelligence Index from Artificial Analysis (AA), one score built from nine tests. It beat Claude Opus 5.5 by about a point. Almost nobody can use it: Google gave it to cyber defenders and vetted partners first, and as of November 2026 the API has no date.

So I rebuilt the index from AA's published weights to see where that point came from. A twentieth of the score rewards a model for rarely making things up, and without that part Opus leads. Carbon is #1 because the index pays for saying "I don't know". I'd ignore the rank, work out what a wrong answer costs you, and pick from what you can call today.

A semicircular gauge labeled cost of a wrong answer, split into zones for Fable 5.1, Gemini 4 Carbon high and Gemini 4 Carbon medium, with the two Carbon zones hatched because they have no public API, an amber needle at 1 marked AA's knowledge score, and the headline #1 depends on the penalty

What Carbon is best at is saying "I don't know"

The number behind all this is the hallucination rate, from AA's knowledge test. Carbon's high setting, the one AA headlines, sits at 11%, against 50% for Opus 5.5.

But that rate is wrong answers divided by every question the model didn't get right. So it tells you how often a model makes something up when it doesn't know, and nothing about how much it knows.

Scaled from AA's results to 200 questions: Opus 5.5 gets 14 more right and 26 more wrong, and right minus wrong peaks at Carbon, between the cautious and the confident.

Gemini 4 Carbon's lead is smaller than what that habit earns it

That habit sits inside the headline number. AA's published weights count this test twice: accuracy at 10% of the index, and one minus the hallucination rate at 5%. The rate alone hands Carbon nearly 2 points against Opus, and its whole lead is 0.9. On the balance below, Opus is ahead right up until that term goes on.

Each block is one test's real gap in index points, on my arithmetic from AA's published weights, and the blank column at the end is a model that answers nothing.

A model that refuses every question collects the full 5 points. Valuing "I don't know" is defensible, so I wouldn't call the weighting wrong. But AA chose a penalty, and at a twentieth of the index it picked the winner.

So which model wins depends on what a wrong answer costs you

AA scores the knowledge test itself as right minus wrong, so one mistake cancels one right answer. Change that cost and the crown moves, sometimes to Anthropic's pricier Fable 5.1.

Fable 5.1 wears the crown while mistakes are cheap, Carbon from about 0.6 and Carbon's medium setting from about 3.6, until the two locked Carbons drop out and it hops to what you can call.

Your number is cleanup time divided by lookup time. When a model says "I don't know", you look the fact up yourself. A right answer saves you that lookup, and a wrong one leaves you the lookup plus the cleanup.

A date in a draft might take five minutes to look up and a minute to fix, so 0.2. A figure in a client email takes five minutes to check and an hour to correct, so 12.

Once a mistake costs about four right answers, Opus 5.5 scores below a model that answers nothing at all. For a dosage or a filed citation, check every answer, whichever model you use.

What I'd call today

You can't call Carbon yet, so for facts I'd use Opus 5.5 at max effort if a wrong answer costs about one right one or less. Above that, I'd use Gemini 4 Argon, the Gemini 4 model you can call. I'd skip Fable: about 3 more right per 100 costs two and a half times the price.

What I still don't know is when Carbon ships. The day it gets a public API, I'd read that dial again.

More from the Blog