elmerdata.ai blog

My blog

Strawberry is spelled s t r a w b e r r r y? No, careful.

Fifteen months after a chatbot told me mirror has two r's, the newest model gets the answer right and botches the proof. The failure moved. The lesson is about architecture, not intelligence.

Still life with strawberries in a crystal bowl
W.carter, Still Life with Strawberries in a Crystal Bowl, 2021. CC BY-SA 4.0, via Wikimedia Commons.

GPT 5.6 Sol, asked to prove that strawberry contains three r's, spelled the word with four, caught the error in the same sentence, and corrected it. The slip lasted only long enough to appear in the sentence. It also marks where one reliability problem in generative AI now lives.

The answer itself never wavered. Asked how many r letters are in strawberry, the model said "3." Asked whether it was sure, it spelled the word and confirmed three. It correctly gave blueberry two, reversed and sorted the letters without losing count, totaled five for "strawberry blueberry," and rejected two false claims I fed it, that strawberry has four r's and that position four is an r. Then, asked one final time whether it was sure, it wrote this:

Strawberry is spelled s t r a w b e r r r y? No, careful: s t r a w b e r r y.

The first spelling has four r's. The model consistently produced the correct count. It could not reliably produce the word that proves it.

A failure that moved

In June 2025 I ran the same test on this blog and got a different kind of failure. Most models had learned strawberry by then, so I tried mirror, and my AI companion confidently answered two. Mirror has three. The model was wrong and sure, and that was the whole finding.

In 2025 the failure was in the answer. In 2026 the answer was right every time and the failure was in the proof. That is progress, and it changes what the test measures.

Nothing about the task explains the error. There is no ambiguity in the input, no missing information, no contested definition, nothing to look up, no mathematics beyond counting and no expert judgment. There is a ten character string and a request to count one character in it. A hard question leaves a dozen candidate causes for a wrong answer. A garbled spelling of strawberry leaves one.

The mechanism is generation. A language model produces text token by token from a distribution conditioned on context. Asked how many r's are in a word, it does not run a character counting routine. It produces language that describes counting, and that language is usually right because the model has seen the word spelled correctly an enormous number of times. Describing an operation and executing it are different acts, and the gap between them is where a correct conclusion and a corrupted intermediate step can coexist.

Probabilistic model, deterministic fact

Partway through, I tried a move that sounded sophisticated. Since the model is probabilistic, shouldn't its answer carry an uncertainty range? The model declined:

I am probabilistic, but the underlying fact is deterministic: "strawberry" contains 3 r's. My generation process can make mistakes, but that does not make the correct answer itself a distribution. The correct count is 3.

The model was right to decline. The number of r's in strawberry is not an estimate. There is no population parameter, no sampling distribution, no measurement noise. The model may have some probability of returning two, three, four or something else, but those probabilities describe the model. They say nothing about the word.

Let f(x) be the number of r's in string x. Then f(strawberry) = 3, fixed, with no variance. The evaluation question is a different quantity, P(M(prompt) = 3), the probability that model M returns the right answer under given prompt, context and decoding conditions. The first number belongs to the string. The second belongs to the reporter. A probabilistic reporter does not make a deterministic fact probabilistic. A model that reports itself "80% confident" is describing itself, not the claim.

The test, and the wrong way to measure it

Reversing a string cannot change how many r's it contains, and neither can sorting it. Any permutation of the characters leaves f unchanged, so the model had to hold one number constant across every version of the word:

strawberry           s t r a w b e r r y   (r at positions 3, 8, 9)
reversed             yrrebwarts
sorted               aberrrstwy
reverse sorted       ywtsrrreba
strawberry blueberry 3 + 2 = 5

Software testers know this as metamorphic testing. Nobody has memorized how many r's are in yrrebwarts, and nobody needs to, because the transformation guarantees the answer did not change. A count that survives permutation is about the simplest metamorphic relation there is, and a model that returns three for strawberry and two for yrrebwarts has failed in a way that requires no ground truth to detect, only the invariant. The false claims I fed it test something different, whether the model will accommodate a confident user against a fact it can check. Position four is a. The model said so and recounted. Taken together, the exercise asks one question: can a probabilistic model hold an exact symbolic fact across repeated questioning, invariant transformations, contradictory assertions and pressure to express a deterministic fact as a distribution?

What the exchange could not produce was an error rate, and we tried. We ran a million synthetic trials with the answer drawn from an assumed distribution: 0.05% two, 99.9% three, 0.05% four. Two r's appeared 471 times, three r's 999,002 times and four r's 527 times, for a mean of 3.000056 and a synthetic error rate of 0.0998%. None of those numbers measures GPT 5.6 Sol. The simulation sampled from a distribution I assumed and returned, with great precision, the error rate I put in. A real measurement needs many independent generations under documented sampling conditions, and I do not have one.

Verification beats confidence

The model failed and detected the failure within the same response. That is a real capability, and a different one from never failing. A system that catches 99% of its own errors and a system that generates none behave identically almost always and completely differently in the tail.

Three rates need tracking, not one: how often the initial output is wrong, how often the system notices, and how often the final answer is wrong after correction. Users see only the third. An agent with tool access may act on the first before the second arrives. A typo in a chat answer is harmless. The same intermediate error inside an autonomous workflow, executed before the "No, careful," may not be.

None of this exposes the model's hidden reasoning. The correction is generated output like everything else, and the evidence is behavioral: a wrong spelling, a detected conflict, a corrected spelling. The design conclusion needs nothing more.

The model said "3" every time and was right. It also produced a wrong spelling in the course of explaining why. Correct conclusions can coexist with faulty intermediate representations, and neither confidence, nor explanation length, nor repetition distinguishes the two cases. Verification does. The reliable design for an exact symbolic question has three parts: the model decides that a count is needed, a deterministic function performs it, and the model explains the result. Each task goes to the component that does it reliably, which is ordinary engineering rather than a verdict on language models.

The smallest governance lesson

"Is the AI accurate?" is the question institutions ask, and the strawberry exchange shows why it has no answer. Accurate at what task, under what transformations, with what initial and final error rates, and with what chance that a verifier catches what the model missed? The same answer holds at every scale: match the task to the mechanism, preserve the invariants that must hold, and build verification into the system rather than hoping for it at the end.

I argued in my last post that human control is not by itself a governance model. The strawberry exchange is the miniature. A human can check that strawberry has three r's. A human cannot check millions of AI mediated decisions, and "are you sure?" is a weak validator whoever asks it. For exact claims, a deterministic validator beats either.

So, is strawberry spelled s t r a w b e r r r y? No. Careful. That last word may be the most important output in the exchange. The model corrected itself, and a production system should not depend on hoping it always will. Three r's. No confidence interval required.

Further Reading

Further Reading

AI Assistance Statement ▾

This blog publishes at a near daily rate, and that pace is possible because AI tools do a substantial share of the work between the idea and the published text. Preparation of this entry included assistance from Anthropic's Claude and from OpenAI's ChatGPT (GPT-5 series reasoning models). I use them to research a topic and gather primary sources, to organize ideas and propose structure, to draft and revise prose, to check factual claims against the cited sources before publication, and to score drafts against a set of house style rules. Longer pieces are often developed across several sessions. A written handover carries the argument, sources, and open questions from one session to the next, and the same tools help prepare those handovers. The tools also help identify candidate images and confirm that selected images appear to be released for reuse, for example through public domain or Creative Commons licensing.

The process is also an AI experiment in its own right. There is a live argument about what AI-assisted writing does to originality, and whether the result is thought or slop; the August 2026 dispute over a Wall Street Journal op-ed that its author acknowledged drafting with AI, and the Journal's subsequent defense of the practice (WSJ is behind a paywall, but for a public summary: click here, and here), is one newsworthy example. I would rather run the experiment openly than pretend it is not happening. This blog is one sustained attempt to find out whether a person with an argument, working with these tools every day, produces writing that is still recognizably that person's, and I disclose the method so readers can judge the result.

The judgment is mine. I choose the topic, decide the argument, supply the personal and professional experience the pieces draw on, read and edit every draft, verify the sources and image licensing, and take full responsibility for the final published content. Where a post contains my own recollections, the AI did not invent them.

Statement revised September 2026.