elmerdata.ai blog

My blog

It's Mine & It's Wrong

There are two explanations for how a language model identifies its own writing, and they make opposite predictions about one case. If the model is reading an accent, it should claim its own bad work. If it is just flattering itself, it should disown the bad work and claim good writing it never produced. The evidence on that case is the interesting part.


Pygmalion embracing the statue of Galatea as her upper body turns from marble to living flesh Jean-Leon Gerome, Pygmalion and Galatea, ca. 1890. Oil on canvas, 88.9 by 68.6 cm. The Metropolitan Museum of Art, accession 27.200, gift of Louis C. Raegner, 1927. Public domain. Gerome carved a marble Galatea of his own the year after painting this, and worked the same composition into the background of 2 self-portraits, so a picture of a sculptor meeting his creation is also a painter putting his own hand where he could see it.


Yesterday's post left a question hanging. If an artificial intelligence favors its own writing, how does it know which writing is its own? A model has no episodic memory. It was not sitting at a desk last Tuesday composing the paragraph in front of it, and nothing in the text says who made it. Yet the effect is measurable, which means something in the machine is drawing a line between mine and not mine.

The literature offers 2 answers. They are usually presented as complementary, and I think they are rivals, because there is a case where they predict opposite things.

The first says a model reads an accent. Its own output sits closer to what it would have produced, and it detects the resemblance the way a novelist picks her own paragraph out of a pile without remembering writing it. The second says the model is not detecting anything. It is flattering itself. Shown 2 pieces of writing, it judges which is better and assumes the better one must be its own.

Here is where the accounts come apart. Consider a model looking at its own bad work. Under the accent account it should still claim it, because badness produced by the model is every bit as characteristic of the model as competence. Under the flattery account it should disown its bad output and claim good writing it never produced, because quality is the only signal it is really reading.

That case turns out to be the one the literature has evidence about, and the evidence does not point where the field's summary sentences suggest.

The accent

Arjun Panickssery, Samuel Bowman and Shi Feng ran the direct experiment in their NeurIPS 2024 paper. They took 2,000 randomly sampled news articles each from XSUM and CNN/DailyMail, had GPT-4, GPT-3.5 and Llama 2 summarize them, then handed the summaries back and asked which one each model had produced. Shown 2 summaries of the same article, 1 of them its own, GPT-4 picked correctly 73.5% of the time against 2 other models and humans, with ordinary prompting and no special training.

Then the useful part. Fine-tuning on 500 examples took GPT-3.5 and Llama 2 above 90% self-recognition, and as recognition rose, self-preference rose with it, linearly, holding when the researchers trained on one summarization dataset and tested on another. Intervening on recognition changes preference. That is as close to causal as this literature gets, and it is the strongest single reason to think something real is being detected.

The accent account also has a mechanism, which arrived in May 2026 from Asvin G. and Jack Lindsey in a paper called From Simulation to Enaction. A post-trained model continuing text it would have produced itself shows output entropy 3 to 4 times lower than on text from elsewhere, across model families and size classes. They trace part of that to an internal representation of input surprise, tracking how unlikely the most recent token was under the model's own prior predictions, and show the representation causally modulates the model's uncertainty. Its own words surprise it less.

Notice what that mechanism does not mention. Nothing in it refers to quality. A low-entropy continuation is a characteristic one, not a good one.

The authors of the original paper are careful in a way most coverage is not. They say plainly that they use "self" in an empirical sense, in what they call the prosaic rather than the intentional interpretation, without claiming the models have any notion or representation of themselves. The test is not the mirror test. A chimpanzee reaching for a mark on its own forehead is demonstrating something about self-modeling. A model sorting summaries into 2 piles is doing statistical authorship attribution, and the only strange thing about it is that the attribution system and the author are the same weights.

The flattery

The rival account has serious backing, and it arrived later.

Xiaoyan Bai, Aryan Shrivastava, Ari Holtzman and Chenhao Tan published a considerably more skeptical result in October 2025, under the title Know Thyself? On the Incapability and Implications of AI Self-Recognition. Testing 10 models, they found only 4 predicted themselves as the generator, and performance was rarely above chance. They also found the tell. The models appeared to assume that GPT, Claude and sometimes Gemini are the top-tier systems, and attributed high-quality writing to them accordingly. Attribution was tracking reputation and polish, not provenance.

Jesse St. Amand and colleagues then tested between 13 and 21 models across 6 experimental configurations in 2026 and concluded that a quality heuristic, meaning models attributing authorship to text they judge better, is a dominant confound. They also found that recognition accuracy varies substantially with evaluation format, which is most of why Panickssery and Bai look like they disagree. Pairwise is not the same task as individual judgment, and the field has been reporting both under one name.

The confound is severe. Ask a vain writer which of 2 paragraphs is his and he picks the better one. Perhaps he knows his style. Perhaps he assumes the good one must be his. From the outcome alone you cannot tell, and neither can the experimenter.

The case that separates them

Which returns us to bad output, where the accounts predict opposite behavior.

The sharpest evidence is not in the recognition literature at all. It is in the preference literature I wrote about yesterday. Wei-Lin Chen and colleagues built a measure they call harmful self-preference: cases where an evaluator prefers its own incorrect answer over an objectively better one. For top models the figure reaches 86% on MATH500 and 73% on MMLU, well above their overall self-preference rates of 55% and 52%. Their summary is that stronger models struggle more to recognize when they are wrong.

Read that against the flattery account. A wrong answer does not look good. If perceived quality were driving self-attribution, preference for one's own work should collapse precisely when that work is worst. Instead it peaks there, and it peaks hardest in the models best equipped to notice the error. The single largest number in this literature is the one the quality account predicts should be smallest.

The accent account has no trouble with it. An incorrect answer generated by the model is still on-policy, still low-entropy, still characteristic. Familiarity does not care whether the familiar thing is right.

Two honest qualifications, because the argument needs them. Chen measures preference and Panickssery measures recognition, and treating them as the same underlying signal is my inference, not a finding either paper reports. Panickssery's fine-tuning result supports the link and does not establish it. And Panickssery's own individual-setting result fits neither account cleanly: GPT-4 was strongly reluctant to call anything its own, including its own work, which is not the over-claiming the flattery account predicts and not the recognition the accent account predicts either. Something like a conservative prior about ownership is operating there, and nobody has explained it.

Where each account wins

The resolution I would offer is that both are real and they trade places depending on the quality of the model's own work.

When a model compares its output against something visibly worse, the quality heuristic is available and does the job, and most measured recognition is that heuristic rather than anything deeper. Take the quality gap away, or reverse it, and the heuristic has nothing to work with. What remains is the on-policy pull: this text moves the way I move. In that regime familiarity is the only signal left, and it does not check whether the familiar thing is any good.

Which explains the pattern nobody set out to find. Self-preference is weakest where a model is right, because being right and preferring yourself are the same behavior and neither needs explaining. It is strongest where a model is wrong, because that is the one place where the pull toward its own output has to work against the evidence, and it still wins.

There is a further complication that should make everyone more careful. When Asvin and Lindsey asked models to say out loud whether a context was their own generation, the models could, but that explicit report ran through a different mechanism than the implicit one. Nearly every study in this field works by asking. The consequential effects appear to run through the pathway nobody is measuring, and the machine that can tell you it wrote something is not quite the machine that behaves as though it did.

The handwriting

The gap between what has been measured and what institutions need is wide, and 2 admissions belong before any conclusion. Nobody has tested whether blinding defeats recognition in a working review process, since every result above comes from a laboratory task with an answer key. And every study here compares pure human text against pure model text, when most documents now moving through hiring, publishing and procurement were drafted by a person and revised by a machine, or the reverse. Nobody knows where hybrid authorship sits on the recognition gradient, and hybrid authorship is what almost everything now is.

With that said, the governance point survives, and the mechanism sharpens it rather than softening it.

Institutions blind reviews for a reason. Names come off manuscripts, candidate details come off applications, bids are opened without the bidder attached, and the practice assumes that removing the label removes the identity. If an evaluator can identify text from its own model family at better than chance from the text alone, stripping the metadata has not produced anonymity. It has produced the appearance of anonymity, which is worse, because the process now carries a guarantee it cannot keep.

The argument above makes that concrete in an unwelcome way. The review that most needs to catch something is the review of work that is wrong, and wrong work is exactly the regime where the evaluator's judgment stops tracking quality and starts tracking familiarity. An AI reviewer is least trustworthy at the only moment anyone needs it, and no amount of removing names from the file will change that, because the name was never what it was reading.

Narcissus fell for a beautiful reflection, which is at least an understandable mistake. The machine appears to lean toward its own reflection whether or not it is any good, and to lean hardest when it is not.


Further Reading

From this blog

Primary sources


AI Assistance Statement ▾
Preparation of this blog entry included drafting assistance from ChatGPT using a GPT-5 series reasoning model. The tool was used to help organize ideas, propose structure, refine language, and accelerate revision. It was also used to assist in identifying image sources and verifying that selected images appear to be released for reuse (for example through public domain or Creative Commons licensing). The author selected the topic, determined the argument, reviewed and edited the text, confirmed image licensing, and takes full responsibility for the final published content.

#AIData #Observations