Hold a braid at arm's length and it looks like a knot: strands crossing over and under, no visible order. Pick it up and follow one strand with your finger and the knot disappears. Every strand goes somewhere, and it crosses the others in the same pattern over and over. Nothing changed except how closely you looked.
For about a decade, the people who study the inside of AI models believed they were holding a real knot. A model is built from millions or billions of tiny units called neurons (single numbers inside the network that rise or fall with each input). The hope was that each neuron would mean one thing, like "curve" or "dog face". Many seemed to mean several unrelated things at once, so the field concluded that models pack ideas together in ways that have to be cut apart with special tools before anyone can read them.
This week Nick Cammarata, one of the researchers who built that picture, published a long essay saying it was mostly a braid. It is called "Neural Networks Are Already Clean" and it is published by Goodfire, the interpretability company where he works: https://already-clean.goodfire.com/. He announced it on X here: https://x.com/nickcammarata/status/2110757819427574056. This post explains what he claims, what his evidence is, where he says he could be wrong, and why it matters to anyone who lets an AI read their private writing. Everything about the science below comes from his essay and post. Where we give our own view, we say so.
Why did anyone think AI models were a tangle?
Interpretability is the science of reading what happens inside a trained model: which internal parts react to what, and how they combine to produce an answer. It matters because today we mostly judge AI by its outputs. If a model gives a good answer for a bad reason, outputs alone won't show it.
Early on, researchers looked at what made each neuron fire and found a puzzle. Some neurons seemed to respond to unrelated things. The famous example, from a 2017 image model called InceptionV1, was a neuron that seemed to fire for both cat faces and car fronts. A neuron like that is called polysemantic (having many meanings).
The leading explanation was superposition: the idea that a network stores more concepts than it has neurons by overlapping unrelated ones that rarely appear together, the way you might store winter coats and beach towels in the same box because you never need both at once. If that is how models work, reading neurons directly is hopeless. You first have to untangle them.
So the field built untangling tools. The main one is the sparse autoencoder (a second, smaller network trained to rewrite a layer's activity as a longer list of "features", each hopefully meaning one thing). A lot of the last few years of interpretability research runs through tools like it.
Cammarata helped write the original papers. In his words: "So there were two explanations for polysemanticity, and I helped propose both." His post on X describes how he came away "pessimistic, because neural networks appeared to fold into arbitrary tangled knots." When he later started OpenAI's first large automated interpretability project, on a model called GPT-2 XL, the essay reports that "fewer than 2% of its neurons came out clean."
What does he say changed?
Better tools, and a much closer look. The post on X puts it simply: "with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful."
The essay goes back to the famous examples one by one. The cat-and-car neuron, he now argues, is a neuron for a V shape: pointed things push it up, round things push it down. Cat ears and car grilles both have points. It was one idea all along, described badly.
He also traces how wrong the old estimates were. For the last layer of InceptionV1, he says he once would have put the share of readable neurons at around 5%. With better visualization tools on the same network he now puts it at about 50%. Retrained with modern methods, about 70%. In a modern network of the same size, a ConvNeXt, "it's about 99%."
The thesis in one line, from the essay: "Neural networks are already clean."
If neurons aren't tangled, why do some look messy?
This is the central idea of the essay, and it is where the braid comes in.
Cammarata names a pattern he calls recursive generalization. A neuron starts out responding to one core thing. As training goes on, it extends to things that are nearby and similar, and then to things near those. Each step is small and sensible. After many steps, the ends of the branches can look unrelated to each other, even though you can walk from one to the other in small, reasonable moves.
His examples are concrete. One language-model neuron responds to the sound "iy" and reaches as far as "do it yourself". Another responds to planetary geology and has offshoots into JavaScript code. From a distance, those look like random combinations. Followed strand by strand, they are a braid.
The difference from superposition is testable, which is his main point. Generalization should be local: things end up together because they share a pattern. Superposition would be non-local: unrelated things packed together because they rarely collide. When he looked closely, the groupings kept turning out local. "I now think I was making superposition the null hypothesis when it should have been the last hypothesis," he writes. His summary of the old habit: "Polysemanticity is often more a claim about the observer than the neuron."
Did the untangling tools find anything the neurons missed?
He tested this directly, because it is the obvious objection: if neurons hide things, a sparse autoencoder should find building blocks the neurons lack.
In his image-model experiments, he reports that it didn't. On the last layer of one model, he found that 88% of the autoencoder's activity was near-copies of single neurons. In another test he squeezed a layer of InceptionV1 from 480 neurons down to 10, forcing the network to cram everything into very little space. The 10 neurons stayed readable, just coarser. An autoencoder trained on that squeezed layer, swapped back into the model, dropped its top-choice accuracy from 58.8% to 43.0%.
He doesn't claim these tools are useless. He says they help most in very small models, and that they are good at grouping neurons that tend to fire together. His claim is narrower: the neurons themselves are the building blocks, and the network can be read using its own parts.
Does this hold for the language models people actually use?
Image models are easier to check, so the essay starts there. The language-model section uses open models, mainly Qwen3-32B and Kimi K3, a model with more than a trillion parameters (the adjustable numbers inside a network). "We found similar results in models up to trillions of parameters, like Kimi K3," he writes.
The method is an attribution graph (a map of which internal neurons pushed the model toward a particular answer), built only from the model's real neurons. Then he tests the map by removing neurons and watching the answer change.
One example shows how readable this can be. Asked a question whose answer was Santa, Qwen was near-certain. Removing eight neurons whose negative side relates to Santa dropped that to 6%. Removing the twenty neurons actually labelled "Christmas" or "Santa" only dropped it to 76%. The meaning lived where a careful reader found it, not where a keyword search would look.
In Kimi K3, asked what to do with a found wallet, the model answers "return". Removing one neuron that the essay describes as a "holds back" neuron raised the probability of that answer from 0.83 to 0.98. He describes this as possibly the first published neuron-only map of a model that size.
He is clear about limits here. The Kimi map is a hand-picked slice. "We haven't begun full audits of this scale yet." The companion technical paper is still forthcoming.
What would it take to read a whole model?
The essay's own analogy is a city, not a knot. "A bigger system isn't necessarily harder to navigate," he writes, as long as you have good maps. He argues for maps you can zoom in and out of that stay connected to the real streets, the actual neurons and circuits. He warns that forcing your own categories on the network too early can make it look far worse than it is: "you may have made Tokyo look much messier than it is."
His proposed route is scale. He imagines a laboratory of automated scientists, AI agents that label neurons, test explanations and trace circuits, doing ordinary science far faster than people can. He connects this to Rich Sutton's "bitter lesson", the observation that simple methods that scale beat clever hand-built ones: "Maybe there's a bitter lesson for interpretability too: scaling ordinary investigation beats engineering a clever new decomposition."
He calls the essay a working hypothesis, and he gives a number for his confidence: "We're maybe 85% sure", by which he means 85% sure that nearly everything comes from generalization rather than packing.
What does this mean for trusting an AI with a journal?
This is our reading, not his. The essay doesn't mention journaling.
We build Argo, a private AI journal. You write or speak, and an AI reads your entries and shows you patterns you can't see from inside your own head. When people ask whether they can trust that, our answer today is about plumbing: entries are encrypted, speech and handwriting are turned into text on your phone, and the AI insights run through private, encrypted, anonymized processing.
That answer protects your words on the way in. It says nothing about what the model does with them once it reads them. Why did it call this week anxious? Did that come from what you wrote, or from an assumption about people like you? Until now the honest answer was that nobody can look. If Cammarata is right, that starts to change. A map like his Santa example, applied to one journal insight, could in principle show which parts of the model drove it.
We want to be precise about what that means. Nobody, including us, can do this for a journal app today. His work is on open models, and the full audits haven't happened. But trust in AI has so far rested on locking the box. This is the first credible argument we've seen that we might also be able to look inside it.
What does it do to the "AI takeover" story?
Most takeover stories share one assumption: AI systems get more capable while staying opaque, so a system could pursue goals we can't see until it's too late. Research we follow on automated AI research, the loop where AI speeds up its own development, makes that worry sharper, because the faster capability moves, the less time people have to understand it.
Cammarata's essay pushes on the opaque part. His key line on deception: "Deception might hide from someone reading a model's outputs, but it has no reason to hide from the model's own circuitry." And his fear is not that it is hidden: "My fear is that deception will be right there, in plain sight, and we won't know how to read it."
If he is right, the same agents that speed up AI development can speed up reading it, and fixes could become, in his words, "simple, careful changes" instead of retraining a system you don't understand. In his post he goes further: "I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon."
Our view: this changes the takeover story from a race we can only lose into a race with two runners. That is our framing, and it rests on a hypothesis that hasn't been tested on a whole frontier model. Two caveats matter. Goodfire sells interpretability, so it benefits if interpretability looks tractable, which is a reason to wait for other labs to replicate the results. And readable parts don't automatically add up to a readable whole.
What to take from it
From a distance, the inside of an AI model looked like a knot. One of the people who drew that picture now says that, looked at closely enough, it is mostly a braid, and that AI agents may soon be able to follow every strand. If that holds, "what is the AI doing with my words?" becomes a question someone can answer.
Read the essay at https://already-clean.goodfire.com/ and Nick Cammarata's post at https://x.com/nickcammarata/status/2110757819427574056. Follow what we build at https://myargoquest.com/blog and https://x.com/myargoquest.