Sorting Algorithms Don't Hallucinate

In October 2025 Anthropic put out a paper on whether language models can introspect.¹ It’s careful work with a careful set of caveats, and within about a day of it landing I started seeing the caveats thrown overboard. The remark I keep running into goes something like this:

Even something as simple as sorting algorithms have emergent properties. Consciousness will definitely emerge from something like LLMs.

At feed-scrolling speed it lands fine. Small systems already do surprising things, an LLM is a very large system, so somewhere up the complexity ladder the surprising thing is a mind waking up. The whole load is carried by one word, “emergent”, and that word is quietly doing two jobs at once.

I don’t want to swat this on the vocabulary, because that’s been done and it stops one step short of the good part. I want to show the argument doesn’t work even if we hand every word its most generous meaning.

Two jobs for one word

Borrow a distinction from David Chalmers.² A thing is weakly emergent when it’s a surprise given the low-level rules but still follows from them, if you were patient enough to trace it through. It’s strongly emergent when it doesn’t follow from the low-level facts at all, not even in principle, no matter how long you sit with them.

Weak emergence is common and it’s dull once you’ve seen it. A sorting algorithm is about the cleanest specimen going. Local swaps, local comparisons, and out the end comes a global order no single comparison ever held; yet nothing in that order arrived from outside the swaps. Hand me the input and the rules and I can tell you the output in advance. Flocking, traffic waves, the clearing price in a market, they all cash out the same way.³ Surprising, sure; spooky, no; the bill is paid in full by the parts.

Strong emergence is the other thing, and the reason people say the word in a hushed voice is that the candidate list is basically empty. Chalmers reckons there’s one entry anyone still argues about, and it’s consciousness: the business of there being something it’s like to be the system, which he says you can’t reach from the mechanism no matter how much of the mechanism you hold.

Pick one

So take the remark and make it choose.

If “consciousness will emerge” means the weak kind, then consciousness is some ordinary property of the network, fixed by the weights and the arithmetic, sitting there waiting to be traced out. Maybe it is. But then the sorting algorithm has bought you nothing, because knowing one simple system has a traceable high-level property tells you zero about which property some other system has. You’d still have to show the particular thing the model computes is the thing that counts as conscious, and that’s the whole argument, not a step in it. “Definitely” is left holding nothing. You’ve assumed there was never anything hard here, then taken a bow for explaining it.

If it means the strong kind, which is the version worth getting excited about, the one where a light comes on inside the model, then the premise is just false. The sorting algorithm isn’t strong emergence. So the argument is now “simple systems show the weak kind, therefore big systems show the strong kind”, and those are two different animals with no road between them. You don’t get strong emergence by piling up weak emergence, because strong emergence isn’t a dial that complexity turns. It’s the one spot Chalmers says the trace runs out, and pointing at a bigger heap of parts points the wrong way.

Either the claim is about a property nobody has actually found in the model, or it only works by swapping the meaning of its main word somewhere in the middle. There’s no third door where the sorting algorithm and the conscious model are the same kind of emergent and the argument still walks through.

We’ve had this word before

None of that stops the remark from feeling like an argument, and it’s worth asking why it does. Eliezer Yudkowsky had this word’s number back in 2007: “emergent” is a semantic stopsign.⁴ Say a system’s cleverness is emergent, or that its consciousness will be, and it feels like you’ve been handed a fact. Ask what the word buys you, though, in predictions or mechanisms or anything you could go and check, and the answer comes back empty. He files it with phlogiston and the old life-force, words that stood in front of combustion and biology for a while without explaining either. That emptiness is the whole trick here. A word that doesn’t settle on one meaning is free to mean the weak kind in the premise and the strong kind in the conclusion, and nobody feels the switch.

The paper says the opposite

There’s a nice irony in where the remark starts. It jumps off the introspection paper, so read what the paper actually says. They inject a known concept into the model’s activations and check whether it notices and reports the intrusion.¹ Sometimes it does. On the best model they tested it manages this about one time in five, and they call the ability unreliable and narrow, missing more often than not.

And they draw the line the remark skips over. What they measure is access to internal state, the sort of thing a system can pull up and report on and act with, which is what Ned Block named access consciousness.⁵ What the remark is worked up about is the other kind, phenomenal consciousness, whether there’s anyone home to have the experience; and the authors say straight out that their work doesn’t touch it and doesn’t tell you whether the model is conscious.¹ That’s the same crack Chalmers puts between the easy problems, about what the system does, and the hard problem, about why any of it is felt.⁶ The paper sits on the doing side and says so. The remark takes a hedged one-in-five result about doing and files it as a done deal about feeling.

Ask what it is

If you actually want to bet on machines being conscious, there’s an honest way to do it, and it goes nowhere near emergence. J.J.C. Smart made the move in 1959: a sensation isn’t correlated with a brain process, and it doesn’t float above one, it is one, nothing spare.⁷ You can’t correlate a thing with itself. Carry that over to machines and the question stops being “will consciousness emerge once it’s complicated enough” and turns into “what physical or functional setup is consciousness, and is the model running it”. Hard question, nobody’s answered it, but at least it says what would have to be true. “It’ll emerge” says nothing; it points at the mechanism and promises to name it later, then bills the delay as a forecast.

It also kills the complexity story on the spot. If being conscious is running a particular setup, you get there by running that setup, not by heaping up whatever’s handy until something gives. A bigger heap of the wrong wiring is a bigger heap of the wrong wiring.

The sort that never wakes up

Which puts the sorting algorithm back on the table, since it was the best card the remark held and it plays against them. Sorting is weak emergence stripped to the bone, a global order built from nothing but local swaps. If a simple system throwing off weak emergence were the first spark of a mind, a sort is exactly where you’d expect to catch the flicker of one. What you catch instead is a sorted list: correct, finished, nobody home. Make it bigger and you have more sorting, not a different kind of thing, and a very large sort is still a sort. If a machine is ever conscious it’ll be because someone built the thing consciousness turns out to be, not because the arithmetic piled up high enough for somebody to move in.

J. Lindsey, “Emergent Introspective Awareness in Large Language Models,” Anthropic, Transformer Circuits, Oct. 29, 2025; companion post “Signs of introspection in large language models,” anthropic.com/research/introspection. The one-in-five figure and the “highly unreliable and limited in scope” line describe Claude Opus 4.1. The authors write that the results “don’t tell us whether Claude (or any other AI system) might be conscious” and “do not directly speak to the question of phenomenal consciousness”. The method is the clever part: injecting a concept vector and asking whether the model notices separates real introspection from a model confabulating a plausible answer after the fact, which is why you can’t settle this by just asking it. ↩ ↩² ↩³
D. J. Chalmers, “Strong and Weak Emergence,” in The Re-Emergence of Emergence, P. Clayton and P. Davies, Eds. Oxford, U.K.: Oxford Univ. Press, 2006. Weak emergence is deducible in principle from the low-level domain; strong emergence is not deducible even in principle. Chalmers is careful that this is a claim about what follows from what, not about how surprised the low-level rules leave us, which is the slip the popular version trades on. ↩
The grown-up version of the word traces to P. W. Anderson, “More Is Different,” Science, vol. 177, no. 4047, pp. 393-396, 1972. Anderson’s point is that each level of organisation can need its own laws, not just the lower-level laws rewritten, and none of it calls for anything you couldn’t in principle trace back down. Weak emergence in Chalmers’ sense, and the respectable ancestor the meme borrows its authority from. ↩
E. Yudkowsky, “The Futility of Emergence,” LessWrong, Aug. 2007. “What do you know, after you have said that intelligence is ‘emergent’? … You can make no new predictions.” He groups the term with phlogiston and vitalism as ways of parcelling a mystery into a word that feels like an answer. My point is narrower than his: it’s the emptiness of the word that lets the weak and strong senses change places without anyone catching the swap. ↩
N. Block, “On a Confusion about a Function of Consciousness,” Behavioral and Brain Sciences, vol. 18, no. 2, pp. 227-247, 1995. Access consciousness is content a system can use for reasoning, report, and steering behaviour; phenomenal consciousness is there being something it’s like to have the content. Block’s whole point is that the two come apart, which is why a result about the first isn’t evidence about the second. ↩
D. J. Chalmers, “Facing Up to the Problem of Consciousness,” Journal of Consciousness Studies, vol. 2, no. 3, pp. 200-219, 1995. The easy problems are about functions and mechanisms and yield to ordinary science; the hard problem is why any of that function is felt from the inside. The sharpest form is Frank Jackson’s Mary, in “Epiphenomenal Qualia,” Philosophical Quarterly, vol. 32, pp. 127-136, 1982: she learns every physical fact about colour from inside a black-and-white room, then learns something new the first time she sees red. ↩
J. J. C. Smart, “Sensations and Brain Processes,” The Philosophical Review, vol. 68, no. 2, pp. 141-156, 1959. Sensations are identical to brain processes rather than riding alongside them, with nothing “over and above” the physical. The identity move is what dissolves the emergence question: there’s no gap for a high-level thing to arise across when the high-level thing just is the low-level one under a different description. ↩