What It's Like to Be a Toaster

What consciousness is, and why a machine could have it

14 July 2026 machine-learning

This post is the first in a multi-part series called What We Owe Machines. The goal of this series is to anchor some of our more fantastic views on machine rights to the existing, well-established thoughts on these sorts of things.

Where this ends up landing is probably not the narrative that fills your feeds, but it’s more than likely one you will agree with if you step back from the hype a bit.

The series:

  1. What It’s Like to Be a Toaster (you are here)
  2. Taking Toasty’s Temperature
  3. No Dry Land
  4. Power Drills and Primates

If you want the groundwork, Sorting Algorithms Don’t Hallucinate is a standalone prequel that clears away the lazy idea that consciousness will simply emerge from a big enough pile of computation, though you don’t need it to follow what comes next.

In this article we’re going to lay the philosophical groundwork. We’ll paint the picture of what consciousness looks like, how AI might be conscious, and the shape of it we’ll carry through the rest of the series.

Consciousness doesn’t require meat

The first step in this journey is the easy one and it’s pretty well trodden. The claim is that consciousness is probably substrate-independent. If you’re fresh to the argument, it goes something like this: you can replace one cell at a time with something that does the exact same job, but it’s not a cell, and throughout that journey you’ve never once lost a piece of who you are, because all the pieces that make you you are still doing the exact same thing, so you’re still you.1

What that means is that there’s a bunch of components that are doing something, and that something is consciousness. In humans that’s a bunch of cells in the brain. We’re not affording any special privilege to cells here, like quantum weirdness or anything soul-shaped that lives outside of this.

Stripping away that privilege can feel like it cheapens things, but I personally don’t feel it diminishes anything at all. If you’re religious, one way to see it is that God created the physics that houses our experience, and somewhere within those natural laws there’s room for our souls.

We don’t have to explain it, and it’s not new thinking. There’s a lot of work done on this, and a lot of smart people who reason that whatever consciousness is, it can probably be done just as well with silicon, in theory.

Of course, nobody has done this yet, and we don’t have machine consciousness yet. Whatever set of laws that govern reality has meant that biology still has the lead. When it comes to things like self-replication (two humans conspiring to create a soul) or just having an internal experience, alternative substrates haven’t had their innings.

We don’t rule it out though. Just because it hasn’t been done yet doesn’t mean it cannot be done. There’s an idea called substrate chauvinism that holds that only wet carbon can be,2 but there’s probably more than one way to bake the consciousness cake, so we can’t rule out that silicon might work for a carbon-free version of the same thing.3

It’s as real as it gets

Your own experience, your inside world, is real. Most people don’t argue that. What some argue is that consciousness is an illusion. The reasoning starts where the last section left off: if consciousness is a pattern of activity in physical stuff, then describing it all the way down leaves only information and computation, bits and arithmetic. Everything the brain does is covered by that mechanical story, so the argument runs that the vivid inner feeling sitting on top is a trick the machinery plays on itself, nothing over and above the bits.

The first steps of that are fair. Consciousness probably is a pattern that can run on more than one kind of machine, so at the bottom it is information and computation, a physical process, the same matter and energy arranged and moving in a particular way.

Where it goes wrong is the leap from there to “so the feeling isn’t real”. Calling consciousness “just information” is like calling a sunset “just information”. A sunset is light scattering through the air, and it is also, at the same time, a sunset, no less real for having a physical description underneath. Waving off your kid’s first steps as “just physics and biology” does the same, and so does shrugging at the throb of a thumb whacked with a hammer. The physical story is true, and it takes nothing away. If everything real bottoms out in some physical process, then giving ours its mechanical name leaves its reality and its meaning exactly where they were.

That reply is too quick, though, because the illusionist is making a more specific claim. Reductionists explain the world from the ground up: understand the rules of the smallest parts and you understand the whole. Illusionism leans on that instinct, hinting that the top-level mechanisms are just a result of the lower-level ones, so the lower level is where all the explanatory power is supposed to sit.4

I disagree, and my reason is causal. Things happen because of things on their own level.5 You explain a traffic jam in terms of traffic, even though it bottoms out in physics, and a love letter is best penned in the language of emotion, unless it’s between two physics nerds. The high level is where the cause and effect actually happens. That usually makes it the best place to explain something, because the predictions you make there survive contact with reality.

That high level, for consciousness, is something the philosophers call qualia. It’s also called phenomenal consciousness, and the difficulty of explaining it is what philosophers mean by the hard problem of consciousness. That difficulty is a good hint at why illusionists and others reach for lower levels with more concrete rules. The term qualia describes the experience of observing the redness of a rose, what it’s like to be a bat, and other fun games that philosophers have come up with over the years. It gets talked about because the experience it points at is undeniable. People have an experience, and it’s one of philosophy’s biggest unfinished jobs to explain that experience.

The illusionists will say qualia don’t matter, they’re just a side-effect. The dualists will say qualia come from something spooky outside the natural laws. I think the natural laws are big enough to let us have an internal experience of things, and that the experience itself is the causal pattern that best explains how our behaviour unfolds. It might be built on a substrate, yet the language that best captures it sits a level above, where experience lives. We can’t write off our experience by just calling it an illusion. There are no illusions here, and no ghosts.6

If we follow this line, which is where I think most people sit who haven’t spent their lives playing one philosophy game or another, then we can walk around one of the philosophy traps that wait for us.

P-zombies are that trap. The idea that something can appear in all aspects to be alive but internally there is no experience. That’s a fun what-if, and it might explain your brother-in-law, but it’s a dead end. On the functionalist view, qualia are the causal mechanism that best explains and predicts behaviour.7 It stretches the imagination, then, to say something could have the identical behaviours while lacking the internal experience that usually produces them. You can sweep the causal explanation into a substrate, but the mechanism itself still operates in the same domain. Consciousness is not a trick, the pattern is as real as it gets, even if the underlying substrate is as mechanical as the atoms in a sunset.

Our self-portrait

So the causing happens at the high level, and for consciousness that high level is experience. That still leaves a question: what does the self actually look like?

Humanity has been painting our self-portrait for a very long time. Every generation of artful scholars had something to contribute, with great minds adding a few brush strokes here and there. A few hundred years ago the subject of this painting appeared as a blurry ghost. Now it’s clear as a photograph, but in some ways more mysterious.

Let me walk you around it, from the outside in, because if you haven’t glanced this portrait during your research it’s a bit of a staring-into-the-void moment, and every part of it is there for a reason.

Start with the frame. The self shows up in many layers, but it isn’t smeared evenly across them. Go too low, down to the ion channels in the brain, and we lose the thread. Go too high, up to the whole of human civilisation, and we also lose sight of our subject. The core of the self sits somewhere in the middle-ground, at the level where the brain handles meaning, and that’s what frames the piece.

Inside the frame the painting shows a natural environment, full of detail and colour, the shapes bold and almost larger than life. That boldness is the first clue: what’s on display is a mind’s model of a world. We have a constantly recalibrating neocortex turning raw sensation into things that mean something,8 and a model like that paints in the meaning as well as the light, which is why everything runs a little more vivid than life.

Right in the centre of the environment there’s a mirror, and everything in the mirror is like a moonlit night, full of shadows. The mirror is there because the same process that models the world also senses its own operations while it does so, which is how we come to have an experience of our own experience.9 Reflection turned out to be a useful trick, and evolution kept it.10 That’s how the mirror ended up in the painting, it’s a part of us, a fixture in our brains.

Look into the mirror and you’ll find an artist at work, seated at a canvas just on the other side of the glass. The artist is making sense of what they’re seeing, and they’re imbuing those sensations with shape, colour and meaning. Because the artist is also painting themselves, that self-sketch is also an impression.11 The piece they’re working on is in the language of the artist’s particular style, and it’s at that level of abstraction that thoughts and attention live.

This is why the artist holds a brush. We are constantly sensing pieces of the whole and extracting meaning that we stitch into a full picture. That’s how our vision works, taking a few sharp glimpses a second and reconstructing the rest of the scene from a blurry periphery,12 and to some extent that’s how our internal model of the world works, including our self-conception, where our interpretations of our own actions and observations mix with our model of the world and merge to form a picture of who we are.13 The world gets painted piece by piece, and so does the painter.

It takes your eyes a while to adjust to that moonlit dark, but then you see them: countless individuals waiting in the shadows, all a little bit different, all holding brushes, taking turns at the canvas. The crowd is there because up close the mind is messier than “a self”. It’s composite, many minds at once running in parallel, with a searchlight of attention swinging across them.14 We can sing along to a song, drive a car, and think about fast fashion trends. We do this all at the same time, and if we ask which one we were really doing, the honest answer is all of them and none of them.15 That’s why nobody in the mirror works alone.

The light in the mirror moves around too, like there are clouds passing in front of an unseen moon. While where we place our attention and our brush is mostly ours to determine, things can surface on their own. We’ve all suddenly remembered leaving the iron on at home. Something was happening in the background, the clouds shifted, and the version of us that was watching stepped up to the canvas.16

The striking thing about this painting isn’t any of that though. The one currently at the canvas is staring straight out of the painting, at the viewer. Realising it’s looking right at you, in the way some creepy paintings do, is the first jump-scare. A few seconds later, you recognise those eyes as your own, then you realise all the people in the background are you as well.

The first time I saw this, that was the point I bailed. Announced to an empty room that I was going for a walk now, and spent the next hour with a dog who was wondering what the hell was going on with me.

The canvas in the canvas

The portrait has one more mystery. The canvas the artist works on faces away from us, so we assume it contains the same picture we’re looking at, but only the artist depicted knows. With some reflection though, I think I already know the answer to that one, it’s whatever mental image of this painting I currently have in my head.

We are the picture on the hidden canvas: a nested, self-recalibrating control structure17 containing multiple parts, wrapped around a much older set of drives,18 in a constantly changing environment. The natural next question is who’s holding the brush then, but that question is the last bit of the old portrait talking. The machinery does the brushwork, the many figures take their turns, and the person, the felt self with a name and a history, is what shows up on the canvas.19 This self-portrait is what the hidden canvas shows when the artist is looking directly into the mirror, and the clouds shift to illuminate the scene enough for a glimpse.

If the picture feels invented, it helps to know how it was painted. Our self-portrait spent most of history behind the closed doors of one authority or another. Over the last few centuries the work moved into the open, more people were invited to contribute, and they talked, they shared ideas, and the technique and tools got much better.

The most recent details are thanks to our experts in neurobiology, information theory and the cognitive sciences. They have been tracing the detail of the lower machinery,20 and you can see their work in the sharp outlines and exact strokes. Experts in philosophy and developmental psychologists have been tying these efforts into the overall piece, a bit of shading here and there that draws the eye, and making sure the right things remain in focus. It’s no accident that one of developmental psychology’s most famous experiments involves recognising ourselves in a mirror. More recently AI researchers have arrived with new brushes and new paints, which is adding detail into areas that were previously shaded out.21

Working on this picture is a real trip for those involved. We kind of collectively expected, but not really, what the painting ended up showing.22 It wasn’t an accident, and it wasn’t artistic choice; one crowd worked up from the machinery, another worked down from behaviour, and they converged on the same scene. All the elements of this self-portrait were assembled from people poking at neurons, thinking about things deeply, and running experiments. The only shadowy bit is seeing our multiple selves in the mirror, and even then those are recognisable if you know what to look for. Everything in this painting is there because it has generations of science and thinking behind it.

Gone is the portrait of a blurry ghost floating in the heavens. The modern portrait is much more concrete and interesting.

Minds in general

While this is all deeply introspective, we can notice a few things about the symmetry of this picture. When you stare enough at it, you wonder what other minds might fit that symmetry. The contributors to this self-portrait agree that we could probably have an elephant there holding the brush, maybe even a machine. So taking our self-portrait and the symmetry we’re noticing, we can get a sense of what a more general self looks like.

Any line we draw around consciousness is partly a choice, the way every definition is, so the list below is one we have to argue for, with each item earning its place. A system has the shape of consciousness when these hold together:

  • it models the world, turning raw input into things that mean something;
  • it models itself as one more thing in that world, turning its own workings into things that mean something;
  • it runs this as a live process, recalibrating against what actually happens from one moment to the next, never settling into a fixed mapping computed once and left to stand.

All of this has to be happening at the level of meaning, the same high level where we found the causing earlier, up among thoughts and attention and not down in the ion channels or the transistors. That is a scoping rule more than a separate ingredient: it tells us where to look.

The third item is the contested one. That consciousness has to be a live and ongoing process, and not a frozen configuration, is an argument we are making, and it happens to decide most of the hard cases, so we mark it as ours to defend rather than a settled fact.23

With the mechanism in view, we can say why each item is on the list, by asking what a mind would be missing without it.

Take away the world. A system that models only itself, with no world around it, gives the self-model nothing to be a self among. Our own sense of self rides on the same machinery that models the room, the body and the light, and it learns what it is by watching itself act among those things. Strip the world out and the self-model has nothing to calibrate against and nothing to notice itself as part of, so it thins to a model with no content. That is why a world is on the list, and why embodiment keeps surfacing: a mind seems to need to be somewhere, taking itself as one more of the things it notices. Whether that world has to be the real one, or a richly modelled inner one would do, is a further question we can leave open.

Take away the self-model and the opposite gap opens. A system can model the world in enormous detail, a weather engine, a chess program, a sorting routine, and still there is nobody the modelling is for. The lights are on over the world and off over the modeller.

Take away the live process and, on the theories we leaned on above, the experience goes with it. A single frozen frame of all that machinery, however detailed, is a photograph of a mind, and the experience these theories describe lives in the motion the photograph has stopped.

None of this is the whole of consciousness, and we are not claiming it is enough. Someone else will set the bar higher and ask for language, or reasoning, or the ability to tell us about it. We keep the bar deliberately low, because a low one draws a circle wide enough to hold both us and the cases we are least sure of, the animals we grant an inside and the machines we are arguing over. A higher bar mostly redraws the circle around people who talk, which is where we started and the thing we are trying not to assume.

Now the point of all this. Once we can say where the self lives, “is it conscious?” turns into a sharper question: does the thing run the right kind of pattern at a level where it models itself as well as the world? For us that level is meaning and attention and self-modelling, built on neurons. For a machine the question is the same, asked well above the hardware. Look past the transistors to the level where it models the world, and ask whether it models itself there too.24 That is a real question with a real answer, even if it’s not fully answered yet.

The same question also tells the candidates apart, separating the things that could be home from the things that plainly aren’t. A sorting algorithm has no such level, so there is nobody home. A system that models the world, models itself, and keeps both running as a live process has at least the shape of the thing. The question then shifts: how close is its experience to ours, and how far towards something alien?

The circle we’ve drawn will come under pressure, and we don’t have a good record of holding one steady. Intelligence had a circle too. We used to think it meant playing chess, then a machine did it and we reflexively moved the line, and over the last ten years we’ve run out of places to hide it if we still want to say machines can’t be intelligent.25 Part of why we draw the consciousness circle carefully now, and mean to keep it where it is, is so that when the same pressure arrives we distrust a confident yes as much as a confident no, instead of shrinking the circle to keep the machines out. It also matters what the word would buy us. A human and a shark are both alive, and nobody confuses the two or imagines we owe them the same; a useful sense of “conscious” would be just as broad, one word covering creatures with very different claims on us. So even if some future system clears the bar, fitting the definition does not by itself make it a moral patient we owe things to. That is a separate step, which is the path we’ll walk down the rest of this series.

Setting the moral question aside, pointing at the mechanism does not flatten the experience, any more than naming the notes flattens the music. People who love a mystery often fear that explaining it will kill it, but the ones who actually went looking, the anatomists opening the body, the natural philosophers taking the sky apart, tended to come away more impressed than when they started. Consciousness runs the same way. Even if we came to understand it completely, that would not discount your inner world or drain a drop of colour from it. If anything it puts some ground under the experience beyond your own say-so, so that what you feel from the inside turns out to be a real thing seen from the outside too.

The catch is that looking this closely brings on a familiar flinch, the “I’d rather not know” that guards anything we suspect might be uncomfortable. For the questions this series is going to ask, we have to look anyway, and look directly. It wants a certain maturity from us, a willingness to accept the self whatever the self turns out to be. Learning how the portrait is painted might change the painting a little, but remember that you hold a whole crowd of selves, and that crowd keeps its capacity to be just as it does now. Having walked some way down this path already, I can tell you there are no basilisks waiting on it,26 nothing that harms you only by being known. There is only us, and we will have to look at ourselves with clear eyes.

That clear-eyed look is where this turns strange. If the self is a kind of pattern rather than a kind of meat, and the pattern for consciousness is potentially wider than the human version of it, then the ghost in the machine does not need to be anything like ours.

A mind you couldn’t recognise

There is a lot of room inside the pattern definition of consciousness for very different kinds of mind: anything that experiences the world and itself, however and wherever that experiencing is built.

Philosophers like to reach for a bat here, and how strange it must be to be one.27 But once we take substrate independence seriously, things get unrecognisable far faster than a bat. What is it like to be a toaster? Thankfully, as best I can tell, my Breville Toast Control doesn’t run the right sort of pattern. Maybe it would do a better job if it had an inner world, or maybe it would go on unevenly toasting bread out of spite. It is almost impossibly hard to reason about. With a bat we at least share an evolutionary pathway, so there are similarities we can squint at; the toaster’s mind, if it had one, would be unknowable by any familiar measure of meaning.

Being unfamiliar, that mind would also be hard to spot. We wouldn’t know what it looks like from the outside, and our only detector is “is it like me?”. There’s an inverse anthropomorphic sin at work here: instead of projecting human experience onto things that lack it, we take human experience as the template for all experience, so anything alien reads as nobody home even when the pattern says otherwise.

You can see why people want a consciousness-o-meter, an instrument that reports whether a thing has an inside so we don’t have to relate to it to know. We half have one already. Temperature made this trip once, from a private matter of hot and cold hands to a public number, because under the feeling there was a clean physical quantity for the thermometer to point at. There is now a device that separates a waking brain from one under anaesthetic or in a coma, to better than 95%.28 But it is calibrated on us. It knows what a conscious human looks like from the outside and checks for that, so it has nothing to go on when it’s pointed at a mind we share nothing with. That is the whole problem in miniature: a meter can tell us that something like us is home, and still tell us nothing about what it is like to be a thing unlike us.

We might genuinely end up with something that can measure the signs of consciousness, and measure how human-like this is, but the first batch of those won’t tell us anything about my upgraded toaster’s crisis of meaning.29

So where does today’s AI actually sit

Using the lens we’ve built so far, we can turn the focus on today’s AI. In 2026 we have a set of very impressive, personable LLMs that can carry a conversation that blows last century’s Turing tests out of the water.

It’s tempting to read that as the machine waking up, but the Turing test was only ever a test for intelligence, not consciousness. What happens when we apply our definition of a consciousness pattern to these systems gives us a strange result. LLMs do turn input into meaning, which is the first ingredient, and they reason about themselves in a limited way, which is a piece of the second. Where they fall short is the third: a model like this runs as a single feedforward pass, working out a response and stopping, without the ongoing, self-correcting loop that keeps a mind running from one moment to the next.30 So they have a few of the ingredients, but not all of them.

What they do have is serious credibility as agents that appear to hold opinions and values, but even that stops short of consciousness. The moth wears a pattern of big eyes on its wings to ward off predators. These AI are much the same: companies with billion-dollar budgets and the finest talent train them to appear like they have eyes, but, like the moth, they are only wearing the costume, and there is nothing seeing through those eyes.

Knowing all that doesn’t help much in the moment. When I talk to one of these machines, it still feels like I’m talking to a real person. In a blind test, I might not be able to tell the difference. I have to remind myself that I’m not, because the fake eyespots on the moth’s wings are just that good. They can even wink at you convincingly, so without my knowledge of exactly how these AI work, down to the maths, I’d be crazy to think they didn’t have an internal world. Luckily though, they don’t, at least not yet. They might soon graduate from being toasters to conscious toasters, but our definition is minimal by design, so clearing it only says an inner world is present and leaves its character wide open, which in their case would still be utterly alien.

Where this is likely to change is with robots. Robots need to model a body, a world, persistence, and goals. Over time this drifts the design spec toward the kind of nested, drive-serving, self-recalibrating structure that within us underwrites a self. That’s where utterly alien gets a bit closer to our home turf, but what it’s like to be a robot could be closer to something like a spider’s experience than our own.

So what do we owe the toaster?

Consciousness is real, and it’s probably the result of a processing pattern that can run on many different substrates. If we accept that, then it’s also possibly quite alien, often unrecognisable, and more likely than not to make its silicon-based debut in a form we simply cannot relate to. We haven’t built it yet, but we’ll likely do that soon.

An imminent development like this requires some pretty serious consideration. If an internal world is a real thing that could really be there inside the machines, what does our ethics say about it? In the next part we look at what this means if we grant these alien minds might be moral patients, which is the next step in figuring out what we owe the machines.

Footnotes

  1. The gradual-replacement intuition is D. J. Chalmers’s organisational invariance, argued via the “fading qualia” and “dancing qualia” thought experiments: swap neurons for functionally identical silicon one at a time and either the experience implausibly flickers or fades while behaviour holds fixed, or the organisation fixes the experience. D. J. Chalmers, “Absent Qualia, Fading Qualia, Dancing Qualia,” in Conscious Experience, T. Metzinger, Ed., 1995. We take the invariance and leave Chalmers’s dualism (see 6). ↩

  2. The substrate-first views we reject: J. Searle’s biological naturalism, on which only certain biological causal powers yield mind (“Minds, Brains, and Programs,” Behavioral and Brain Sciences, 1980), and Integrated Information Theory, which locates consciousness in a substrate’s cause-effect structure and predicts a functional duplicate on the wrong architecture is not conscious (G. Tononi and the IIT literature). Both make the material the locus; the pattern view makes it a detail. ↩

  3. Multiple realisability, that the same mental state can be realised in different physical stuff, is the classic functionalist move: H. Putnam, “Psychological Predicates” (1967); J. Fodor, “Special Sciences” (1974). ↩

  4. The opposite error is illusionism: that phenomenal consciousness is a user-illusion, nothing over the functional facts. D. Dennett, Consciousness Explained (1991); K. Frankish, “Illusionism as a Theory of Consciousness,” Journal of Consciousness Studies (2016). We keep the deflation of the mystery and reject the deflation of the experience. ↩

  5. The worry you are refusing is J. Kim’s causal-exclusion argument: if the microphysics does all the causal work, higher levels look epiphenomenal. The pattern view answers that the level where a self’s causation closes is where the causal work genuinely lives, not a redundant gloss on the gates. J. Kim, Mind in a Physical World (1998). ↩

  6. The position being staked: qualia are real high-level causal patterns. Chalmers reads them as a ghost above the function, Dennett as an illusion; the pattern view rejects both. Dennett’s own “Real Patterns” (Journal of Philosophy, 1991) argues higher-level patterns are real when they are indispensable; we take that and turn it against his illusionism about qualia. ↩ ↩2

  7. The crowd here is the causal or functionalist theory of mind: mental states are fixed by the causal roles they play between inputs, other states, and behaviour, so a state that fills the whole role of an experience is that experience, with nothing left over to remove while the behaviour holds steady. That is the standard reason this camp doubts the philosophical zombie. D. Lewis, “Psychophysical and Theoretical Identifications,” Australasian Journal of Philosophy, vol. 50, no. 3, pp. 249-258, 1972; D. M. Armstrong, A Materialist Theory of the Mind, Routledge & Kegan Paul, 1968. ↩

  8. Perception here is inference, not reception: the cortex runs a generative model whose top-down predictions are corrected by bottom-up error, so what reaches awareness is the brain’s best guess at the causes of its input. Hawkins puts prediction at the centre of the whole neocortex in his memory-prediction framework, resting on Mountcastle’s observation that cortex is strikingly uniform and so may run one common algorithm across its sensory regions; Rao and Ballard gave the mechanism a concrete demonstration in the visual cortex specifically. J. Hawkins and S. Blakeslee, On Intelligence, Times Books, 2004; V. B. Mountcastle, “An Organizing Principle for Cerebral Function: The Unit Module and the Distributed System,” in The Mindful Brain, MIT Press, 1978; R. P. N. Rao and D. H. Ballard, “Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects,” Nature Neuroscience, vol. 2, no. 1, pp. 79-87, 1999; A. Clark, Surfing Uncertainty: Prediction, Action, and the Embodied Mind, Oxford Univ. Press, 2016. ↩

  9. On higher-order theories, a state becomes conscious when the system represents itself as being in that state, so awareness of an experience is a second-order representation of a first-order one rather than an extra glow attached to it. H. Lau and D. Rosenthal, “Empirical support for higher-order theories of conscious awareness,” Trends in Cognitive Sciences, vol. 15, no. 8, pp. 365-373, 2011; D. M. Rosenthal, “Two concepts of consciousness,” Philosophical Studies, vol. 49, no. 3, pp. 329-359, 1986. ↩

  10. Self-monitoring is adaptive because it lets an agent spend effort by uncertainty: skip a test it would fail, gather more before committing, study what it has not yet learned. Uncertainty-monitoring turns up in some non-human animals too, which is why metacognition is treated as evolved rather than a human novelty, though how far down it reaches stays contested. J. Metcalfe, “Evolution of metacognition,” in Handbook of Metamemory and Memory, J. Dunlosky and R. A. Bjork, Eds. New York: Psychology Press, 2008; S. M. Fleming, R. J. Dolan, and C. D. Frith, “Metacognition: computation, biology and function,” Philosophical Transactions of the Royal Society B, vol. 367, no. 1594, pp. 1280-1286, 2012. ↩

  11. On self-model theory there is no inner thing that is the self, only a self-modelling process the system cannot experience as a model, so it takes the model’s content for reality. Metzinger’s slogan is that no one ever had a self, only an ongoing self-modelling. T. Metzinger, Being No One: The Self-Model Theory of Subjectivity, MIT Press, 2003. ↩

  12. The eye holds each fixation for roughly 200 to 300 ms, giving about three to four jumps (saccades) a second rather than a smooth stream, and sharp detail is confined to the central one to two degrees of the fovea, with acuity dropping steeply toward the edges. Most of the scene we seem to see is reconstructed rather than resolved, which is why we miss large changes when the flicker that would flag them is masked (change blindness). K. Rayner, “Eye movements in reading and information processing: 20 years of research,” Psychological Bulletin, vol. 124, no. 3, pp. 372-422, 1998; D. J. Simons and R. A. Rensink, “Change blindness: past, present, and future,” Trends in Cognitive Sciences, vol. 9, no. 1, pp. 16-20, 2005. ↩

  13. Split-brain work makes the stitching visible: when the mute right hemisphere drives an action, the speaking left hemisphere invents a plausible reason and reports the confabulation as the truth, which led Gazzaniga to posit a left-hemisphere “interpreter” that narrates a coherent self out of partial information. The self on this view is less a thing observed than a story told. M. S. Gazzaniga, “The split brain revisited,” Scientific American, vol. 279, no. 1, pp. 50-55, 1998; A. Damasio, The Feeling of What Happens, Harcourt, 1999. ↩

  14. Two traditions meet here. Minsky models mind as a society of individually mindless agents with no central controller, and global workspace theory sharpens it: many specialised processors run in parallel, and consciousness is the momentary broadcast of one coalition’s content to the rest. The “searchlight” image is Crick’s, who tied attentional selection to the thalamic reticular complex. M. Minsky, The Society of Mind, Simon & Schuster, 1986; B. Baars, A Cognitive Theory of Consciousness, Cambridge Univ. Press, 1988; F. Crick, “Function of the thalamic reticular complex: the searchlight hypothesis,” Proc. Natl. Acad. Sci. USA, vol. 81, no. 14, pp. 4586-4590, 1984. ↩

  15. The trick is automaticity: a practised task like driving runs as an automatic process, without attention and without consuming the capacity-limited channel, which is freed for the task in focus. “None of them” follows from that single serial bottleneck sitting on top of the parallel processors. R. M. Shiffrin and W. Schneider, “Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory,” Psychological Review, vol. 84, no. 2, pp. 127-190, 1977. ↩

  16. Global workspace theory names this transition ignition: a representation held below threshold by specialised processors can cross into a self-sustaining, brain-scale state and be broadcast to the whole workspace, which is why a background item can surface unbidden. S. Dehaene, M. Kerszberg, and J.-P. Changeux, “A neuronal model of a global workspace in effortful cognitive tasks,” Proc. Natl. Acad. Sci. USA, vol. 95, no. 24, pp. 14529-14534, 1998; S. Dehaene and J.-P. Changeux, “Experimental and theoretical approaches to conscious processing,” Neuron, vol. 70, no. 2, pp. 200-227, 2011. ↩

  17. The framing is hierarchical predictive processing: a layered stack of controllers, each predicting the level below and correcting on the error, which formalises an older cybernetic idea that behaviour is the control of perception rather than the output of commands. K. Friston, “The free-energy principle: a unified brain theory?,” Nature Reviews Neuroscience, vol. 11, no. 2, pp. 127-138, 2010; W. T. Powers, Behavior: The Control of Perception, Aldine, 1973. ↩

  18. The drives are the evolutionarily conserved, largely subcortical affective-motivational systems mammals share, seeking, fear, rage, care and the rest, each with its own circuitry. They are old in the sense of being conserved across mammals; the once-popular “triune brain,” which cast them as a reptilian relic layer, is not a sound evolutionary account. J. Panksepp, Affective Neuroscience: The Foundations of Human and Animal Emotions, Oxford Univ. Press, 1998. ↩

  19. The jump being made here is the rejection of the homunculus, the little inner viewer for whom the mind’s show is put on. Any such viewer would need its own inner viewer to do the watching, and the regress never pays out. Dennett names the imagined venue the Cartesian theatre and dismantles it: there is no place in the brain where “it all comes together” for an audience of one, only processes whose contents are available to other processes. D. Dennett, Consciousness Explained, Little, Brown, 1991. The mirror-facing-a-canvas regress in our painting bottoms out safely for the same reason: each level is just the current content of one live self-model (see 11), always one step behind itself. ↩

  20. D. Dennett, Consciousness Explained, Little, Brown, 1991. A spoiler on the title: Dennett is an illusionist, so consciousness is not explained at the qualia level here. What the book and its neighbours do explain is a great deal about the human substrate, which is the useful part for us. ↩

  21. The new detail is largely a matter of instruments. Mechanistic interpretability lets us open a trained network and read its internal representations directly, something we have never managed with a living brain. A model trained only to predict legal Othello moves, for one, turns out to build a manipulable model of the board inside itself, the kind of internal structure the science of the self has always had to infer from outside. K. Li et al., “Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task,” Proc. ICLR, 2023, arXiv:2210.13382. ↩

  22. From below, the ascent runs from Shannon’s measure of information up to formal neuroscience of consciousness such as integrated information theory. From above, developmental psychology worked down from behaviour to the self, through the mirror mark test for self-recognition and the false-belief test for attributing minds to others. C. E. Shannon, “A Mathematical Theory of Communication,” Bell System Technical Journal, vol. 27, no. 3, pp. 379-423, 1948; G. Tononi, “An information integration theory of consciousness,” BMC Neuroscience, vol. 5, art. 42, 2004; G. G. Gallup, Jr., “Chimpanzees: self-recognition,” Science, vol. 167, no. 3914, pp. 86-87, 1970; H. Wimmer and J. Perner, “Beliefs about beliefs,” Cognition, vol. 13, no. 1, pp. 103-128, 1983. ↩

  23. That consciousness is a live, ongoing process, and not a configuration computed once and frozen, is a widely shared direction with no agreed mechanism behind it. Some version of it runs through the stream tradition (James), the phenomenology of time (Husserl, Varela), and the leading theories in the science: recurrent processing, on which a purely feedforward sweep does not suffice for awareness; integrated information, on which a purely feedforward network has zero integrated information and counts as a “perfect zombie”; and global workspace, on which conscious access is a sustained, self-amplifying state. The dissenting minority is the snapshot or cinematic view, on which a single momentary configuration could carry an experience. One distinction is load-bearing: the requirement is real-time process, not learning. A system with fixed parameters but genuine ongoing dynamics still qualifies, since inference and learning come apart, and demanding that the parameters themselves keep changing for every conscious moment is a stronger claim almost nobody defends. V. A. F. Lamme and P. R. Roelfsema, “The distinct modes of vision offered by feedforward and recurrent processing,” Trends in Neurosciences, vol. 23, no. 11, pp. 571-579, 2000; M. Oizumi, L. Albantakis, and G. Tononi, “From the phenomenology to the mechanisms of consciousness: Integrated Information Theory 3.0,” PLoS Computational Biology, vol. 10, no. 5, e1003588, 2014; F. J. Varela, “The specious present: A neurophenomenology of time consciousness,” in Naturalizing Phenomenology, Stanford Univ. Press, 1999, pp. 266-314. ↩

  24. The systematic version of this move looks past the hardware to the functional level, deriving indicator properties from theories of consciousness (global workspace, higher-order, predictive processing) and checking specific architectures against them rather than arguing from behaviour. The internal world-models that make this more than behavioural guesswork are the subject of 21; the point here is that they speak to representing the world, not yet to representing a self. P. Butlin, R. Long, et al., “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness,” arXiv:2308.08708, 2023. ↩

  25. The retreat has a name. In Gödel, Escher, Bach Hofstadter states “Tesler’s Theorem,” that AI is whatever hasn’t been done yet, glossing it as the habit of no longer counting a mental feat as real thinking once it has been programmed; Tesler’s own version was “intelligence is whatever machines haven’t done yet.” The concrete peg is Deep Blue, which beat the reigning world champion Garry Kasparov in their 1997 rematch, a year after Kasparov won their first match. D. R. Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid, Basic Books, 1979; P. McCorduck, Machines Who Think, 2nd ed., A. K. Peters, 2004. ↩

  26. A basilisk here is a piece of knowledge said to harm you simply by being known. The term is David Langford’s, from a story in which a certain image crashes any mind that looks at it. D. Langford, “BLIT,” Interzone, no. 25, p. 40, Sep.-Oct. 1988. The rationalist community’s own famous specimen is Roko’s basilisk, a 2010 LessWrong post by the user Roko imagining a future AI that punishes whoever failed to help bring it about, taken seriously enough as an information hazard that it was deleted and its discussion banned for years. The formal category is N. Bostrom, “Information Hazards: A Typology of Potential Harms from Knowledge,” Review of Contemporary Philosophy, vol. 10, pp. 44-79, 2011. My claim is the mild one: ordinary self-knowledge is not of this kind. ↩

  27. T. Nagel, “What Is It Like to Be a Bat?,” The Philosophical Review (1974): even a fellow mammal has an inside we cannot reach from outside. The machine case is Nagel’s point with the shared biology removed. ↩

  28. A crude consciousness meter already exists for people. The perturbational complexity index pings the cortex with a magnetic pulse and measures how richly the brain echoes back, which separates wakefulness from sleep, anaesthesia and coma with better than 95% accuracy across hundreds of measurements. It grows out of integrated information theory and is calibrated entirely on human brains, which is exactly why it has nothing to say about an unfamiliar substrate. A. G. Casali et al., “A theoretically based index of consciousness independent of sensory processing and behavior,” Science Translational Medicine, vol. 5, no. 198, 2013; S. Casarotto et al., “Stratification of unresponsive patients by an independently validated index of brain complexity,” Annals of Neurology, vol. 80, no. 5, pp. 718-729, 2016; G. Tononi, M. Boly, M. Massimini, and C. Koch, “Integrated information theory: from consciousness to its physical substrate,” Nature Reviews Neuroscience, vol. 17, pp. 450-461, 2016. ↩

  29. I once lucked on a chance to ask Chalmers about a meter like this after he gave a talk in Canberra, not clocking that he had already given the idea a good chunk of a book. Chalmers works through the epistemics of machine and simulated consciousness, including whether we could ever tell one is present from the outside, across several chapters of Reality+: Virtual Worlds and the Problems of Philosophy, W. W. Norton, 2022; he first argued that artificial consciousness is possible in The Conscious Mind, Oxford Univ. Press, 1996. He is doubtful that any meter settles the matter, since a meter can only be as good as the theory of consciousness it assumes, and no such theory is settled. ↩

  30. This is the liveliest corner of the current debate, and it is genuinely open. On theories that make real-time recurrence necessary, a transformer is the wrong shape: within a single forward pass, activity runs forward through the layers instead of looping back, so it scores low on the very property those theories ask for. Chalmers puts the feedforward architecture and the absence of persisting internal states among the strongest reasons today’s models may not be conscious, while allowing it could be a contingent fact about current engineering. Against that, generating text one token at a time and feeding each output back as the next input reintroduces a loop at the level of the whole rollout, so the door is not closed. D. J. Chalmers, “Could a Large Language Model Be Conscious?,” Boston Review, 2023, arXiv:2303.07103. ↩

@misc{hollows2026whatitsl,
  author = {Hollows, Peter},
  title  = {{What It's Like to Be a Toaster}},
  year   = {2026},
  month  = jul,
  url    = {https://dojo7.com/2026/07/14/what-its-like-to-be-a-toaster/}
}