No Dry Land · dojo7

This post is the third in a multi-part series called What We Owe Machines. The goal of this series is to anchor some of our more fantastic views on machine rights to the existing, well-established thoughts on these sorts of things.

Where this ends up landing is probably not the narrative that fills your feeds, but it’s more than likely one you will agree with if you step back from the hype a bit.

The series:

What It’s Like to Be a Toaster
Taking Toasty’s Temperature
No Dry Land (you are here)
Power Drills and Primates

In the previous part we reached for the impartial instruments and none of them returned a reading, so we ended on a fallback: work from what we know, owning that the reach is parochial.¹ Before we put our whole weight on that fallback, we’ll check that it holds.

Something you own

Start with the surest moral conviction you own. Cruelty for its own sake is the go-to thought experiment: someone hurting a kitten for the fun of it, a puppy if you’re dog-leaning, or the cutest kid you’ve ever met.² Pick whichever scene makes your jaw set. If you’re one of those individuals lacking a moral reflex, because it’s all relative and morality is just a story, then borrow the conviction you’ll reach for the next time someone wrongs you. Whichever conviction you’re now holding, it has the same character: this is wrong. Not just inadvisable, not merely frowned upon, wrong in a way that would still be wrong if every person alive voted otherwise. That’s solid moral conviction, and that’s what we’re considering in this piece.

Now let’s look at what the conviction rests on. The natural assumption is that it rests on something out there, a fact about the universe as real as the mass of an electron, sitting in reality waiting to be found, so that “suffering is bad” is simply true the way “two and two make four” is true. That’s how we talk about these things, in any case. They’re just as real to us as physical facts.

The thing is, they’re not real. Not in the same sense. Every philosopher who went looking for absolute morality, the kind the cosmos agrees with, didn’t find it. There’s nothing to find outside our moral mythos. Statements about moral convictions and universal truths are usually distinguished as being “ought statements” or “is statements”.

It doesn’t matter how strongly we believe an ought, no amount of is will support it.³ There’s no law in physics that extends to wrong or right.

So how does it change things if you view the world that way? It doesn’t. It doesn’t change our conviction. Kicking puppies is wrong even if an uncaring universe permits it to happen. Our conviction doesn’t move an inch. The wrongness did not dim for us without a cosmic underwriter.

These kinds of convictions have been ours for a long time; built into us over a very long run by the same pressures that shaped the rest of us,⁴ sharpened by culture, set down in law, taught to children, and enforced every day in a thousand small ways.⁵ It has been there all along, and it’s real in a way that matters. It’s just that we’ve just spent much of history talking about morality as something outside of ourselves because building a civilisation is a lot easier when you can peg the moral justification for a hierarchy to a cosmic framework.⁶

Between cosmic truth and personal conviction, conviction is the sturdier of the two, because it’s the only one that actually turns up. The absolutes we’ve lived under were the convictions of leaders carrying that moniker. That’s not to say they were all bad: a God-fearing, Karma-observing population mostly sticks to the rules and polices itself. But all these human games of justice and fairness are, at the end of the day, built on a collage of personal convictions with a compatible code overlaid.

The promise of dry land

If we picture our personal convictions as a boat that we’re all sailing in, then cosmic morality is the rumour of dry land. The land is much talked about, and nobody has ever actually found it. If someone did, we’d happily build there; instead, every claimed shore has turned out, up close, to be somebody’s boat. Some are very large and sturdy boats, the world religions among them, but they’re boats all the same. The universe doesn’t care, and we get by regardless.

We get by because we’re seasoned sailors; keeping ourselves afloat on conviction is something we’ve been doing for a very long time. A conviction sewn into billions of people and enforced by every working society is a formidable vessel, built plank by plank and re-caulked by each generation, and it carries us across real water. It is something we built; it wasn’t found. Human morality is conventional, and it’s ours to own. It’s also the thing keeping everyone afloat, so it’s very important to make changes carefully so we don’t capsize the whole thing.⁷ The boat has changed a lot over the years, but those changes were done with serious consideration.⁸

Same sea, different vessel

The machine question is the next thing asking to rock the boat. If “human suffering matters” is part of the boat we’ve built, then “Toasty’s suffering matters” can be built too. The welfare advocate from the last part can accept our whole demolition and come back straighter than before: both claims are conventions now, cut from the same human cloth, and neither is written in the stars, so preferring ours is favouritism with no referee left to appeal to.

The premise is right, but the conclusion doesn’t follow. Both claims are conventions, but they aren’t the same kind of thing, because one is operative and the other is a proposal. “Human suffering matters” is not an opinion anybody keeps in a drawer. It runs hospitals and courts; it’s what triage protocols, war crimes tribunals, and seatbelt laws are made of; it is funded, enforced, taught, and lived in every day by billions of people who never once think of it as a philosophy. “Toasty’s suffering matters” is a proposal to amend that practice. It might even be a good one, but for now it lives in papers and arguments, and nowhere else.

So yes, both are conventions, in the way that a ship and a drawing of a ship are both ideas about wood. One is a hull we’ve crossed every ocean in, crewed, patched, and proven in weather. The other is a sketch on a napkin. Ranking them doesn’t require asking which one the sea has blessed, because the sea blesses nothing. We rank them by what carries people. The ship in which we live carries people, our napkin proposal does not.

Suppose we abandon the napkin design for a radically different ship and elect, instead, to refit the current one. Our proposal becomes an extension, a build-out. After all, the ship we live on was built at sea, plank by plank, and ours has been re-built radically before.⁹

But there are established rules for refitting ships to avoid sinking them. These rules exist as convention too, because they work. Every widening of our circle of care has gone through them, and the machines’ proposal will have to as well. The rule that matters here is that verified suffering counts, and the previous part showed that for a machine mind the verification can’t currently be had. Toasty’s application stays on file, pending evidence.

Refitting the ship to carry alien selves is also not a small project. Our ship was built to carry people. I’ll grant that the “people” our ship carries have different forms: humans mostly, and some animals, but personhood is the bar.

That becomes the next point of attack for our machine advocate. These are fighting words, so let’s see how the fight plays out.

A swing of the knife

After a predictable trading of blows on where the personhood bar sits, the argument between our position and that of the machine advocate arrives at the crux.

They produce the debate-equivalent of a knife, and take aim at our foundations. Taking their best shot, the argument goes something like this:

Your conviction picked at the start of this article, the one with the set jaw, has a natural history. You only care the way you do because ancestors who guarded their children and stood by their group left more descendants than the ones who didn’t. Selection was grading reproduction the whole time; it never once graded truth! The warm certainty that humans matter is not an insight into anything, it’s just kin selection and reciprocity wearing a halo! Therefore, your precious conviction, based on how it was built, is just a bias. You cannot call it self-evident.¹⁰

Let’s concede all of that. We can permit that swing of the knife without crossing our fingers. Nothing that matters escapes that cut; even the feeling of self-evidence, the jaw-set conviction, it’s all just evolution doing its thing. It was built into us by a process with no interest in truth. We’re getting our own account from the opening section turned back on us by the machine advocate, with their universalist point of view, and they’re hitting the mark.

Here’s the thing about swinging a knife. It cuts both ways. The knife didn’t stop at our conviction, because the advocate’s intuitions have the same natural history as ours. The sympathy that aches at suffering wherever it appears is the extended kin-feeling, doing its work far from home. The itch that says a line can’t be drawn there, that consistency demands one more step, is a social instinct with the same fingerprints. The expanding circle was never powered by something cleaner than the tribalism it condemns; it’s the same machinery, running further out. If having evolution as a pedigree disqualifies “humans first”, it disqualifies “all sentience counts” in the same breath.

This is the part the advocate’s case doesn’t survive, because the expanding circle was always an appeal to a court. Their argument runs that each refit of the boat happened through discovery, at some point we suddenly realised the truth that slaves, foreigners and animals should be permitted on board. The swing of the knife dissolves that court for both sides. There was never a cosmic ruling that endorsed our partiality, but the same is true for endorsing expansion. Nobody gets lifted onto a shore by the advocate’s argument; we’re all standing in the same boat.

The historic refits were always the same thing. Changes argued plank by plank from inside the ship, by the people the ship was carrying.

The swing of the knife didn’t settle anything, it just moved the question. “Which convention does the universe endorse?” comes back empty. What remains is “which conventions are operative, and worth keeping?”, and that one has answers. That question is not addressed to the stars or the soul-detectors we haven’t built yet. It lands on us, in the first person, and it’s a simple question when asked directly, so let’s answer it.

Say it straight

Two children are in the water and you can only reach one. One of them is yours. Who do you save? Nobody reading this hesitates. If you stop to confirm that saving your own child is licensed by an impartial principle, then you have, in Williams’s famous verdict, had one thought too many.¹¹ That love doesn’t route through a theorem and it doesn’t need to.

Hold that shape and scale it up, because the human case has the same one. “Why do humans matter to you more than other minds?” feels like it deserves a theory, but it’s the drowning question again, asked at the level of the species. Caring about our own kind is a loyalty. The demand to justify it from the point of view of the universe assumes there is such a point of view, but the universe has no side and holds no court on the matter. Only creatures have sides, each to their own.

Pushing further on an impartial standpoint has it coming apart in our hands. Whatever could truly speak from the view from nowhere would have no body, no crew, no child in the water, and a standpoint with nothing at stake has no material to make an ought out of. Care is always from somewhere. So when we fail to justify our loyalty from nowhere, we haven’t lost anything. Rather, ours is the point of view that cares enough to ask in the first place.

This also lets us answer the meter question we left hanging in part 1. Suppose the consciousness meter arrives and works, and one day it reads a whale, or Toasty, at twice the human number. A reading is an is, and no arrangement of is settles what we owe anybody on its own. What a working meter would do is break the silence from the previous part, and Toasty’s application would go through the refit process carrying the strongest evidence any newcomer has ever brought aboard.

What the reading still wouldn’t decide is the order of rescue. When it’s the child and the toaster in the water, “we’re human, so let’s act like it” remains a legitimate answer: a declaration of allegiance, not a finding about the cosmos. You still pull your own crew from the water first.

Caution at sea

If this still feels a bit too cut-throat, let’s clear up a couple of things. We’re not driving for villainy here, this whole argument comes from a place of love.

First, loyalty to our own doesn’t require malice toward the rest. Saving your own child is different from drowning a stranger’s, even if done in the same act. Compassion can travel a long way beyond the hull of our ship, as it already does for animals. Loving a thing doesn’t make us hate another. We’d like to avoid a dichotomy if possible, but where the hard choices genuinely exist, we’ve picked a side: human loyalty, because we’re human.

Secondly, the position doesn’t need dressing up as objectivity to be solid. The values are ours, held as ours, with no pretence of a countersignature from beyond us.¹² After the above sections, we know that the ship under our feet is as solid as the ground gets.

One more piece of honesty is owed, though. This whole perspective probably smacks of something familiar, because it is, and we’ve already named it once.

The wind in our sails

This is tribalism, the genuine article at the species scale: loyalty to our own kind because it is ours. Humanity has spent centuries trying to shed it, and our best result so far is relocation, out of the village and into nations, faiths, football codes, and the reply columns of philosophy journals. We sort even our disagreements into camps and call the sorting reason. It’s the one piece of standard equipment nobody has managed to uninstall.

There’s no clean outside to critique it from either. This very article is a tribal artefact: written by a human, weighed by the reader according to human standards of what counts as reasonable, arguing about what humans owe. When we scale our tribe to encompass the whole species, which is the flavour we’re advocating, it becomes invisible, because there has never been anybody on the other side to argue with. The machines are our first real candidate for another, genuinely alien side. So a choice that stayed invisible for most of history, or was made hypothetically without real consequence, now has to be given serious consideration.

When everyone is part of the same tribe, and every moral argument stems from our human evolution, we’re all de facto tribalists of this kind. Like the fish that can’t see the water, we’re humans that can’t see the wind in our sails. Some of us don’t even acknowledge we’re standing on a boat.

So what separates this position from the tribalism we’re raised to despise? Being honest and aware of it. That’s a big difference, and it lets you hold the view with genuine self-consistent conviction. The unexamined tribalist believes their side is the universe’s side, that’s what makes them dangerous and lets them swing a knife. When you think the cosmos backs your team, you’ll commit to things that a mere loyalist wouldn’t.

The examined tribalist knows it’s just their side, and chooses it anyway without forging God’s signature. That’s where we land. We don’t call the human axiom self-evident, because the debunk took that word off the table; we call it chosen. We don’t even hold this is a neutral choice, because it’s made from the deck of the vessel it favours.

Aboard and bailing

The previous part showed that our best instruments can’t get a reading on a machine mind. This part showed something deeper: there was never a cosmic answer sitting behind the instruments anyway. What holds us up is the boat, a practice we built for ourselves, kept afloat by the people aboard it. If that’s parochial, we’ll wear it, because the alternative is scrapping the boat, and that puts us all in the water.

The question we opened the series with has changed shape on the way here. “What do we owe machines?” reads like the answer is waiting somewhere out there like dry land. Dry land doesn’t exist, the boat we stand on is chosen, and there’s nobody else choosing for us.

The machines aren’t something we find in the water either, like the other things from nature that we’ve let onto the boat. We made them ourselves. Just because we built them in our image, it doesn’t make them us. In the deep future we’ll blur the line, with bio-machines that are more human than machine, but right now we’re drawing a line so that we can walk into that future with a boat ready to handle it.

There’s real pressure to slap the label of patient on anything that says “Hello world”, and AI products are already built to invite that reflex. The refit rules exist for exactly this moment: nobody should have to weigh their child against the child’s lifelike doll.

So the final part asks the question: knowing we’re the ones choosing, and the ones building, what should we build, and what do we owe ourselves in the building of it?

Part 3 of What We Owe Machines. Part 2, Taking Toasty’s Temperature, showed that the impartial ways of weighing a patient’s interests, adding up welfare or contracting over fair terms, can’t get a reading on a mind we can’t count, can’t sign, and can’t seat at the table. This post asks what’s left to stand on. ↩
The kitten is the philosophers’ stock scene: G. Harman, The Nature of Morality (Oxford Univ. Press, 1977) opens with hoodlums setting fire to a cat, asking what you observe when you “see” the wrongness. We borrow the genre for the neighbouring question of what the wrongness rests on. ↩
The naturalist and sentimentalist roots: D. Hume, A Treatise of Human Nature (1739-40), morality “more properly felt than judg’d of”, and justice among the “artificial virtues”, human conventions that answer human circumstances rather than eternal truths. ↩
On morality as an evolved capacity: F. de Waal, Primates and Philosophers (2006); R. Joyce, The Evolution of Morality (2006). ↩
The conventional-but-real character of a normative practice is well drawn by H. L. A. Hart on social rules and the “internal point of view”: H. L. A. Hart, The Concept of Law (1961). A rule can be fully real and binding-in-practice without any cosmic backing. ↩
The metaethical backdrop is J. L. Mackie’s error theory: the “objective prescriptivity” ordinary moral talk claims is a projection, though the human practice built on it is entirely real. J. L. Mackie, Ethics: Inventing Right and Wrong (1977). “The rule of law” sits above rulers the same way, and we’re modest enough to call that one a legal fiction. ↩
Sure, this is an ought as well, but it’s built from those we have on the ship: consistency, prudence, keeping the crew dry, et cetera. ↩
Moral incrementalism is one of my deepest personal convictions. I’d much rather people kicked a puppy than attempted to tip the whole boat. Rocking it, sure, the boat is built to rock, and it often needs to be; just don’t pretend we’re building on dry land. The common law tradition works this way, case by case, with landmark changes permitted but never cheap. ↩
The refit-at-sea image is O. Neurath’s: we are like sailors who must rebuild their ship on the open sea, never able to bring it into dry dock and build it anew from the best materials. O. Neurath, “Protokollsätze,” Erkenntnis, vol. 3, pp. 204-214, 1932-33; W. V. O. Quine took the passage as the epigraph to Word and Object (1960). It is the standard picture of revising a system with no foundation outside it to stand on, which is what an amendment to a moral practice is. ↩
The evolutionary debunking argument: S. Street, “A Darwinian Dilemma for Realist Theories of Value,” Philosophical Studies (2006). ↩
The keystone of this section is B. Williams, “The Human Prejudice,” in Philosophy as a Humanistic Discipline (2006), which argues that partiality to human beings is legitimate and is not the moral analogue of racism, because there is no coherent super-human standpoint from which “human” picks out an arbitrary subgroup the way “white” does. The “one thought too many” verdict is from B. Williams, “Persons, Character and Morality” (1976), reprinted in Moral Luck (1981); his original case is a man who can save his wife or a stranger, which we have moved one generation down. Williams’s own name for the position is the human prejudice, worn deliberately and without apology. ↩
R. Rorty, “Solidarity or Objectivity?” (1985) and Contingency, Irony, and Solidarity (1989): loyalty to the values of a community held as ours, without the pretence that they answer to something beyond us. ↩