liberal

The Butlerian Jihad is looking better all the time

PNRJ book reviews, cognitive science, computer science, core principles, critique of neoclassical economics, inequality, politics, public policy, tribal paradigm , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Mar 24 JDN 2460395

A review of The Age of Em by Robin Hanson

In the Dune series, the Butlerian Jihad was a holy war against artificial intelligence that resulted in a millenias-long taboo against all forms of intelligent machines. It was effectively a way to tell a story about the distant future without basically everything being about robots or cyborgs.

After reading Robin Hanson’s book, I’m starting to think that maybe we should actually do it.

Thus it is written: “Thou shalt not make a machine in the likeness of a human mind.”

Hanson says he’s trying to reserve judgment and present objective predictions without evaluation, but it becomes very clear throughout that this is the future he wants, as well as—or perhaps even instead of—the world he expects.

In many ways, it feels like he has done his very best to imagine a world of true neoclassical rational agents in perfect competition, a sort of sandbox for the toys he’s always wanted to play with. Throughout he very much takes the approach of a neoclassical economist, making heroic assumptions and then following them to their logical conclusions, without ever seriously asking whether those assumptions actually make any sense.

To his credit, Hanson does not buy into the hype that AGI will be successful any day now. He predicts that we will achieve the ability to fully emulate human brains and thus create a sort of black-box AGI that behaves very much like a human within about 100 years. Given how the Blue Brain Project has progressed (much slower than its own hype machine told us it would—and let it be noted that I predicted this from the very beginning), I think this is a fairly plausible time estimate. He refers to a mind emulated in this way as an “em”; I have mixed feelings about the term, but I suppose we did need some word for that, and it certainly has conciseness on its side.

Hanson believes that a true understanding of artificial intelligence will only come later, and the sort of AGI that can be taken apart and reprogrammed for specific goals won’t exist for at least a century after that. Both of these sober, reasonable predictions are deeply refreshing in a field that’s been full of people saying “any day now” for the last fifty years.

But Hanson’s reasonableness just about ends there.

In The Age of Em, government is exactly as strong as Hanson needs it to be. Somehow it simultaneously ensures a low crime rate among a population that doubles every few months while also having no means of preventing that population growth. Somehow ensures that there is no labor collusion and corporations never break the law, but without imposing any regulations that might reduce efficiency in any way.

All of this begins to make more sense when you realize that Hanson’s true goal here is to imagine a world where neoclassical economics is actually true.

He realized it didn’t work on humans, so instead of giving up the theory, he gave up the humans.

Hanson predicts that ems will casually make short-term temporary copies of themselves called “spurs”, designed to perform a particular task and then get erased. I guess maybe he would, but I for one would not so cavalierly create another person and then make their existence dedicated to doing a single job before they die. The fact that I created this person, and they are very much like me, seem like reasons to care more about their well-being, not less! You’re asking me to enslave and murder my own child. (Honestly, the fact that Robin Hanson thinks ems will do this all the time says more about Robin Hanson than anything else.) Any remotely sane society of ems would ban the deletion of another em under any but the most extreme circumstances, and indeed treat it as tantamount to murder.

Hanson predicts that we will only copy the minds of a few hundred people. This is surely true at some point—the technology will take time to develop, and we’ll have to start somewhere. But I don’t see why we’d stop there, when we could continue to copy millions or billions of people; and his choices of who would be emulated, while not wildly implausible, are utterly terrifying.

He predicts that we’d emulate genius scientists and engineers; okay, fair enough, that seems right. I doubt that the benefits of doing so will be as high as many people imagine, because scientific progress actually depends a lot more on the combined efforts of millions of scientists than on rare sparks of brilliance by lone geniuses; but those people are definitely very smart, and having more of them around could be a good thing. I can also see people wanting to do this, and thus investing in making it happen.

He also predicts that we’d emulate billionaires. Now, as a prediction, I have to admit that this is actually fairly plausible; billionaires are precisely the sort of people who are rich enough to pay to be emulated and narcissistic enough to want to. But where Hanson really goes off the deep end here is that he sees this as a good thing. He seems to honestly believe that billionaires are so rich because they are so brilliant and productive. He thinks that a million copies of Elon Musks would produce a million hectobillionaires—when in reality it would produce a million squabbling narcissists, who at best had to split the same $200 billion wealth between them, and might very well end up with less because they squander it.

Hanson has a long section on trying to predict the personalities of ems. Frankly this could just have been dropped entirely; it adds almost nothing to the book, and the book is much too long. But the really striking thing to me about that section is what isn’t there. He goes through a long list of studies that found weak correlations between various personality traits like extroversion or openness and wealth—mostly comparing something like the 20th percentile to the 80th percentile—and then draws sweeping conclusions about what ems will be like, under the assumption that ems are all drawn from people in the 99.99999th percentile. (Yes, upper-middle-class people are, on average, more intelligent and more conscientious than lower-middle-class people. But do we even have any particular reason to think that the personalities of people who make $150,000 are relevant to understanding the behavior of people who make $15 billion?) But he completely glosses over the very strong correlations that specifically apply to people in that very top super-rich class: They’re almost all narcissists and/or psychopaths.

Hanson predicts a world where each em is copied many, many times—millions, billions, even trillions of times, and also in which the very richest ems are capable of buying parallel processing time that lets them accelerate their own thought processes to a million times faster than a normal human. (Is that even possible? Does consciousness work like that? Who knows!?) The world that Hanson is predicting is thus one where all the normal people get outnumbered and overpowered by psychopaths.

Basically this is the most abjectly dystopian cyberpunk hellscape imaginable. And he talks about it the whole time as if it were good.

It’s like he played the game Action Potential and thought, “This sounds great! I’d love to live there!” I mean, why wouldn’t you want to owe a life-debt on your own body and have to work 120-hour weeks for a trillion-dollar corporation just to make the payments on it?

Basically, Hanson doesn’t understand how wealth is actually acquired. He is educated as an economist, yet his understanding of capitalism basically amounts to believing in magic. He thinks that competitive markets just somehow perfectly automatically allocate wealth to whoever is most productive, and thus concludes that whoever is wealthy now must just be that productive.

I can see no other way to explain his wildly implausible predictions that the em economy will double every month or two. A huge swath of the book depends upon this assumption, but he waits until halfway through the book to even try to defend it, and then does an astonishingly bad job of doing so. (Honestly, even if you buy his own arguments—which I don’t—they seem to predict that population would grow with Moore’s Law—doubling every couple of years, not every couple of months.)

Whereas Keynes predicted based on sound economic principles that economic growth would more or less proceed apace and got his answer spot-on, Hanson predicts that for mysterious, unexplained reasons economic growth will suddenly increase by two orders of magnitude—and I’m pretty sure he’s going to be wildly wrong.

Hanson also predicts that ems will be on average poorer than we are, based on some sort of perfect-competition argument that doesn’t actually seem to mesh at all with his predictions of spectacularly rapid economic and technological growth. I think the best way to make sense of this is to assume that it means the trend toward insecure affluence will continue: Ems will have an objectively high standard of living in terms of what they own, what games they play, where they travel, and what they eat and drink (in simulation), but they will constantly be struggling to keep up with the rent on their homes—or even their own bodies. This is a world where (the very finest simulation of) Dom Perignon is $7 a bottle and wages are $980 an hour—but monthly rent is $284,000.

Early in the book Hanson argues that this life of poverty and scarcity will lead to more conservative values, on the grounds that people who are poorer now seem to be more conservative, and this has something to do with farmers versus foragers. Hanson’s explanation of all this is baffling; I will quote it at length, just so it’s clear I’m not misrepresenting it:

The other main (and independent) axis of value variation ranges between poor and rich societies. Poor societies place more value on conformity, security, and traditional values such as marriage, heterosexuality, religion, patriotism, hard work, and trust in authority. In contrast, rich societies place more value on individualism, self-direction, tolerance, pleasure, nature, leisure, and trust. When the values of individuals within a society vary on the same axis, we call this a left/liberal (rich) versus right/conservative (poor) axis.

Foragers tend to have values more like those of rich/liberal people today, while subsistence farmers tend to have values more like those of poor/conservative people today. As industry has made us richer, we have on average moved from conservative/farmer values to liberal/forager values. This value movement can make sense if cultural evolution used the social pressures farmers faced, such as conformity and religion, to induce humans, who evolved to find forager behaviors natural, to instead act like farmers. As we become rich, we don’t as strongly fear the threats behind these social pressures. This connection may result in part from disease; rich people are healthier, and healthier societies fear less.

The alternate theory that we have instead learned that rich forager values are more true predicts that values should have followed a random walk over time, and be mostly common across space. It also predicts the variance of value changes tracking the rate at which relevant information appears. But in fact industrial-era value changes have tracked the wealth of each society in much more steady and consistent fashion. And on this theory, why did foragers ever acquire farmer values?

[…]

In the scenario described in this book, many strange-to-forager behaviors are required, and median per-person (i.e. per-em) incomes return to near-subsistence levels. This suggests that the em era may reverse the recent forager-like trend toward more liberality; ems may have more farmer-like values.

The Age of Em, p. 26-27

There’s a lot to unpack here, but maybe it’s better to burn the whole suitcase.

First of all, it’s not entirely clear that this is really a single axis of variation, that foragers and farmers differ from each other in the same way as liberals and conservatives. There’s some truth to that at least—both foragers and liberals tend to be more generous, both farmers and conservatives tend to enforce stricter gender norms. But there are also clear ways that liberal values radically deviate from forager values: Forager societies are extremely xenophobic, and typically very hostile to innovation, inequality, or any attempts at self-aggrandizement (a phenomenon called “fierce egalitarianism“). San Francisco epitomizes rich, liberal values, but it would be utterly alien and probably regarded as evil by anyone from the Yanomamo.

Second, there is absolutely no reason to predict any kind of random walk. That’s just nonsense. Would you predict that scientific knowledge is a random walk, with each new era’s knowledge just a random deviation from the last’s? Maybe next century we’ll return to geocentrism, or phrenology will be back in vogue? On the theory that liberal values (or at least some liberal values) are objectively correct, we would expect them to advance as knowledge does—improving over time, and improving faster in places that have better institutions for research, education, and free expression. And indeed, this is precisely the pattern we have observed. (Those places are also richer, but that isn’t terribly surprising either!)

Third, while poorer regions are indeed more conservative, poorer people within a region actually tend to be more liberal. Nigeria is poorer and more conservative than Norway, and Mississippi is poorer and more conservative than Massachusetts. But higher-income households in the United States are more likely to vote Republican. I think this is particularly true of people living under insecure affluence: We see the abundance of wealth around us, and don’t understand why we can’t learn to share it better. We’re tired of fighting over scraps while the billionaires claim more and more. Millennials and Zoomers absolutely epitomize insecure affluence, and we also absolutely epitomize liberalism. So, if indeed ems live a life of insecure affluence, we should expect them to be like Zoomers: “Trans liberation now!” and “Eat the rich!” (Or should I say, “Delete the rich!”)

And really, doesn’t that make more sense? Isn’t that the trend our society has been on, for at least the last century? We’ve been moving toward more and more acceptance of women and minorities, more and more deviation from norms, more and more concern for individual rights and autonomy, more and more resistance to authority and inequality.

The funny thing is, that world sounds a lot better than the one Hanson is predicting.

A world of left-wing ems would probably run things a lot better than Hanson imagines: Instead of copying the same hundred psychopaths over and over until we fill the planet, have no room for anything else, and all struggle to make enough money just to stay alive, we could moderate our population to a more sustainable level, preserve diversity and individuality, and work toward living in greater harmony with each other and the natural world. We could take this economic and technological abundance and share it and enjoy it, instead of killing ourselves and each other to make more of it for no apparent reason.

The one good argument Hanson makes here is expressed in a single sentence: “And on this theory, why did foragers ever acquire farmer values?” That actually is a good question; why did we give up on leisure and egalitarianism when we transitioned from foraging to agriculture?

I think scarcity probably is relevant here: As food became scarcer, maybe because of climate change, people were forced into an agricultural lifestyle just to have enough to eat. Early agricultural societies were also typically authoritarian and violent. Under those conditions, people couldn’t be so generous and open-minded; they were surrounded by threats and on the verge of starvation.

I guess if Hanson is right that the em world is also one of poverty and insecurity, we might go back to those sort of values, borne of desperation. But I don’t see any reason to think we’d give up all of our liberal values. I would predict that ems will still be feminist, for instance; in fact, Hanson himself admits that since VR avatars would let us change gender presentation at will, gender would almost certainly become more fluid in a world of ems. Far from valuing heterosexuality more highly (as conservatives do, a “farmer value” according to Hanson), I suspect that ems will have no further use for that construct, because reproduction will be done by manufacturing, not sex, and it’ll be so easy to swap your body into a different one that hardly anyone will even keep the same gender their whole life. They’ll think it’s quaint that we used to identify so strongly with our own animal sexual dimorphism.

But maybe it is true that the scarcity induced by a hyper-competitive em world would make people more selfish, less generous, less trusting, more obsessed with work. Then let’s not do that! We don’t have to build that world! This isn’t a foregone conclusion!

There are many other paths yet available to us.

Indeed, perhaps the simplest would be to just ban artificial intelligence, at least until we can get a better handle on what we’re doing—and perhaps until we can institute the kind of radical economic changes necessary to wrest control of the world away from the handful of psychopaths currently trying their best to run it into the ground.

I admit, it would kind of suck to not get any of the benefits of AI, like self-driving cars, safer airplanes, faster medical research, more efficient industry, and better video games. It would especially suck if we did go full-on Butlerian Jihad and ban anything more complicated than a pocket calculator. (Our lifestyle might have to go back to what it was in—gasp! The 1950s!)

But I don’t think it would suck nearly as much as the world Robin Hanson thinks is in store for us if we continue on our current path.

So I certainly hope he’s wrong about all this.

Fortunately, I think he probably is.

The scissors of supply and demand

PNRJ core principles, public policy , , , , , , , , , , ,

JDN 2457299 EDT 17:03

In recent posts I talked about demand and then I talked about supply. Now it’s time to talk about both at once–which is where the real magic happens. Alfred Marshall famously compared supply and demand to the lower and upper blades of a pair of scissors:

We might as reasonably dispute whether it is the upper or the under blade of a pair of scissors that cuts a piece of paper, as whether value is governed by utility or cost of production. It is true that when one blade is held still, and the cutting is effected by moving the other, we may say with careless brevity that the cutting is done by the second; but the statement is not strictly accurate, and is to be excused only so long as it claims to be merely a popular and not a strictly scientific account of what happens.

~Alfred Marshall, Principles of Economics

Before Marshall, it was actually rather common to debate whether prices are determined by supply or by demand. Actually there seems to be a certain branch of Marxists today who insist upon the “labor theory of value” that seems to rest upon a similar sort of confusion, basically saying that the real value of something is entirely determined by its cost of supply. If the value of something were strictly determined by the labor put into making it, there would be literally no reason to ever make anything. If the value you get from a good is precisely equal to the labor put into it, there is no net benefit to ever making any goods. At most, embodying labor in a product might allow you to transfer labor from one person to another; but there would be no such thing as real economic growth. In order to have real economic growth, products must end up being worth more than what it cost to make them—that is, their value of demand must exceed their cost of supply.

Toward the other end of the political spectrum, we have “Say’s Law”, which says that “supply creates its own demand”; that is, that there is never any such thing as too much or too little overall demand in an economy, because supplying a good automatically makes that good available to trade for something else. I hate to even call it a “law” because isn’t even like the Pirate Code; it’s not even useful as a guideline, it’s just flat wrong. There is absolutely no reason that making something would make someone else want to buy it from you. You can make all sorts of things that nobody wants to buy; the possibilities are endless, really. Balls of lint dusted with powdered sugar, broken ballpoint pens dipped in motor oil, burnt-out lightbulbs covered in melted Swiss cheese. It’s possible that someone might want to buy such bizarre items (call them “postmodernist found art” or something), but there clearly isn’t a large market for such goods, even if you should decide to manufacture thousands of them. Even in an aggregate sense, there’s also no particular reason to think that we can’t have an economy where millions of products pile up on shelves because no one can afford to buy them; indeed, that’s basically what happens in a recession.

In fact, the converse, “demand creates its own supply”, is considerably closer to true. It’s still not strictly true—centuries of searching for the elixir of immortality have failed to produce it, though modern genetic engineering just might finally succeed where all else has failed. (After all, every new technology is impossible… until it isn’t.) But in the long run, this converse law (it doesn’t have a name so far as I know) does contain an important grain of truth: If people want something badly enough, they will spend enormous resources in order to find a way to get it. If you know that a lot of people want something that no one is supplying, it behooves you to find a way to provide it—it might just make you a billionaire. Over centuries of technological advancement, humanity has found ways to provide many goods and services that were previously thought impossible, and one of the central benefits of a capitalist economy is that it provides powerful economic incentives for entrepreneurs to innovate and find ways to provide goods that people have always wanted but never had. Yet, even so, it isn’t true that demand creates its own supply—certainly not in the short run.

Neither supply or demand on its own does much of anything. You can have insatiable demand for something nobody can supply (the aforementioned elixir of immortality), and it still won’t be sold. You can have endless supply of something nobody demands (vacuum?), and it will remain worthless. It’s only when you have both supply and demand that a market becomes possible.

One of the central insights of modern economics is that prices and quantities in a capitalist market are determined simultaneously by supply and demand. In general, both supply and demand are constantly changing in response to events in the world, and thus the prices and quantities of goods shift from one equilibrium to another. In order to predict exactly how they will shift, we would need to know how both supply and demand have changed.

As Marshall alludes to in the above quotation, in some cases we can take either supply or demand as fixed and then the other one is what matters; but these are only special cases. In general, both supply and demand are subject to the winds of changing markets, and we need to keep track of both at once. If that sounds really difficult, that’s because it is—most of what economists do in the real world ultimately amounts to finding ways to distinguish supply effects from demand effects in various situations. Even most statistical methods in econometrics were basically designed as means of separating out demand-related causes from supply-related causes.

A lot of policy questions ultimately depend upon whether supply or demand is the dominant factor: If the business cycle is primarily driven by changes in demand, it makes sense to use monetary and fiscal policy to stabilize the economy (short version: it is, and it does). If it were instead driven by supply (“supply-side economics”), it would instead be better to make structural changes that reduce costs of production. (Why is this obviously wrong? Because there weren’t sudden increases in production costs in 2008—but there was a sudden collapse of consumer buying power. Maybe the 1973 recession can be explained by a sudden increase in oil prices, but there was no such supply shock in 2008.) If the labor market is primarily driven by demand, we need to find ways to get business to hire more people; but if it’s primarily driven by supply, we need to find ways to get people to get off their butts and try to find work. (Again, I think it’s pretty obvious that the former is true, not the latter—since at least 2000 there have never been as many job openings in the US as there were unemployed people.)

In the above policy questions the liberal view is the demand-side and the conservative view is the supply-side, but that need not be the case. Regarding renewable energy, for example, the more liberal view is that lots of people would want to buy electric cars and solar panels, if they were made available, but they aren’t—we are supply-constrained. The more conservative view is that the reason they aren’t selling more is that nobody particularly wants them and trying to force them on us is a fool’s errand—we are demand-constrained. Likewise when it comes to banking, liberals generally think that the reason there isn’t more credit is that banks refuse to supply loans, while conservatives (particularly from the banks themselves) usually argue that it’s because people aren’t willing to take the risk of taking out more loans.

The point, however, is that a lot of policy debates ultimately hinge upon the question of whether demand or supply is more important in driving a particular market—and since sometimes they are both important, sometimes the policy solution requires a combination of different approaches. One of the advantages of quantitative economic analysis is that we can determine exactly how much the costs and benefits of each policy option will be, and thereby choose the one that is most cost-effective.

In this way, “supply or demand?” is a lot like “nature or nurture?”; the answer is always “both”, but there are times when one factor or the other is more important for the policy question at hand.