motte and bailey

The trouble with Rationalism and Effective Altruism

PNRJ cognitive science, core principles, ethics, futurism, heuristics and biases, tribal paradigm , , , , , , , , , , , , , , , , , , , , ,

Oct 11 JDN 2461325

I have discussed before the notion of a motte and bailey, in which uses ambiguity to present a view in two different forms: A motte, which is highly defensible but often banal or uncontroversial, and a bailey, which is highly interesting and radical but ultimately indefensible. Like the medieval strategy, you want to live in the bailey, but you can only defend the motte.

I think the “rationalist” and “effective altruism” communities suffer quite severely from this dynamic.

I use the scare quotes precisely because many so-called “rationalists” deeply fail to conform to the principles of rationalism, and many so-called “effective altruists” don’t seem particularly altruistic.

This saddens me greatly, because the core principles of rationalism and effective altruism are not simply correct: They are among the best ideas that have ever been formulated by human minds.

The core of rationalism is this:

Reason is the best source of knowledge and truth.

The core of effective altruism is this:

The best acts are those that do the most good for the least cost.

Both of these ideas seem beyond defensible—more like unassailable, perhaps even undeniable, as in literally impossible to coherently deny. They have the ring of not simply truth, but deep truth, perhaps even capital-T Truth, fundamental laws that have the power and certainty of logical or mathematical theorems.

Moreover, they are not banal or uncontroversial! It is quite bizarre that they are not; it speaks of deep irrationality within human brains and human societies that anyone would even try to deny these things. Yet the very existence of religion as we know it proves that rationalism is not uncontroversial, and many people still cling to moral systems that condemn people for harmless acts or create needless suffering.

This is the motte, and it is a fine motte indeed: Beautiful, imposing, perfectly-engineered, and utterly impregnable. The closest real-world equivalent is something like the walls of Troy, which were (as the story goes) invincible against conventional force and could only be defeated by deception. (Or perhaps if you consider all of Japan a fortress: Conquering that island nation from the outside required air power and nuclear weapons.)

I love this motte will all my heart, but I am deeply concerned about the bailey.

What is the bailey of rationalism and effective altruism?

First, let me address the parts that I am not too concerned about; I think many people go a bit too far with them, but I believe they are basically sound:

  • Bayesianism: The best way to reason about uncertainty is using Bayesian statistics.
  • Utilitarianism: Morality consists in doing the greatest good for the greatest number.
  • Expected utility: The best way to make decisions under uncertainty is to maximize expected utility.
  • Behavioral economics: Economic behavior is best understood by recognizing that human beings use heuristics and are subject to biases that prevent them from being fully rational.
  • Longtermism: One of our most important moral concerns is what kind of world we are making for future generations of humanity.

As I have stated them, I consider these views to be basically sound. Now, let me rephrase them in ways that they are often taken too far:

  • McNamara fallacy: Only what you can mathematically quantify matters, and if you are not explicitly computing a posterior distribution from a prior distribution and a likelihood, you are not making a rational judgment under uncertainty.
  • Act utilitarianism: Morality consists in taking each action in isolation and maximizing the utility it creates, without regard for the deep uncertainty and complex ramifications involved in doing so.
  • Shut up and multiply: Regardless of whether it violates common sense, rests upon flimsy assumptions, or involves deep uncertainty and unacceptable risk, the proper thing to do when making decisions is always to go with what you calculate yields the highest expected utility.
  • Fallacy fallacy: If your opponent is reasoning in a fallacious way, you are free to completely dismiss their beliefs.
  • Utopian longtermism: As long as you can convince yourself that what you are doing will provide some miniscule increase in the probability of a utopian future for our galaxy-colonizing descendants, you are free to ignore the consequences of your actions for people who are alive today.

These are already a big problem, especially the last one. I have seen far too many “rationalists” and “effective altruists” pivot away from projects that could have done enormous good today with a high degree of certainty (such as improving economic policy, or supporting environmental causes, or donating to medical research or malaria prevention) in favor of trying to advance AI (often in the name of “AI safety”, but for some reason never actually involving regulating anything) toward some pie-in-the-sky vision of a utopian future.

In particular, I am deeply concerned about people who attach huge payoffs to tiny probabilities, and then convince themselves that “expected utility” means they must care about those things more than anything else. This is essentially a secular version of Pascal’s Wager, and it can drive people to terrible, terrible things.

It is far better for you to do small good things that are certain. Yes, we should care about people thousands of years from now; but the truth is, we really have no idea what the future will be like thousands of years from now. So how about we do our best to help the people who are here, now?

If you don’t find this argument convincing, consider this:

Helping people now can push us toward a brighter future.

Suppose, for the sake of argument, that we are truly most concerned about what happens thousands of years from now when humans have colonized the galaxy. What sort of society would you want to be responsible for that colonization?

Would it be one where people used esoteric and highly-uncertain mathematical arguments to convince themselves that they should abuse and exploit other people in the name of some unprovable “greater good”?

Or would it be one where people helped each other, locally and globally, showed compassion for all living things, and sought to help everyone achieve their full potential?

I contend that spreading kindness and compassion might very well be the very best thing you could do to make humanity’s future brighter. It will not only increase the probability that humanity survives long enough to colonize the galaxy, but also increase the probability that the society which does so is a good society, made of people who are kind and compassionate.

Sure, it wouldn’t hurt to donate to, or work for, AI safety or advocacy for AI regulation. I’m not saying you shouldn’t do those things. But they are not the one most important thing—and indeed, anyone who tells you that something is the one most important thing is basically trying to hack your hedonic mind.

But there is more to the bailey here, and it gets worse.

The majority of people who identify as rationalists and/or effective altruists also seem to believe the following things:

  • Libertarianism: Government is always the problem, never the solution. We should always trust in free markets rather that democratic governments.
  • Prediction markets: Rather than trust experts or even admit that some things are fundamentally unpredictable, we should base our decisions on the results of prediction markets, and we should have prediction markets for virtually everything.
  • AI-doomism: The end of the world is nigh, and this is a terrible thing; any day now, artificial general intelligence will take over the world and wipe out humanity.
  • Singularitarianism: The end of the world as we know it is nigh, and this is a good thing; any day now, artificial general intelligence will take over the world and lead us into a glorious utopian future.

This excellent and eye-opening video on the conference Manifest does a nice job of expressing just how weird and dangerous the culture around these beliefs can become. (And while they are much rarer, there have absolutely been people who tried to adopt rationalist ideas and became literal cults.)

Of course AI-doomism and Singularitarianism are contradictory, but in an odd sort of way: Both sides agree that AGI is imminent and will change everything; they simply disagree about whether this will turn out very good or very bad. As a result, many people who seem to assign a higher-than-normal probability of AI doom also assign a higher-than-normal probability of Singularity. This is not incoherent (as long as all their probabilities add to 1, of course); it simply means that they assign a high probability to AGI being imminent and world-changing, and aren’t sure whether that will turn out good or bad.

(I was once cited as a “Singularitarian blogger”, for this post; I am deeply flattered to be thus cited, but I really don’t think of myself as a Singularitarian. There is a weak sense of Singularitarianism that I guess I agree with: Artificial intelligence will, sooner than most people think, effect radical changes upon human society, and this could be a very good thing if it is handled correctly. But I do not believe we are on the cusp of the Culture.)

The weirdest part, for me at least, is how libertarianism and AI-doomism collide: People will tell me that they believe that AI has a greater than 10%—some say 50% or even 90%—chance of imminently wiping out the human race, and yet they reject any attempt to ban or even significantly regulate AI.

I believe that there is a less than 1% chance of AI wiping out the human race, and yet I believe that we urgently need severe regulations on AI that would substantially slow its development and limit its capabilities. (I think Dynomight makes the best argument for this risk being genuine, cutting through a lot of irrelevant nonsense. As it is, I actually think IQ-300 aliens would likely be better than AGI, because a species of genuinely-intelligent beings (as opposed to computer software that simulates but probably doesn’t actually exhibit intelligence) that is advanced enough to achieve interstellar travel is likely to be peaceful and cooperative enough that they would do us more good than harm. Also, Dynomight’s timeline of “the next few decades” is a lot more plausible than the panicky doomers who say “the next few years”.)

In part I want to regulate AI more because I can already see ways that AI is causing harm (this Jacobin article provides a nice summary), but even without those things, a 1% chance of total annihilation is well worth a great deal of regulation and slowed innovation. Indeed, let me get on record saying that reducing the chances of human annihilation by one percentage point would be well worth slowing economic and technological development by a century.

Libertarianism and prediction markets of course go quite well together, and indeed I rarely encounter one without the other. But this faith in the market is deeply misplaced—especially if we believe, as I do, in behavioral economics!—and in particular, prediction markets are in fact fundamentally flawed as a means of making decisions because of how they handle conditional probabilities. It’s actually kind of a corollary of Goodhart’s Law: Once a measure becomes a target, it ceases to be a good measure, because rational agents try to game the system. Prediction markets can work very well if the outcome is totally beyond human control and wouldn’t totally render the outcome moot (e.g. an earthquake); but once human decisions can affect the result, the prediction market effectively becomes insider trading, and for really big things (like existential risk), the outcome itself could easily render the prediction market—or even money itself—utterly irrelevant.

We must defend the motte with all our might—but perhaps it’s best to abandon some of this bailey.

Yes, by all means, abandon religion, learn to think more rationally, support AI safety, and do what you can to do the most good for the least cost. Those things are good, and right, and important.

But if you find yourself thinking that you should sacrifice yourself, or abandon ordinary concepts of morality, in the name of some ultimate good that will save us from imminent destruction, you need to take a step back, and re-immerse yourself in the rest of society. You are not going to single-handedly save the world; nobody is. If the world is to be saved, we will all do it together.

The Efficient Roulette Hypothesis

PNRJ core principles, critique of neoclassical economics, financial advice, market failure, public policy , , , , , , ,

Nov 27 JDN 2459911

The efficient market hypothesis is often stated in several different ways, and these are often treated as equivalent. There are at least three very different definitions of it that people seem to use interchangeably:

  1. Market prices are optimal and efficient.
  2. Market prices aggregate and reflect all publicly-available relevant information.
  3. Market prices are difficult or impossible to predict.

The first reading, I will call the efficiency hypothesis, because, well, it is what we would expect a phrase like “efficient market hypothesis” to mean. The ordinary meaning of those words would imply that we are asserting that market prices are in some way optimal or near-optimal, that markets get prices “right” in some sense at least the vast majority of the time.

The second reading I’ll call the information hypothesis; it implies that market prices are an information aggregation mechanism which automatically incorporates all publicly-available information. This already seems quite different from efficiency, but it seems at least tangentially related, since information aggregation could be one useful function that markets serve.

The third reading I will call the unpredictability hypothesis; it says simply that market prices are very difficult to predict, and so you can’t reasonably expect to make money by anticipating market price changes far in advance of everyone else. But as I’ll get to in more detail shortly, that doesn’t have the slightest thing to do with efficiency.

The empirical data in favor of the unpredictability hypothesis is quite overwhelming. It’s exceedingly hard to beat the market, and for most people, most of the time, the smartest way to invest is just to buy a diversified portfolio and let it sit.

The empirical data in favor of the information hypothesis is mixed, but it’s at least plausible; most prices do seem to respond to public announcements of information in ways we would expect, and prediction markets can be surprisingly accurate at forecasting the future.

The empirical data in favor of the efficiency hypothesis, on the other hand, is basically nonexistent. On the one hand this is a difficult hypothesis to test directly, since it isn’t clear what sort of benchmark we should be comparing against—so it risks being not even wrong. But if you consider basically any plausible standard one could try to set for how an efficient market would run, our actual financial markets in no way resemble it. They are erratic, jumping up and down for stupid reasons or no reason at all. They are prone to bubbles, wildly overvaluing worthless assets. They have collapsed governments and ruined millions of lives without cause. They have resulted in the highest-paying people in the world doing jobs that accomplish basically nothing of genuine value. They are, in short, a paradigmatic example of what inefficiency looks like.

Yet, we still have economists who insist that “the efficient market hypothesis” is a proven fact, because the unpredictability hypothesis is clearly correct.

I do not think this is an accident. It’s not a mistake, or an awkwardly-chosen technical term that people are misinterpreting.

This is a motte and bailey doctrine.

Motte-and-bailey was a strategy in medieval warfare. Defending an entire region is very difficult, so instead what was often done was constructing a small, highly defensible fortification—the motte—while accepting that the land surrounding it—the bailey—would not be well-defended. Most of the time, the people stayed on the bailey, where the land was fertile and it was relatively pleasant to live. But should they be attacked, they could retreat to the motte and defend themselves until the danger was defeated.

A motte-and-bailey doctrine is an analogous strategy used in argumentation. You use the same words for two different versions of an idea: The motte is a narrow, defensible core of your idea that you can provide strong evidence for, but it isn’t very strong and may not even be interesting or controversial. The bailey is a broad, expansive version of your idea that is interesting and controversial and leads to lots of significant conclusions, but can’t be well-supported by evidence.

The bailey is the efficiency hypothesis: That market prices are optimal and we are fools to try to intervene or even regulate them because the almighty Invisible Hand is superior to us.

The motte is the unpredictability hypothesis: Market prices are very hard to predict, and most people who try to make money by beating the market fail.

By referring to both of these very different ideas as “the efficient market hypothesis”, economists can act as if they are defending the bailey, and prescribe policies that deregulate financial markets on the grounds that they are so optimal and efficient; but then when pressed for evidence to support their beliefs, they can pivot to the motte, and merely show that markets are unpredictable. As long as people don’t catch on and recognize that these are two very different meanings of “the efficient market hypothesis”, then they can use the evidence for unpredictability to support their goal of deregulation.

Yet when you look closely at this argument, it collapses. Unpredictability is not evidence of efficiency; if anything, it’s the opposite. Since the world doesn’t really change on a minute-by-minute basis, an efficient system should actually be relatively predictable in the short term. If prices reflected the real value of companies, they would change only very gradually, as the fortunes of the company change as a result of real-world events. An earthquake or a discovery of a new mine would change stock prices in relevant industries; but most of the time, they’d be basically flat. The occurrence of minute-by-minute or even second-by-second changes in prices basically proves that we are not tracking any genuine changes in value.

Roulette wheels are extremely unpredictable by design—by law, even—and yet no one would accuse them of being an efficient way of allocating resources. If you bet on roulette wheels and try to beat the house, you will almost surely fail, just as you would if you try to beat the stock market—and dare I say, for much the same reasons?

So if we’re going to insist that “efficiency” just means unpredictability, rather than actual, you know, efficiency, then we should all speak of the Efficient Roulette Hypothesis. Anything we can’t predict is now automatically “efficient” and should therefore be left unregulated.

The “market for love” is a bad metaphor

PNRJ core principles, critique of neoclassical economics , , , , , , , , , , , , , , , , ,

Feb 14 JDN 2458529

Valentine’s Day was this past week, so let’s talk a bit about love.

Economists would never be accused of being excessively romantic. To most neoclassical economists, just about everything is a market transaction. Love is no exception.

There are all sorts of articles and books and an even larger number of research papers going back multiple decades and continuing all the way through until today using the metaphor of the “marriage market”.

In a few places, marriage does actually function something like a market: In China, there are places where your parents will hire brokers and matchmakers to select a spouse for you. But even this isn’t really a market for love or marriage. It’s a market for matchmaking services. The high-tech version of this is dating sites like OkCupid.
And of course sex work actually occurs on markets; there is buying and selling of services at monetary prices. There is of course a great deal worth saying on that subject, but it’s not my topic for today.

But in general, love is really nothing like a market. First of all, there is no price. This alone should be sufficient reason to say that we’re not actually dealing with a market. The whole mechanism that makes a market a market is the use of prices to achieve equilibrium between supply and demand.

A price doesn’t necessarily have to be monetary; you can barter apples for bananas, or trade in one used video game for another, and we can still legitimately call that a market transaction with a price.

But love isn’t like that either. If your relationship with someone is so transactional that you’re actually keeping a ledger of each thing they do for you and each thing you do for them so that you could compute a price for services, that isn’t love. It’s not even friendship. If you really care about someone, you set such calculations aside. You view their interests and yours as in some sense shared, aligned toward common goals. You stop thinking in terms of “me” and “you” and start thinking in terms of “us”. You don’t think “I’ll scratch your back if you scratch mine.” You think “We’re scratching each other’s backs today.”

This is of course not to say that love never involves conflict. On the contrary, love always involves conflict. Successful relationships aren’t those where conflict never happens, they are those where conflict is effectively and responsibly resolved. Your interests and your loved ones’ are never completely aligned; there will always be some residual disagreement. But the key is to realize that your interests are still mostly aligned; those small vectors of disagreement should be outweighed by the much larger vector of your relationship.

And of course, there can come a time when that is no longer the case. Obviously, there is domestic abuse, which should absolutely be a deal-breaker for anyone. But there are other reasons why you may find that a relationship ultimately isn’t working, that your interests just aren’t as aligned as you thought they were. Eventually those disagreement vectors just get too large to cancel out. This is painful, but unavoidable. But if you reach the point where you are keeping track of actions on a ledger, that relationship is already dead. Sooner or later, someone is going to have to pull the plug.

Very little of what I’ve said in the preceding paragraphs is likely to be controversial. Why, then, would economists think that it makes sense to treat love as a market?

I think this comes down to a motte and bailey doctrine. A more detailed explanation can be found at that link, but the basic idea of a motte and bailey is this: You have a core set of propositions that is highly defensible but not that interesting (the “motte”), and a broader set of propositions that are very interesting, but not as defensible (the “bailey”). The terms are related to a medieval defensive strategy, in which there was a small, heavily fortified tower called a motte, surrounded by fertile, useful land, the bailey. The bailey is where you actually want to live, but it’s hard to defend; so if the need arises, you can pull everyone back into the motte to fight off attacks. But nobody wants to live in the motte; it’s just a cramped stone tower. There’s nothing to eat or enjoy there.

The motte comprised of ideas that almost everyone agrees with. The bailey is the real point of contention, the thing you are trying to argue for—which, by construction, other people must not already agree with.

Here are some examples, which I have intentionally chosen from groups I agree with:

Feminism can be a motte and bailey doctrine. The motte is “women are people”; the bailey is abortion rights, affirmative consent and equal pay legislation.

Rationalism can be a motte and bailey doctrine. The motte is “rationality is good”; the bailey is atheism, transhumanism, and Bayesian statistics.

Anti-fascism can be a motte and bailey doctrine. The motte is “fascists are bad”; the bailey is black bloc Antifa and punching Nazis.

Even democracy can be a motte and bailey doctrine. The motte is “people should vote for their leaders”; my personal bailey is abolition of the Electoral College, a younger voting age, and range voting.

Using a motte and bailey doctrine does not necessarily make you wrong. But it’s something to be careful about, because as a strategy it can be disingenuous. Even if you think that the propositions in the bailey all follow logically from the propositions in the motte, the people you’re talking to may not think so, and in fact you could simply be wrong. At the very least, you should be taking the time to explain how one follows from the other; and really, you should consider whether the connection is actually as tight as you thought, or if perhaps one can believe that rationality is good without being Bayesian or believe that women are people without supporting abortion rights.

I think when economists describe love or marriage as a “market”, they are applying a motte and bailey doctrine. They may actually be doing something even worse than that, by equivocating on the meaning of “market”. But even if any given economist uses the word “market” totally consistently, the fact that different economists of the same broad political alignment use the word differently adds up to a motte and bailey doctrine.

The doctrine is this: “There have always been markets.”

The motte is something like this: “Humans have always engaged in interaction for mutual benefit.”

This is undeniably true. In fact, it’s not even uninteresting. As mottes go, it’s a pretty nice one; it’s worth spending some time there. In the endless quest for an elusive “human nature”, I think you could do worse than to focus on our universal tendency to engage in interaction for mutual benefit. (Don’t other species do it too? Yes, but that’s just it—they are precisely the ones that seem most human.)

And if you want to define any mutually-beneficial interaction as a “market trade”, I guess it’s your right to do that. I think this is foolish and confusing, but legislating language has always been a fool’s errand.

But of course the more standard meaning of the word “market” implies buyers and sellers exchanging goods and services for monetary prices. You can extend it a little to include bartering, various forms of financial intermediation, and the like; but basically you’re still buying and selling.

That makes this the bailey: “Humans have always engaged in buying and selling of goods and services at prices.”

And that, dear readers, is ahistorical nonsense. We’ve only been using money for a few thousand years, and it wasn’t until the Industrial Revolution that we actually started getting the majority of our goods and services via market trades. Economists like to tell a story where bartering preceded the invention of money, but there’s basically no evidence of that. Bartering seems to be what people do when they know how money works but don’t have any money to work with.

Before there was money, there were fundamentally different modes of interaction: Sharing, ritual, debts of honor, common property, and, yes, love.

These were not markets. They perhaps shared some very broad features of markets—such as the interaction for mutual benefit—but they lacked the defining attributes that make a market a market.

Why is this important? Because this doctrine is used to transform more and more of our lives into actual markets, on the grounds that they were already “markets”, and we’re just using “more efficient” kinds of markets. But in fact what’s happening is we are trading one fundamental mode of human interaction for another: Where we used to rely upon norms or trust or mutual affection, we instead rely upon buying and selling at prices.

In some cases, this actually is a good thing: Markets can be very powerful, and are often our best tool when we really need something done. In particular, it’s clear at this point that norms and trust are not sufficient to protect us against climate change. All the “Reduce, Reuse, Recycle” PSAs in the world won’t do as much as a carbon tax. When millions of lives are at stake, we can’t trust people to do the right thing; we need to twist their arms however we can.

But markets are in some sense a brute-force last-resort solution; they commodify and alienate (Marx wasn’t wrong about that), and despite our greatly elevated standard of living, the alienation and competitive pressure of markets seem to be keeping most of us from really achieving happiness.

This is why it’s extremely dangerous to talk about a “market for love”. Love is perhaps the last bastion of our lives that has not been commodified into a true market, and if it goes, we’ll have nothing left. If sexual relationships built on mutual affection were to disappear in favor of apps that will summon a prostitute or a sex robot at the push of a button, I would count that as a great loss for human civilization. (How we should regulate prostitution or sex robots are a different question, which I said I’d leave aside for this post.) A “market for love” is in fact a world with no love at all.