Efficient Market Hypothesis

The Efficient Roulette Hypothesis

pnrj core principles, critique of neoclassical economics, financial advice, market failure, public policy , , , , , , ,

Nov 27 JDN 2459911

The efficient market hypothesis is often stated in several different ways, and these are often treated as equivalent. There are at least three very different definitions of it that people seem to use interchangeably:

  1. Market prices are optimal and efficient.
  2. Market prices aggregate and reflect all publicly-available relevant information.
  3. Market prices are difficult or impossible to predict.

The first reading, I will call the efficiency hypothesis, because, well, it is what we would expect a phrase like “efficient market hypothesis” to mean. The ordinary meaning of those words would imply that we are asserting that market prices are in some way optimal or near-optimal, that markets get prices “right” in some sense at least the vast majority of the time.

The second reading I’ll call the information hypothesis; it implies that market prices are an information aggregation mechanism which automatically incorporates all publicly-available information. This already seems quite different from efficiency, but it seems at least tangentially related, since information aggregation could be one useful function that markets serve.

The third reading I will call the unpredictability hypothesis; it says simply that market prices are very difficult to predict, and so you can’t reasonably expect to make money by anticipating market price changes far in advance of everyone else. But as I’ll get to in more detail shortly, that doesn’t have the slightest thing to do with efficiency.

The empirical data in favor of the unpredictability hypothesis is quite overwhelming. It’s exceedingly hard to beat the market, and for most people, most of the time, the smartest way to invest is just to buy a diversified portfolio and let it sit.

The empirical data in favor of the information hypothesis is mixed, but it’s at least plausible; most prices do seem to respond to public announcements of information in ways we would expect, and prediction markets can be surprisingly accurate at forecasting the future.

The empirical data in favor of the efficiency hypothesis, on the other hand, is basically nonexistent. On the one hand this is a difficult hypothesis to test directly, since it isn’t clear what sort of benchmark we should be comparing against—so it risks being not even wrong. But if you consider basically any plausible standard one could try to set for how an efficient market would run, our actual financial markets in no way resemble it. They are erratic, jumping up and down for stupid reasons or no reason at all. They are prone to bubbles, wildly overvaluing worthless assets. They have collapsed governments and ruined millions of lives without cause. They have resulted in the highest-paying people in the world doing jobs that accomplish basically nothing of genuine value. They are, in short, a paradigmatic example of what inefficiency looks like.

Yet, we still have economists who insist that “the efficient market hypothesis” is a proven fact, because the unpredictability hypothesis is clearly correct.

I do not think this is an accident. It’s not a mistake, or an awkwardly-chosen technical term that people are misinterpreting.

This is a motte and bailey doctrine.

Motte-and-bailey was a strategy in medieval warfare. Defending an entire region is very difficult, so instead what was often done was constructing a small, highly defensible fortification—the motte—while accepting that the land surrounding it—the bailey—would not be well-defended. Most of the time, the people stayed on the bailey, where the land was fertile and it was relatively pleasant to live. But should they be attacked, they could retreat to the motte and defend themselves until the danger was defeated.

A motte-and-bailey doctrine is an analogous strategy used in argumentation. You use the same words for two different versions of an idea: The motte is a narrow, defensible core of your idea that you can provide strong evidence for, but it isn’t very strong and may not even be interesting or controversial. The bailey is a broad, expansive version of your idea that is interesting and controversial and leads to lots of significant conclusions, but can’t be well-supported by evidence.

The bailey is the efficiency hypothesis: That market prices are optimal and we are fools to try to intervene or even regulate them because the almighty Invisible Hand is superior to us.

The motte is the unpredictability hypothesis: Market prices are very hard to predict, and most people who try to make money by beating the market fail.

By referring to both of these very different ideas as “the efficient market hypothesis”, economists can act as if they are defending the bailey, and prescribe policies that deregulate financial markets on the grounds that they are so optimal and efficient; but then when pressed for evidence to support their beliefs, they can pivot to the motte, and merely show that markets are unpredictable. As long as people don’t catch on and recognize that these are two very different meanings of “the efficient market hypothesis”, then they can use the evidence for unpredictability to support their goal of deregulation.

Yet when you look closely at this argument, it collapses. Unpredictability is not evidence of efficiency; if anything, it’s the opposite. Since the world doesn’t really change on a minute-by-minute basis, an efficient system should actually be relatively predictable in the short term. If prices reflected the real value of companies, they would change only very gradually, as the fortunes of the company change as a result of real-world events. An earthquake or a discovery of a new mine would change stock prices in relevant industries; but most of the time, they’d be basically flat. The occurrence of minute-by-minute or even second-by-second changes in prices basically proves that we are not tracking any genuine changes in value.

Roulette wheels are extremely unpredictable by design—by law, even—and yet no one would accuse them of being an efficient way of allocating resources. If you bet on roulette wheels and try to beat the house, you will almost surely fail, just as you would if you try to beat the stock market—and dare I say, for much the same reasons?

So if we’re going to insist that “efficiency” just means unpredictability, rather than actual, you know, efficiency, then we should all speak of the Efficient Roulette Hypothesis. Anything we can’t predict is now automatically “efficient” and should therefore be left unregulated.

Small deviations can have large consequences.

pnrj core principles, critique of neoclassical economics, market failure , , , , , ,

Jun 26 JDN 2459787

A common rejoinder that behavioral economists get from neoclassical economists is that most people are mostly rational most of the time, so what’s the big deal? If humans are 90% rational, why worry so much about the other 10%?

Well, it turns out that small deviations from rationality can have surprisingly large consequences. Let’s consider an example.

Suppose we have a market for some asset. Without even trying to veil my ulterior motive, let’s make that asset Bitcoin. Its fundamental value is of course $0; it’s not backed by anything (not even taxes or a central bank), it has no particular uses that aren’t already better served by existing methods, and it’s not even scalable.

Now, suppose that 99% of the population rationally recognizes that the fundamental value of the asset is indeed $0. But 1% of the population doesn’t; they irrationally believe that the asset is worth $20,000. What will the price of that asset be, in equilibrium?

If you assume that the majority will prevail, it should be $0. If you did some kind of weighted average, you’d think maybe its price will be something positive but relatively small, like $200. But is this actually the price it will take on?

Consider someone who currently owns 1 unit of the asset, and recognizes that it is fundamentally worthless. What should they do? Well, if they also know that there are people out there who believe it is worth $20,000, the answer is obvious: They should sell it to those people. Indeed, they should sell it for something quite close to $20,000 if they can.

Now, suppose they don’t already own the asset, but are considering whether or not to buy it. They know it’s worthless, but they also know that there are people who will buy it for close to $20,000. Here’s the kicker: This is a reason for them to buy it at anything meaningfully less than $20,000.

Suppose, for instance, they could buy it for $10,000. Spending $10,000 to buy something you know is worthless seems like a terribly irrational thing to do. But it isn’t irrational, if you also know that somewhere out there is someone who will pay $20,000 for that same asset and you have a reasonable chance of finding that person and selling it to them.

The equilibrium outcome, then, is that the price of the asset will be almost $20,000! Even though 99% of the population recognizes that this asset is worthless, the fact that 1% of people believe it’s worth as much as a car will result in it selling at that price. Thus, even a slight deviation from a perfectly-rational population can result in a market that is radically at odds with reality.

And it gets worse! Suppose that in fact everyone knows that the asset is worthless, but most people think that there is some small portion of the population who believes the asset has value. Then, it will still be priced at that value in equilibrium, as people trade it back and forth searching in vain for the person who really wants it! (This is called the Greater Fool theory.)

That is, the price of an asset in a free market—even in a market where most people are mostly rational most of the time—will in fact be determined by the highest price anyone believes that anyone else thinks it has. And this is true of essentially any asset market—any market where people are buying something, not to use it, but to sell it to someone else.

Of course, beliefs—and particularly beliefs about beliefs—can very easily change, so that equilibrium price could move in any direction basically without warning.

Suddenly, the cycle of bubble and crash, boom and bust, doesn’t seem so surprising does it? The wonder is that prices ever become stable at all.


Then again, do they? Last I checked, the only prices that were remotely stable were for goods like apples and cars and televisions, goods that are bought and sold to be consumed. (Or national currencies managed by competent central banks, whose entire job involves doing whatever it takes to keep those prices stable.) For pretty much everything else—and certainly any purely financial asset that isn’t a national currency—prices are indeed precisely as wildly unpredictable and utterly irrational as this model would predict.

So much for the Efficient Market Hypothesis? Sadly I doubt that the people who still believe this nonsense will be convinced.

Unsolved problems

pnrj core principles, critique of neoclassical economics, macroeconomics, public policy, science and academia , , , , , ,

Oct 20 JDN 2458777

The beauty and clearness of the dynamical theory, which asserts heat and light to be modes of motion, is at present obscured by two clouds. The first came into existence with the undulatory theory of light, and was dealt with by Fresnel and Dr. Thomas Young; it involved the question, how could the earth move through an elastic solid, such as essentially is the luminiferous ether? The second is the Maxwell-Boltzmann doctrine regarding the partition of energy.


~ Lord Kelvin, April 27, 1900

The above quote is part of a speech where Kelvin basically says that physics is a completed field, with just these two little problems to clear up, “two clouds” in a vast clear horizon. Those “two clouds” Kelvin talked about, regarding the ‘luminiferous ether’ and the ‘partition of energy’? They are, respectively, relativity and quantum mechanics. Almost 120 years later we still haven’t managed to really solve them, at least not in a way that works consistently as part of one broader theory.

But I’ll give Kelvin this: He knew where the problems were. He vastly underestimated how complex and difficult those problems would be, but he knew where they were.

I’m not sure I can say the same about economists. We don’t seem to have even reached the point where we agree where the problems are. Consider another quotation:

For a long while after the explosion of macroeconomics in the 1970s, the field looked like a battlefield. Over time however, largely because facts do not go away, a largely shared vision both of fluctuations and of methodology has emerged. Not everything is fine. Like all revolutions, this one has come with the destruction of some knowledge, and suffers from extremism and herding. None of this deadly however. The state of macro is good.


~ Oliver Blanchard, 2008

The timing of Blanchard’s remark is particularly ominous: It is much like the turkey who declares, the day before Thanksgiving, that his life is better than ever.

But the content is also important: Blanchard didn’t say that microeconomics is in good shape (which I think one could make a better case for). He didn’t even say that economics, in general, is in good shape. He specifically said, right before the greatest economic collapse since the Great Depression, that macroeconomics was in good shape. He didn’t merely underestimate the difficulty of the problem; he didn’t even see where the problem was.

If you search the Web, you can find a few lists of unsolved problems in economics. Wikipedia has such a list that I find particularly bad; Mike Moffatt offers a better list that still has significant blind spots.

Wikipedia’s list is full of esoteric problems that require deeply faulty assumptions to even exist, like the ‘American option problem’ which assumes that the Black-Scholes model is even remotely an accurate description of how option prices work, or the ‘tatonnement problem’ which ignores the fact that there may be many equilibria and we might never reach one at all, or the problem they list under ‘revealed preferences’ which doesn’t address any of the fundamental reasons why the entire concept of revealed preferences may fail once we apply a realistic account of cognitive science. (I could go pretty far afield with that last one—and perhaps I will in a later post—but for now, suffice it to say that human beings often freely choose to do things that we later go on to regret.) I think the only one that Wikipedia’s list really gets right is ‘Unified models of human biases’. The ‘home bias in trade’ and ‘Feldstein-Horioka Puzzle’ problems are sort of edging toward genuine problems, but they’re bound up in too many false assumptions to really get at the right question, which is actually something like “How do we deal with nationalism?” Referring to the ‘Feldstein-Horioka Puzzle’ misses the forest for the trees. Likewise, the ‘PPP Puzzle’ and the ‘Exchange rate disconnect puzzle’ (and to some extent the ‘equity premium puzzle’ as well) are really side effects of a much deeper problem, which is that financial markets in general are ludicrously volatile and inefficient and we have no idea why.

And Wikipedia’s list doesn’t have some of the largest, most important problems in economics. Moffatt’s list does better, including good choices like “What Caused the Industrial Revolution?”, “What Is the Proper Size and Scope of Government?”, and “What Truly Caused the Great Depression?”, but it also includes some of the more esoteric problems like the ‘equity premium puzzle’ and the ‘endogeneity of money’. The way he states the problem “What Causes the Variation of Income Among Ethnic Groups?” suggests that he doesn’t quite understand what’s going on there either. More importantly, Moffatt still leaves out very obviously important questions like “How do we achieve economic development in poor countries?” (Or as I sometimes put it, “What did South Korea do from 1950 to 2000, and how can we do it again?”), “How do we fix shortages of housing and other necessities?”, “What is causing the global rise of income and wealth inequality?”, “How altruistic are human beings, to whom, and under what conditions?” and “What makes financial markets so unstable?” Ironically, ‘Unified models of human biases’, the one problem that Wikipedia got right, is missing from Moffatt’s list.

And I’m also humble enough to realize that some of the deepest problems in economics may be ones that we don’t even quite know how to formulate yet. We like to pretend that economics is a mature science, almost on the coattails of physics; but it’s really a very young science, more like psychology. We go through these ‘cargo cult science‘ rituals of p-values and econometric hypothesis tests, but there are deep, basic forces we don’t understand. We have precisely prepared all the apparatus for the detection of the phlogiston, and by God, we’ll get that 0.05 however we have to. (Think I’m being too harsh? “Real Business Cycle” theory essentially posits that the Great Depression was caused by everyone deciding that they weren’t going to work for a few years, and as whole countries fell into the abyss from failing financial markets, most economists still clung to the Efficient Market Hypothesis.) Our whole discipline requires major injections of intellectual humility: We not only don’t have all the answers; we’re not even sure we have all the questions.

I think the esoteric nature of questions like ‘the equity premium puzzle’ and the ‘tatonnement problem‘ is precisely the source of their appeal: It’s the sort of thing you can say you’re working on and sound very smart, because the person you’re talking to likely has no idea what you’re talking about. (Or else they are a fellow economist, and thus in on the con.) If you said that you’re trying to explain why poor countries are poor and why rich countries are rich—and if economics isn’t doing that, then what in the world are we doing?—you’d have to admit that we honestly have only the faintest idea, and that millions of people have suffered from bad advice economists gave their governments based on ideas that turned out to be wrong.

It’s really quite problematic how closely economists are tied to policymaking (except when we do really know what we’re talking about?). We’re trying to do engineering without even knowing physics. Maybe there’s no way around it: We have to make some sort of economic policy, and it makes more sense to do it based on half-proven ideas than on completely unfounded ideas. (Engineering without physics worked pretty well for the Romans, after all.) But it seems to me that we could be relying more, at least for the time being, on the experiences and intuitions of the people who have worked on the ground, rather than on sophisticated theoretical models that often turn out to be utterly false. We could eschew ‘shock therapy‘ approaches that try to make large interventions in an economy all at once, in favor of smaller, subtler adjustments whose consequences are more predictable. We could endeavor to focus on the cases where we do have relatively clear knowledge (like rent control) and avoid those where the uncertainty is greatest (like economic development).

At the very least, we could admit what we don’t know, and admit that there is probably a great deal we don’t know that we don’t know.