Why are humans so bad with probability?

Apr 29 JDN 2458238

In previous posts on deviations from expected utility and cumulative prospect theory, I’ve detailed some of the myriad ways in which human beings deviate from optimal rational behavior when it comes to probability.

This post is going to be a bit different: Yes, we behave irrationally when it comes to probability. Why?

Why aren’t we optimal expected utility maximizers?
This question is not as simple as it sounds. Some of the ways that human beings deviate from neoclassical behavior are simply because neoclassical theory requires levels of knowledge and intelligence far beyond what human beings are capable of; basically anything requiring “perfect information” qualifies, as does any game theory prediction that involves solving extensive-form games with infinite strategy spaces by backward induction. (Don’t feel bad if you have no idea what that means; that’s kind of my point. Solving infinite extensive-form games by backward induction is an unsolved problem in game theory; just this past week I saw a new paper presented that offered a partial potential solutionand yet we expect people to do it optimally every time?)

I’m also not going to include questions of fundamental uncertainty, like “Will Apple stock rise or fall tomorrow?” or “Will the US go to war with North Korea in the next ten years?” where it isn’t even clear how we would assign a probability. (Though I will get back to them, for reasons that will become clear.)

No, let’s just look at the absolute simplest cases, where the probabilities are all well-defined and completely transparent: Lotteries and casino games. Why are we so bad at that?

Lotteries are not a computationally complex problem. You figure out how much the prize is worth to you, multiply it by the probability of winning—which is clearly spelled out for you—and compare that to how much the ticket price is worth to you. The most challenging part lies in specifying your marginal utility of wealth—the “how much it’s worth to you” part—but that’s something you basically had to do anyway, to make any kind of trade-offs on how to spend your time and money. Maybe you didn’t need to compute it quite so precisely over that particular range of parameters, but you need at least some idea how much $1 versus $10,000 is worth to you in order to get by in a market economy.

Casino games are a bit more complicated, but not much, and most of the work has been done for you; you can look on the Internet and find tables of probability calculations for poker, blackjack, roulette, craps and more. Memorizing all those probabilities might take some doing, but human memory is astonishingly capacious, and part of being an expert card player, especially in blackjack, seems to involve memorizing a lot of those probabilities.

Furthermore, by any plausible expected utility calculation, lotteries and casino games are a bad deal. Unless you’re an expert poker player or blackjack card-counter, your expected income from playing at a casino is always negative—and the casino set it up that way on purpose.

Why, then, can lotteries and casinos stay in business? Why are we so bad at such a simple problem?

Clearly we are using some sort of heuristic judgment in order to save computing power, and the people who make lotteries and casinos have designed formal models that can exploit those heuristics to pump money from us. (Shame on them, really; I don’t fully understand why this sort of thing is legal.)

In another previous post I proposed what I call “categorical prospect theory”, which I think is a decently accurate description of the heuristics people use when assessing probability (though I’ve not yet had the chance to test it experimentally).

But why use this particular heuristic? Indeed, why use a heuristic at all for such a simple problem?

I think it’s helpful to keep in mind that these simple problems are weird; they are absolutely not the sort of thing a tribe of hunter-gatherers is likely to encounter on the savannah. It doesn’t make sense for our brains to be optimized to solve poker or roulette.

The sort of problems that our ancestors encountered—indeed, the sort of problems that we encounter, most of the time—were not problems of calculable probability risk; they were problems of fundamental uncertainty. And they were frequently matters of life or death (which is why we’d expect them to be highly evolutionarily optimized): “Was that sound a lion, or just the wind?” “Is this mushroom safe to eat?” “Is that meat spoiled?”

In fact, many of the uncertainties most important to our ancestors are still important today: “Will these new strangers be friendly, or dangerous?” “Is that person attracted to me, or am I just projecting my own feelings?” “Can I trust you to keep your promise?” These sorts of social uncertainties are even deeper; it’s not clear that any finite being could ever totally resolve its uncertainty surrounding the behavior of other beings with the same level of intelligence, as the cognitive arms race continues indefinitely. The better I understand you, the better you understand me—and if you’re trying to deceive me, as I get better at detecting deception, you’ll get better at deceiving.

Personally, I think that it was precisely this sort of feedback loop that resulting in human beings getting such ridiculously huge brains in the first place. Chimpanzees are pretty good at dealing with the natural environment, maybe even better than we are; but even young children can outsmart them in social tasks any day. And once you start evolving for social cognition, it’s very hard to stop; basically you need to be constrained by something very fundamental, like, say, maximum caloric intake or the shape of the birth canal. Where chimpanzees look like their brains were what we call an “interior solution”, where evolution optimized toward a particular balance between cost and benefit, human brains look more like a “corner solution”, where the evolutionary pressure was entirely in one direction until we hit up against a hard constraint. That’s exactly what one would expect to happen if we were caught in a cognitive arms race.

What sort of heuristic makes sense for dealing with fundamental uncertainty—as opposed to precisely calculable probability? Well, you don’t want to compute a utility function and multiply by it, because that adds all sorts of extra computation and you have no idea what probability to assign. But you’ve got to do something like that in some sense, because that really is the optimal way to respond.

So here’s a heuristic you might try: Separate events into some broad categories based on how frequently they seem to occur, and what sort of response would be necessary.

Some things, like the sun rising each morning, seem to always happen. So you should act as if those things are going to happen pretty much always, because they do happen… pretty much always.

Other things, like rain, seem to happen frequently but not always. So you should look for signs that those things might happen, and prepare for them when the signs point in that direction.

Still other things, like being attacked by lions, happen very rarely, but are a really big deal when they do. You can’t go around expecting those to happen all the time, that would be crazy; but you need to be vigilant, and if you see any sign that they might be happening, even if you’re pretty sure they’re not, you may need to respond as if they were actually happening, just in case. The cost of a false positive is much lower than the cost of a false negative.

And still other things, like people sprouting wings and flying, never seem to happen. So you should act as if those things are never going to happen, and you don’t have to worry about them.

This heuristic is quite simple to apply once set up: It can simply slot in memories of when things did and didn’t happen in order to decide which category they go in—i.e. availability heuristic. If you can remember a lot of examples of “almost never”, maybe you should move it to “unlikely” instead. If you get a really big number of examples, you might even want to move it all the way to “likely”.

Another large advantage of this heuristic is that by combining utility and probability into one metric—we might call it “importance”, though Bayesian econometricians might complain about that—we can save on memory space and computing power. I don’t need to separately compute a utility and a probability; I just need to figure out how much effort I should put into dealing with this situation. A high probability of a small cost and a low probability of a large cost may be equally worth my time.

How might these heuristics go wrong? Well, if your environment changes sufficiently, the probabilities could shift and what seemed certain no longer is. For most of human history, “people walking on the Moon” would seem about as plausible as sprouting wings and flying away, and yet it has happened. Being attacked by lions is now exceedingly rare except in very specific places, but we still harbor a certain awe and fear before lions. And of course availability heuristic can be greatly distorted by mass media, which makes people feel like terrorist attacks and nuclear meltdowns are common and deaths by car accidents and influenza are rare—when exactly the opposite is true.

How many categories should you set, and what frequencies should they be associated with? This part I’m still struggling with, and it’s an important piece of the puzzle I will need before I can take this theory to experiment. There is probably a trade-off between more categories giving you more precision in tailoring your optimal behavior, but costing more cognitive resources to maintain. Is the optimal number 3? 4? 7? 10? I really don’t know. Even I could specify the number of categories, I’d still need to figure out precisely what categories to assign.

Is grade inflation a real problem?

Mar 4 JDN 2458182

You can’t spend much time teaching at the university level and not hear someone complain about “grade inflation”. Almost every professor seems to believe in it, and yet they must all be participating in it, if it’s really such a widespread problem.

This could be explained as a collective action problem, a Tragedy of the Commons: If the incentives are always to have the students with the highest grades—perhaps because of administrative pressure, or in order to get better reviews from students—then even if all professors would prefer a harsher grading scheme, no individual professor can afford to deviate from the prevailing norms.

But in fact I think there is a much simpler explanation: Grade inflation doesn’t exist.

In economic growth theory, economists make a sharp distinction between inflation—increase in prices without change in underlying fundamentals—and growth—increase in the real value of output. I contend that there is no such thing as grade inflation—what we are in fact observing is grade growth.
Am I saying that students are actually smarter now than they were 30 years ago?

Yes. That’s exactly what I’m saying.

But don’t take it from me. Take it from the decades of research on the Flynn Effect: IQ scores have been rising worldwide at a rate of about 0.3 IQ points per year for as long as we’ve been keeping good records. Students today are about 10 IQ points smarter than students 30 years ago—a 2018 IQ score of 95 is equivalent to a 1988 score of 105, which is equivalent to a 1958 score of 115. There is reason to think this trend won’t continue indefinitely, since the effect is mainly concentrated at the bottom end of the distribution; but it has continued for quite some time already.

This by itself would probably be enough to explain the observed increase in grades, but there’s more: College students are also a self-selected sample, admitted precisely because they were believed to be the smartest individuals in the application pool. Rising grades at top institutions are easily explained by rising selectivity at top schools: Harvard now accepts 5.6% of applicants. In 1942, Harvard accepted 92% of applicants. The odds of getting in have fallen from 9:1 in favor to 19:1 against. Today, you need a 4.0 GPA, a 36 ACT in every category, glowing letters of recommendation, and hundreds of hours of extracurricular activities (or a family member who donated millions of dollars, of course) to get into Harvard. In the 1940s, you needed a high school diploma and a B average.

In fact, when educational researchers have tried to quantitatively study the phenomenon of “grade inflation”, they usually come back with the result that they simply can’t find it. The US department of education conducted a study in 1995 showing that average university grades had declined since 1965. Given that the Flynn effect raised IQ by almost 10 points during that time, maybe we should be panicking about grade deflation.

It really wouldn’t be hard to make that case: “Back in my day, you could get an A just by knowing basic algebra! Now they want these kids to take partial derivatives?” “We used to just memorize facts to ace the exam; but now teachers keep asking for reasoning and critical thinking?”

More recently, a study in 2013 found that grades rose at the high school level, but fell at the college level, and showed no evidence of losing any informativeness as a signaling mechanism. The only recent study I could find showing genuinely compelling evidence for grade inflation was a 2017 study of UK students estimating that grades are growing about twice as fast as the Flynn effect alone would predict. Most studies don’t even consider the possibility that students are smarter than they used to be—they just take it for granted that any increase in average grades constitutes grade inflation. Many of them don’t even control for the increase in selectivity—here’s one using the fact that Harvard’s average rose from 2.7 to 3.4 from 1960 to 2000 as evidence of “grade inflation” when Harvard’s acceptance rate fell from almost 30% to only 10% during that period.

Indeed, the real mystery is why so many professors believe in grade inflation, when the evidence for it is so astonishingly weak.

I think it’s availability heuristic. Who are professors? They are the cream of the crop. They aced their way through high school, college, and graduate school, then got hired and earned tenure—they were one of a handful of individuals who won a fierce competition with hundreds of competitors at each stage. There are over 320 million people in the US, and only 1.3 million college faculty. This means that college professors represent about the top 0.4% of high-scoring students.

Combine that with the fact that human beings assort positively (we like to spend time with people who are similar to us) and use availability heuristic (we judge how likely something is based on how many times we have seen it).

Thus, when a professor compares to her own experience of college, she is remembering her fellow top-scoring students at elite educational institutions. She is recalling the extreme intellectual demands she had to meet to get where she is today, and erroneously assuming that these are representative of most the population of her generation. She probably went to school at one of a handful of elite institutions, even if she now teaches at a mid-level community college: three quarters of college faculty come from the top one quarter of graduate schools.

And now she compares to the students she has to teach, most of whom would not be able to meet such demands—but of course most people in her generation couldn’t either. She frets for the future of humanity only because not everyone is a genius like her.

Throw in the Curse of Knowledge: The professor doesn’t remember how hard it was to learn what she has learned so far, and so the fact that it seems easy now makes her think it was easy all along. “How can they not know how to take partial derivatives!?” Well, let’s see… were you born knowing how to take partial derivatives?

Giving a student an A for work far inferior to what you’d have done in their place isn’t unfair. Indeed, it would clearly be unfair to do anything less. You have years if not decades of additional education ahead of them, and you are from self-selected elite sample of highly intelligent individuals. Expecting everyone to perform as well as you would is simply setting up most of the population for failure.

There are potential incentives for grade inflation that do concern me: In particular, a lot of international student visas and scholarship programs insist upon maintaining a B or even A- average to continue. Professors are understandably loathe to condemn a student to having to drop out or return to their home country just because they scored 81% instead of 84% on the final exam. If we really intend to make C the average score, then students shouldn’t lose funding or visas just for scoring a B-. Indeed, I have trouble defending any threshold above outright failing—which is to say, a minimum score of D-. If you pass your classes, that should be good enough to keep your funding.

Yet apparently even this isn’t creating too much upward bias, as students who are 10 IQ points smarter are still getting about the same scores as their forebears. We should be celebrating that our population is getting smarter, but instead we’re panicking over “easy grading”.

But kids these days, am I right?

Nuclear power is safe. Why don’t people like it?

Sep 24, JDN 2457656

This post will have two parts, corresponding to each sentence. First, I hope to convince you that nuclear power is safe. Second, I’ll try to analyze some of the reasons why people don’t like it and what we might be able to do about that.

Depending on how familiar you are with the statistics on nuclear power, the idea that nuclear power is safe may strike you as either a completely ridiculous claim or an egregious understatement. If your primary familiarity with nuclear power safety is via the widely-publicized examples of Chernobyl, Three Mile Island, and more recently Fukushima, you may have the impression that nuclear power carries huge, catastrophic risks. (You may also be confusing nuclear power with nuclear weapons—nuclear weapons are indeed the greatest catastrophic risk on Earth today, but equating the two is like equating automobiles and machine guns because both of them are made of metal and contain lubricant, flammable materials, and springs.)

But in fact nuclear energy is astonishingly safe. Indeed, even those examples aren’t nearly as bad as people have been led to believe. Guess how many people died as a result of Three Mile Island, including estimated increased cancer deaths from radiation exposure?

Zero. There are zero confirmed deaths and the consensus estimate of excess deaths caused by the Three Mile Island incident by all causes combined is zero.

What about Fukushima? Didn’t 10,000 people die there? From the tsunami, yes. But the nuclear accident resulted in zero fatalities. If anything, those 10,000 people were killed by coal—by climate change. They certainly weren’t killed by nuclear.

Chernobyl, on the other hand, did actually kill a lot of people. Chernobyl caused 31 confirmed direct deaths, as well as an estimated 4,000 excess deaths by all causes. On the one hand, that’s more than 9/11; on the other hand, it’s about a month of US car accidents. Imagine if people had the same level of panic and outrage at automobiles after a month of accidents that they did at nuclear power after Chernobyl.

The vast majority of nuclear accidents cause zero fatalities; other than Chernobyl, none have ever caused more than 10. Deepwater Horizon killed 11 people, and yet for some reason Americans did not unite in opposition against ever using oil (or even offshore drilling!) ever again.

In fact, even that isn’t fair to nuclear power, because we’re not including the thousands of lives saved every year by using nuclear instead of coal and oil.

Keep in mind, the WHO estimates 10 to 100 million excess deaths due to climate change over the 21st century. That’s an average of 100,000 to 1 million deaths every year. Nuclear power currently produces about 11% of the world’s energy, so let’s do a back-of-the-envelope calculation for how many lives that’s saving. Assuming that additional climate change would be worse in direct proportion to the additional carbon emissions (which is conservative), and assuming that half that energy would be replaced by coal or oil (also conservative, using Germany’s example), we’re looking at about a 6% increase in deaths due to climate change if all those nuclear power plants were closed. That’s 6,000 to 60,000 lives that nuclear power plants save every year.

I also haven’t included deaths due to pollution—note that nuclear power plants don’t pollute air or water whatsoever, and only produce very small amounts of waste that can be quite safely stored. Air pollution in all its forms is responsible for one in eight deaths worldwide. Let me say that again: One in eight of all deaths in the world is caused by air pollution—so this is on the order of 7 million deaths per year, every year. We burn our way to a biannual Holocaust. Most of this pollution is actually caused by burning wood—fireplaces, wood stoves, and bonfires are terrible for the air—and many countries would actually see a substantial reduction in their toxic pollution if they switched to oil or even coal in favor of wood. But a large part of that pollution is caused by coal, and a nontrivial amount is caused by oil. Coal-burning factories and power plants are responsible for about 1 million deaths per year in China alone. Most of that pollution could be prevented if those power plants were nuclear instead.

Factor all that in, and nuclear power currently saves tens if not hundreds of thousands of lives per year, and expanding it to replace all fossil fuels could save millions more. Indeed, a more precise estimate of the benefits of nuclear power published a few years ago in Environmental Science and Technology is that nuclear power plants have saved some 1.8 million human lives since their invention, putting them on a par with penicillin and the polio vaccine.

So, I hope I’ve convinced you of the first proposition: Nuclear power plants are safe—and not just safe, but heroic, in fact one of the greatest life-saving technologies ever invented. So, why don’t people like them?

Unfortunately, I suspect that no amount of statistical data by itself will convince those who still feel a deep-seated revulsion to nuclear power. Even many environmentalists, people who could be nuclear energy’s greatest advocates, are often opposed to it. I read all the way through Naomi Klein’s This Changes Everything and never found even a single cogent argument against nuclear power; she simply takes it as obvious that nuclear power is “more of the same line of thinking that got us in this mess”. Perhaps because nuclear power could be enormously profitable for certain corporations (which is true; but then, it’s also true of solar and wind power)? Or because it also fits this narrative of “raping and despoiling the Earth” (sort of, I guess)? She never really does explain; I’m guessing she assumes that her audience will simply share her “gut feeling” intuition that nuclear power is dangerous and untrustworthy. One of the most important inconvenient truths for environmentalists is that nuclear power is not only safe, it is almost certainly our best hope for stopping climate change.

Perhaps all this is less baffling when we recognize that other heroic technologies are often also feared or despised for similarly bizarre reasons—vaccines, for instance.

First of all, human beings fear what we cannot understand, and while the human immune system is certainly immensely complicated, nuclear power is based on quantum mechanics, a realm of scientific knowledge so difficult and esoteric that it is frequently used as the paradigm example of something that is hard to understand. (As Feynman famously said, “I think I can safely say that nobody understands quantum mechanics.”) Nor does it help that popular treatments of quantum physics typically bear about as much resemblance to the actual content of the theory as the X-Men films do to evolutionary biology, and con artists like Deepak Chopra take advantage of this confusion to peddle their quackery.

Nuclear radiation is also particularly terrifying because it is invisible and silent; while a properly-functioning nuclear power plant emits less ionizing radiation than the Capitol Building and eating a banana poses substantially higher radiation risk than talking on a cell phone, nonetheless there is real danger posed by ionizing radiation, and that danger is particularly terrifying because it takes a form that human senses cannot detect. When you are burned by fire or cut by a knife, you know immediately; but gamma rays could be coursing through you right now and you’d feel no different. (Huge quantities of neutrinos are coursing through you, but fear not, for they’re completely harmless.) The symptoms of severe acute radiation poisoning also take a particularly horrific form: After the initial phase of nausea wears off, you can enter a “walking ghost phase”, where your eventual death is almost certain due to your compromised immune and digestive systems, but your current condition is almost normal. This makes the prospect of death by nuclear accident a particularly vivid and horrible image.

Vividness makes ideas more available to our memory; and thus, by the availability heuristic, we automatically infer that it must be more probable than it truly is. You can think of horrific nuclear accidents like Chernobyl, and all the carnage they caused; but all those millions of people choking to death in China don’t make for a compelling TV news segment (or at least, our TV news doesn’t seem to think so). Vividness doesn’t actually seem to make things more persuasive, but it does make them more memorable.

Yet even if we allow for the possibility that death by radiation poisoning is somewhat worse than death by coal pollution (if I had to choose between the two, okay, maybe I’d go with the coal), surely it’s not ten thousand times worse? Surely it’s not worth sacrificing entire cities full of people to coal in order to prevent a handful of deaths by nuclear energy?

Another reason that has been proposed is a sense that we can control risk from other sources, but a nuclear meltdown would be totally outside our control. Perhaps that is the perception, but if you think about it, it really doesn’t make a lot of sense. If there’s a nuclear meltdown, emergency services will report it, and you can evacuate the area. Yes, the radiation moves at the speed of light; but it also dissipates as the inverse square of distance, so if you just move further away you can get a lot safer quite quickly. (Think about the brightness of a lamp in your face versus across a football field. Radiation works the same way.) The damage is also cumulative, so the radiation risk from a meltdown is only going to be serious if you stay close to the reactor for a sustained period of time. Indeed, it’s much easier to avoid nuclear radiation than it is to avoid air pollution; you can’t just stand behind a concrete wall to shield against air pollution, and moving further away isn’t possible if you don’t know where it’s coming from. Control would explain why we fear cars less than airplanes (which is also statistically absurd), but it really can’t explain why nuclear power scares people more than coal and oil.

Another important factor may be an odd sort of bipartisan consensus: While the Left hates nuclear power because it makes corporations profitable or because it’s unnatural and despoils the Earth or something, the Right hates nuclear power because it requires substantial government involvement and might displace their beloved fossil fuels. (The Right’s deep, deep love of the fossil fuel industry now borders on the pathological. Even now that they are obviously economically inefficient and environmentally disastrous, right-wing parties around the world continue to defend enormous subsidies for oil and coal companies. Corruption and regulatory capture could partly explain this, but only partly. Campaign contributions can’t explain why someone would write a book praising how wonderful fossil fuels are and angrily denouncing anyone who would dare criticize them.) So while the two sides may hate each other in general and disagree on most other issues—including of course climate change itself—they can at least agree that nuclear power is bad and must be stopped.

Where do we go from here, then? I’m not entirely sure. As I said, statistical data by itself clearly won’t be enough. We need to find out what it is that makes people so uniquely terrified of nuclear energy, and we need to find a way to assuage those fears.

And we must do this now. For every day we don’t—every day we postpone the transition to a zero-carbon energy grid—is another thousand people dead.