The inherent atrocity of “border security”

Jun 24 JDN 2458294

By now you are probably aware of the fact that a new “zero tolerance” border security policy under the Trump administration has resulted in 2,000 children being forcibly separated from their parents by US government agents. If you weren’t, here are a variety of different sources all telling the same basic story of large-scale state violence and terror.

Make no mistake: This is an atrocity. The United Nations has explicitly condemned this human rights violation—to which Trump responded by making an unprecedented threat of withdrawing unilaterally from the UN Human Rights Council.

#ThisIsNotNormal, and Trump was everything we feared—everything we warned—he would be: Corrupt, incompetent, cruel, and authoritarian.

Yet Trump’s border policy differs mainly in degree, not kind, from existing US border policy. There is much more continuity here than most of us would like to admit.

The Trump administration has dramatically increased “interior removals”, the most obviously cruel acts, where ICE agents break into the houses of people living in the US and take them away. Don’t let the cold language fool you; this is literally people with guns breaking into your home and kidnapping members of your family. This is characteristic of totalitarian governments, not liberal democracies.

And yet, the Obama administration actually holds the record for most deportations (though only because they included “at-border deportations” which other administrations did not). A major policy change by George W. Bush started this whole process of detaining people at the border instead of releasing them and requiring them to return for later court dates.

I could keep going back; US border enforcement has gotten more and more aggressive as time goes on. US border security staffing has quintupled since just 1990. There was a time when the United States was a land of opportunity that welcomed “your tired, your poor, your huddled masses”; but that time is long past.

And this, in itself, is a human rights violation. Indeed, I am convinced that border security itself is inherently a human rights violation, always and everywhere; future generations will not praise us for being more restrained than Trump’s abject and intentional cruelty, but condemn us for acting under the same basic moral framework that justified it.

There is an imaginary line in the sand just a hundred miles south of where I sit now. On one side of the line, a typical family makes $66,000 per year. On the other side, a typical family makes only $20,000. On one side of the line, life expectancy is 81 years; on the other, 77. This means that over their lifetime, someone on this side of the line can expect to make over one million dollars more than they would if they had lived on the other side. Step across this line, get a million dollars; it sounds ridiculous, but it’s an empirical fact.

This would be bizarre enough by itself; but now consider that on that line there are fences, guard towers, and soldiers who will keep you from crossing it. If you have appropriate papers, you can cross; but if you don’t, they will arrest and detain you, potentially for months. This is not how we treat you if you are carrying contraband or have a criminal record. This is how we treat you if you don’t have a passport.

How can we possibly reconcile this with the principles of liberal democracy? Philosophers have tried, to be sure. Yet they invariably rely upon some notion that the people who want to cross our border are coming from another country where they were already granted basic human rights and democratic representation—which is almost never the case. People who come here from the UK or the Netherlands or generally have the proper visas. Even people who come here from China usually have visas—though China is by no means a liberal democracy. It’s people who come here from Haiti and Nicaragua who don’t—and these are some of the most corrupt and impoverished nations in the world.

As I said in an earlier post, I was not offended that Trump characterized countries like Haiti and Syria as “shitholes”. By any objective standard, that is accurate; these countries are terrible, terrible places to live. No, what offends me is that he thinks this gives us a right to turn these people away, as though the horrible conditions of their country somehow “rub off” on them and make them less worthy as human beings. On the contrary, we have a word for people who come from “shithole” countries seeking help, and that word is “refugee”.

Under international law, “refugee” has a very specific legal meaning, under which most immigrants do not qualify. But in a broader moral sense, almost every immigrant is a refugee. People don’t uproot themselves and travel thousands of miles on a whim. They are coming here because conditions in their home country are so bad that they simply cannot tolerate them anymore, and they come to us desperately seeking our help. They aren’t asking for handouts of free money—illegal immigrants are a net gain for our fiscal system, paying more in taxes than they receive in benefits. They are looking for jobs, and willing to accept much lower wages than the workers already here—because those wages are still dramatically higher than what they had where they came from.

Of course, that does potentially mean they are competing with local low-wage workers, doesn’t it? Yes—but not as much as you might think. There is only a very weak relationship between higher immigration and lower wages (some studies find none at all!), even at the largest plausible estimates, the gain in welfare for the immigrants is dramatically higher than the loss in welfare for the low-wage workers who are already here. It’s not even a question of valuing them equally; as long as you value an immigrant at least one tenth as much as a native-born citizen, the equation comes out favoring more immigration.

This is for two reasons: One, most native-born workers already are unwilling to do the jobs that most immigrants do, such as picking fruit and laying masonry; and two, increased spending by immigrants boosts the local economy enough to compensate for any job losses.

 

But even aside from the economic impacts, what is the moral case for border security?

I have heard many people argue that “It’s our home, we should be able to decide who lives here.” First of all, there are some major differences between letting someone live in your home and letting someone come into your country. I’m not saying we should allow immigrants to force themselves into people’s homes, only that we shouldn’t arrest them when they try cross the border.

But even if I were to accept the analogy, if someone were fleeing oppression by an authoritarian government and asked to live in my home, I would let them. I would help hide them from the government if they were trying to escape persecution. I would even be willing to house people simply trying to escape poverty, as long as it were part of a well-organized program designed to ensure that everyone actually gets helped and the burden on homeowners and renters was not too great. I wouldn’t simply let homeless people come live here, because that creates all sorts of coordination problems (I can only fit so many, and how do I prioritize which ones?); but I’d absolutely participate in a program that coordinates placement of homeless families in apartments provided by volunteers. (In fact, maybe I should try to petition for such a program, as Southern California has a huge homelessness rate due to our ridiculous housing prices.)

Many people seem to fear that immigrants will bring crime, but actually they reduce crime rates. It’s really kind of astonishing how much less crime immigrants commit than locals. My hypothesis is that immigrants are a self-selected sample; the kind of person willing to move thousands of miles isn’t the kind of person who commits a lot of crimes.
I understand wanting to keep out terrorists and drug smugglers, but there are already plenty of terrorists and drug smugglers here in the US; if we are unwilling to set up border security between California and Nevada, I don’t see why we should be setting it up between California and Baja California. But okay, fine, we can keep the customs agents who inspect your belongings when you cross the border. If someone doesn’t have proper documentation, we can even detain and interrogate them—for a few hours, not a few months. The goal should be to detect dangerous criminals and nothing else. Once we are confident that you have not committed any felonies, we should let you through—frankly, we should give you a green card. We should only be willing to detain someone at the border for the same reasons we would be willing to detain a citizen who already lives here—that is, probable cause for an actual crime. (And no, you don’t get to count “illegal border crossing” as a crime, because that’s begging the question. By the same logic I could justify detaining people for jaywalking.)

A lot of people argue that restricting immigration is necessary to “preserve local culture”; but I’m not even sure that this is a goal sufficiently important to justify arresting and detaining people, and in any case, that’s really not how culture works. Culture is not advanced by purism and stagnation, but by openness and cross-pollination. From anime to pizza, many of our most valued cultural traditions would not exist without interaction across cultural boundaries. Introducing more Spanish speakers into the US may make us start saying no problemo and vamonos, but it’s not going to destroy liberal democracy. If you value culture, you should value interactions across different societies.

Most importantly, think about what you are trying to justify. Even if we stop doing Trump’s most extreme acts of cruelty, we are still talking about using military force to stop people from crossing an imaginary line. ICE basically treats people the same way the SS did. “Papers, please” isn’t something we associate with free societies—it’s characteristic of totalitarianism. We are so accustomed to border security (or so ignorant of its details) that we don’t see it for the atrocity it so obviously is.

National borders function something very much like feudal privilege. We have our “birthright”, which grants us all sorts of benefits and special privileges—literally tripling our incomes and extending our lives. We did nothing to earn this privilege. If anything, we show ourselves to be less deserving (e.g. by committing more crimes). And we use the government to defend our privilege by force.

Are people born on the other side of the line less human? Are they less morally worthy? On what grounds do we point guns at them and lock them away for the “crime” of wanting to live here?

What Trump is doing right now is horrific. But it is not that much more horrific than what we were already doing. My hope is that this will finally open our eyes to the horrors that we had been participating in all along.

The evolution of human cooperation

Jun 17 JDN 2458287

If alien lifeforms were observing humans (assuming they didn’t turn out the same way—which they actually might, for reasons I’ll get to shortly), the thing that would probably baffle them the most about us is how we organize ourselves into groups. Each individual may be part of several groups at once, and some groups are closer-knit than others; but the most tightly-knit groups exhibit extremely high levels of cooperation, coordination, and self-sacrifice.

They might think at first that we are eusocial, like ants or bees; but upon closer study they would see that our groups are not very strongly correlated with genetic relatedness. We are somewhat more closely related to those in our groups than to those outsides, usually; but it’s a remarkably weak effect, especially compared to the extremely high relatedness of worker bees in a hive. No, to a first approximation, these groups are of unrelated humans; yet their level of cooperation is equal to if not greater than that exhibited by the worker bees.

However, the alien anthropologists would find that it is not that humans are simply predisposed toward extremely high altruism and cooperation in general; when two humans groups come into conflict, they are capable of the most extreme forms of violence imaginable. Human history is full of atrocities that combine the indifferent brutality of nature red in tooth and claw with the boundless ingenuity of a technologically advanced species. Yet except for a small proportion perpetrated by individual humans with some sort of mental pathology, these atrocities are invariably committed by one unified group against another. Even in genocide there is cooperation.

Humans are not entirely selfish. But nor are they paragons of universal altruism (though some of them aspire to be). Humans engage in a highly selective form of altruism—virtually boundless for the in-group, almost negligible for the out-group. Humans are tribal.

Being a human yourself, this probably doesn’t strike you as particularly strange. Indeed, I’ve mentioned it many times previously on this blog. But it is actually quite strange, from an evolutionary perspective; most organisms are not like this.

As I said earlier, there is actually reason to think that our alien anthropologist would come from a species with similar traits, simply because such cooperation may be necessary to achieve a full-scale technological civilization, let alone the capacity for interstellar travel. But there might be other possibilities; perhaps they come from a eusocial species, and their large-scale cooperation is within an extremely large hive.

It’s true that most organisms are not entirely selfish. There are various forms of cooperation within and even across species. But these usually involve only close kin, and otherwise involve highly stable arrangements of mutual benefit. There is nothing like the large-scale cooperation between anonymous unrelated individuals that is exhibited by all human societies.

How would such an unusual trait evolve? It must require a very particular set of circumstances, since it only seems to have evolved in a single species (or at most a handful of species, since other primates and cetaceans display some of the same characteristics).

Once evolved, this trait is clearly advantageous; indeed it turned a local apex predator into a species so successful that it can actually intentionally control the evolution of other species. Humans have become a hegemon over the entire global ecology, for better or for worse. Cooperation gave us a level of efficiency in producing the necessities of survival so great that at this point most of us spend our time working on completely different tasks. If you are not a farmer or a hunter or a carpenter (and frankly, even if you are a farmer with a tractor, a hunter with a rifle, or a carpenter with a table saw), you are doing work that would simply not have been possible without very large-scale human cooperation.

This extremely high fitness benefit only makes the matter more puzzling, however: If the benefits are so great, why don’t more species do this? There must be some other requirements that other species were unable to meet.

One clear requirement is high intelligence. As frustrating as it may be to be a human and watch other humans kill each other over foolish grievances, this is actually evidence of how smart humans are, biologically speaking. We might wish we were even smarter still—but most species don’t have the intelligence to make it even as far as we have.

But high intelligence is likely not sufficient. We can’t be sure of that, since we haven’t encountered any other species with equal intelligence; but what we do know is that even Homo sapiens didn’t coordinate on anything like our current scale for tens of thousands of years. We may have had tribal instincts, but if so they were largely confined to a very small scale. Something happened, about 50,000 years ago or so—not very long ago in evolutionary time—that allowed us to increase that scale dramatically.

Was this a genetic change? It’s difficult to say. There could have been some subtle genetic mutation, something that wouldn’t show up in the fossil record. But more recent expansions in human cooperation to the level of the nation-state and beyond clearly can’t be genetic; they were much too fast for that. They must be a form of cultural evolution: The replicators being spread are ideas and norms—memes—rather than genes.

So perhaps the very early shift toward tribal cooperation was also a cultural one. Perhaps it began not as a genetic mutation but as an idea—perhaps a metaphor of “universal brotherhood” as we often still hear today. The tribes that believed this ideas prospered; the tribes that didn’t were outcompeted or even directly destroyed.

This would explain why it had to be an intelligent species. We needed brains big enough to comprehend metaphors and generalize concepts. We needed enough social cognition to keep track of who was in the in-group and who was in the out-group.

If it was indeed a cultural shift, this should encourage us. (And since the most recent changes definitely were cultural, that is already quite encouraging.) We are not limited by our DNA to only care about a small group of close kin; we are capable of expanding our scale of unity and cooperation far beyond.
The real question is whether we can expand it to everyone. Unfortunately, there is some reason to think that this may not be possible. If our concept of tribal identity inherently requires both an in-group and an out-group, then we may never be able to include everyone. If we are only unified against an enemy, never simply for our own prosperity, world peace may forever remain a dream.

But I do have a work-around that I think is worth considering. Can we expand our concept of the out-group to include abstract concepts? With phrases like “The War on Poverty” and “The War on Terror”, it would seem in fact that we can. It feels awkward; it is somewhat imprecise—but then, so was the original metaphor of “universal brotherhood”. Our brains are flexible enough that they don’t actually seem to need the enemy to be a person; it can also be an idea. If this is right, then we can actually include everyone in our in-group, as long as we define the right abstract out-group. We can choose enemies like poverty, violence, cruelty, and despair instead of other nations or ethnic groups. If we must continue to fight a battle, let it be a battle against the pitiless indifference of the universe, rather than our fellow human beings.

Of course, the real challenge will be getting people to change their existing tribal identities. In the moment, these identities seem fundamentally intractable. But that can’t really be the case—for these identities have changed over historical time. Once-important categories have disappeared; new ones have arisen in their place. Someone in 4th century Constantinople would find the conflict between Democrats and Republicans as baffling as we would find the conflict between Trinitarians and Arians. The ongoing oppression of Native American people by White people would be unfathomable to someone of the 11th century Onondaga, who could scarcely imagine an enemy more different than the Seneca west of them. Even the conflict between Russia and NATO would probably seem strange to someone living in France in 1943, for whom Germany was the enemy and Russia was at least the enemy of the enemy—and many of those people are still alive.

I don’t know exactly how these tribal identities change (I’m working on it). It clearly isn’t as simple as convincing people with rational arguments. In fact, part of how it seems to work is that someone will shift their identity slowly enough that they can’t perceive the shift themselves. People rarely seem to appreciate, much less admit, how much their own minds have changed over time. So don’t ever expect to change someone’s identity in one sitting. Don’t even expect to do it in one year. But never forget that identities do change, even within an individual’s lifetime.

False equivalence is not centrism

False equivalence is not centrism

Feb 4 JDN 2458154

Turning and turning in the widening gyre

The falcon cannot hear the falconer;

Things fall apart; the centre cannot hold;

Mere anarchy is loosed upon the world,

The blood-dimmed tide is loosed, and everywhere

The ceremony of innocence is drowned;

The best lack all conviction, while the worst

Are full of passionate intensity.

~ W.B. Yeats, The Second Coming


Centrism is not very popular these days, but I believe this is because neither its alleged adherents nor its alleged opponents actually have a clear understanding of what centrism is supposed to be. Most of what is called “centrism” in this polarized era (the US is now more politically polarized than it has been in decades) is actually false equivalence.

Most people who express pride in their “centrism” adopt a heuristic which basically amounts to taking the two positions that are most loudly proclaimed in public and averaging them. One side says “Kill all puppies”, the other side says “Don’t kill puppies”, and they proudly and self-righteously declare that the only sensible policy is to kill precisely 50% of the puppies. Anyone who says “the two parties are the same” or “liberals deny science too” is guilty of this false equivalence—and it’s all too common.

But this is not what centrism is supposed to be. A good centrist isn’t someone who looks at their existing Overton Window and chooses the mean value. A good centrist is someone who understands and appreciates Horseshoe Theory. Horseshoe Theory says that the political spectrum is not actually a straight line from left to right; it’s more of a horseshoe shape, where the far-left and the far-right curl down and toward one another. A good centrist is someone who values the top of the horseshoe, more strongly than they value whatever particular policies might move you toward the left or the right edge.

What does the top of the horseshoe represent? Democracy.

A good centrist is someone who really, truly believes in defending democracy.

What the far-left and the far-right have in common is authoritarianism:

For those on either edge of the horseshoe, people who disagree with (the collectivization of all wealth/the superiority of my master race) aren’t simply wrong, they are evil. Persuading them to vote my way is a waste of time. Freedom of speech is dangerous, because it allows them to spread their evil ideas. It would be better to suppress freedom of speech, so that only people who know the truth (read: agree with me) are allowed to speak.

Along similar lines, Slate Star Codex recently published an excellent blog post on how people seem to separate into two very broad political worldviews: There are Mistake Theorists, who think that most of the world’s problems are due to honest ignorance and error; and there are Conflict Theorists, who think that most of the world’s problems are due to the malign influence of evil enemy factions. The far-left and the far-right are overwhelmingly composed of Conflict Theorists. A good centrist is a Mistake Theorist through and through.

Being a good centrist means fighting to defend the institutions that make freedom possible. Here is a whole list of policies that neither the far-left nor the far-right particularly values that we as centrists must:

  1. Voting rights: We must fight against voter suppression and disenfranchisement wherever it occurs. We must stand up to defend the principle “one person, one vote” wherever necessary.
  2. Equality under the law: We must protect the rights of everyone to have equal representation and equal standing as citizens—including, but by no means limited to, women, racial minorities, LGBT people, and people with disabilities.

  3. Election reform: We must find ways to undermine gerrymandering, the Electoral College, and the campaign finance system that allows corporations and wealthy individuals to exert disproportionate influence.

  4. Freedom of speech: We must protect the right of everyone to speak, including those whose views we find abhorrent. Our efforts should be focused most on those who have the least representation in our discourse.

  5. Individual privacy: We must fight against the creeping rise of the surveillance state and the use of extra-legal means of intelligence gathering, particularly in domestic spying. We should be outraged that the House of Representatives voted to extend the NSA’s warrantless wiretap authority after what Edward Snowden revealed about the NSA.

  6. Demilitarization and deincarceration: We must fight to contain or reverse the expansion of military and penal force that has given the United States not only a military larger than the next ten countries combined, but also the world’s highest rate of incarceration.

On some of these issues we might find agreement with the left or (less likely) the right—but even when we don’t, we must press forward. In particular, the goal of equality under the law often aligns with the goal of left-wing social justice—but there are cases where it doesn’t, cases where hatred of White straight men or a craving for vengeance against past injustice drives the left to demand things that would violate this principle. And the atavistic joy of punching Nazis in the face must never overwhelm our sacred commitment to the principles of free speech.

This doesn’t mean we can’t also adopt detailed policy views that align with the left or the right (or both). I for one support single-payer healthcare (left), progressive taxation (left), renewable energy (left), open borders (left), zoning reform (right), reductions in corporate taxes (right), free trade (right, or so I thought?), and a basic income (both—yet strangely we can’t seem to make it happen).

But being a good centrist means that these detailed policy prescriptions are always less important to you than the core principles of democracy itself. When they find out that the rest of the country is against them on something, a leftist or a rightist starts looking for ways to undermine the public will and get the policy they want. A centrist accepts that they have been outvoted and starts looking for ways to persuade the majority that they are mistaken.

Centrism is about defending the guardrails of democracy. False equivalence is not centrism; it is an obstacle to centrism. It prevents us from seeing when one side has clearly damaged those guardrails much more than the other. So let me come out and say it: At this historical juncture, in the United States, the right wing is a far greater threat to the core principles of democracy than the left. This is not to say that the left is inherently incapable of threatening democracy, or never will do so in the future; but it is to say that right here, right now, it’s the right wing we should be worried about. Punching Nazis will never be as threatening to the core of freedom as warrantless wiretaps or the discrediting of the mainstream press.

“DSGE or GTFO”: Macroeconomics took a wrong turn somewhere

Dec 31, JDN 2458119

The state of macro is good,” wrote Oliver Blanchard—in August 2008. This is rather like the turkey who is so pleased with how the farmer has been feeding him lately, the day before Thanksgiving.

It’s not easy to say exactly where macroeconomics went wrong, but I think Paul Romer is right when he makes the analogy between DSGE (dynamic stochastic general equilbrium) models and string theory. They are mathematically complex and difficult to understand, and people can make their careers by being the only ones who grasp them; therefore they must be right! Nevermind if they have no empirical support whatsoever.

To be fair, DSGE models are at least a little better than string theory; they can at least be fit to real-world data, which is better than string theory can say. But being fit to data and actually predicting data are fundamentally different things, and DSGE models typically forecast no better than far simpler models without their bold assumptions. You don’t need to assume all this stuff about a “representative agent” maximizing a well-defined utility function, or an Euler equation (that doesn’t even fit the data), or this ever-proliferating list of “random shocks” that end up taking up all the degrees of freedom your model was supposed to explain. Just regressing the variables on a few years of previous values of each other (a “vector autoregression” or VAR) generally gives you an equally-good forecast. The fact that these models can be made to fit the data well if you add enough degrees of freedom doesn’t actually make them good models. As Von Neumann warned us, with enough free parameters, you can fit an elephant.

But really what bothers me is not the DSGE but the GTFO (“get the [expletive] out”); it’s not that DSGE models are used, but that it’s almost impossible to get published as a macroeconomic theorist using anything else. Defenders of DSGE typically don’t even argue anymore that it is good; they argue that there are no credible alternatives. They characterize their opponents as “dilettantes” who aren’t opposing DSGE because we disagree with it; no, it must be because we don’t understand it. (Also, regarding that post, I’d just like to note that I now officially satisfy the Athreya Axiom of Absolute Arrogance: I have passed my qualifying exams in a top-50 economics PhD program. Yet my enmity toward DSGE has, if anything, only intensified.)

Of course, that argument only makes sense if you haven’t been actively suppressing all attempts to formulate an alternative, which is precisely what DSGE macroeconomists have been doing for the last two or three decades. And yet despite this suppression, there are alternatives emerging, particularly from the empirical side. There are now empirical approaches to macroeconomics that don’t use DSGE models. Regression discontinuity methods and other “natural experiment” designs—not to mention actual experiments—are quickly rising in popularity as economists realize that these methods allow us to actually empirically test our models instead of just adding more and more mathematical complexity to them.

But there still seems to be a lingering attitude that there is no other way to do macro theory. This is very frustrating for me personally, because deep down I think what I would like to do as a career is macro theory: By temperament I have always viewed the world through a very abstract, theoretical lens, and the issues I care most about—particularly inequality, development, and unemployment—are all fundamentally “macro” issues. I left physics when I realized I would be expected to do string theory. I don’t want to leave economics now that I’m expected to do DSGE. But I also definitely don’t want to do DSGE.

Fortunately with economics I have a backup plan: I can always be an “applied micreconomist” (rather the opposite of a theoretical macroeconomist I suppose), directly attached to the data in the form of empirical analyses or even direct, randomized controlled experiments. And there certainly is plenty of work to be done along the lines of Akerlof and Roth and Shiller and Kahneman and Thaler in cognitive and behavioral economics, which is also generally considered applied micro. I was never going to be an experimental physicist, but I can be an experimental economist. And I do get to use at least some theory: In particular, there’s an awful lot of game theory in experimental economics these days. Some of the most exciting stuff is actually in showing how human beings don’t behave the way classical game theory predicts (particularly in the Ultimatum Game and the Prisoner’s Dilemma), and trying to extend game theory into something that would fit our actual behavior. Cognitive science suggests that the result is going to end up looking quite different from game theory as we know it, and with my cognitive science background I may be particularly well-positioned to lead that charge.

Still, I don’t think I’ll be entirely satisfied if I can’t somehow bring my career back around to macroeconomic issues, and particularly the great elephant in the room of all economics, which is inequality. Underlying everything from Marxism to Trumpism, from the surging rents in Silicon Valley and the crushing poverty of Burkina Faso, to the Great Recession itself, is inequality. It is, in my view, the central question of economics: Who gets what, and why?

That is a fundamentally macro question, but you can’t even talk about that issue in DSGE as we know it; a “representative agent” inherently smooths over all inequality in the economy as though total GDP were all that mattered. A fundamentally new approach to macroeconomics is needed. Hopefully I can be part of that, but from my current position I don’t feel much empowered to fight this status quo. Maybe I need to spend at least a few more years doing something else, making a name for myself, and then I’ll be able to come back to this fight with a stronger position.

In the meantime, I guess there’s plenty of work to be done on cognitive biases and deviations from game theory.

Statistics you should have been taught in high school, but probably weren’t

Oct 15, JDN 2458042

Today I’m trying something a little different. This post will assume a lot less background knowledge than most of the others. For some of my readers, this post will probably seem too basic, obvious, even boring. For others, it might feel like a breath of fresh air, relief at last from the overly-dense posts I am generally inclined to write out of Curse of Knowledge. Hopefully I can balance these two effects well enough to gain rather than lose readers.

Here are four core statistical concepts that I think all adults should know, necessary for functional literacy in understanding the never-ending stream of news stories about “A new study shows…” and more generally in applying social science to political decisions. In theory shese should all be taught as part of a core high school curriculum, but typically they either aren’t taught or aren’t retained once students graduate. (Really, I think we should replace one year of algebra with one semester of statistics and one semester of logic. Most people don’t actually need algebra, but they absolutely do need logic and statistics.)

  1. Mean and median

The mean and the median are quite simple concepts, and you’ve probably at least heard of them before, yet confusion between them has caused a great many misunderstandings.

Part of the problem is the word “average”. Normally, the word “average” applies to the mean—for example, a batting average, or an average speed. But in common usage the word “average” can also mean “typical” or “representative”—an average person, an average family. And in many cases, particularly when in comes to economics, the mean is in no way typical or representative.

The mean of a sample of values is just the sum of all those values, divided by the number of values. The mean of the sample {1,2,3,10,1000} is (1+2+3+10+1000)/5 = 203.2

The median of a sample of values is the middle one—order the values, choose the one in the exact center. If you have an even number, take the mean of the two values on either side. So the median of the sample {1,2,3,10,1000} is 3.

I intentionally chose an extreme example: The mean and median of these samples are completely different. But this is something that can happen in real life.

This is vital for understanding the distribution of income, because for almost all countries (and certainly for the world as a whole), the mean income is substantially higher (usually between 50% and 100% higher) than the median income. Yet the mean income is what is reported as “per capita GDP”, but the median income is a much better measure of actual standard of living.

As for the word “average”, it’s probably best to just remove it from your vocabulary. Say “mean” instead if that’s what you intend, or “median” if that’s what you’re using instead.

  1. Standard deviation and mean absolute deviation

Standard deviation is another one you’ve probably seen before.

Standard deviation is kind of a weird concept, honestly. It’s so entrenched in statistics that we’re probably stuck with it, but it’s really not a very good measure of anything intuitively interesting.

Mean absolute deviation is a much more intuitive concept, and much more robust to weird distributions (such as those of incomes and financial markets), but it isn’t as widely used by statisticians for some reason.

The standard deviation is defined as the square root of the mean of the squared differences between the individual values in sample and the mean of that sample. So for my {1,2,3,10,1000} example, the standard deviation is sqrt(((1-203.2)^2 + (2-203.2)^2 + (3-203.2)^2 + (10-203.2)^2 + (1000-203.2)^2)/5) = 398.4.

What can you infer from that figure? Not a lot, honestly. The standard deviation is bigger than the mean, so we have some sense that there’s a lot of variation in our sample. But interpreting exactly what that means is not easy.

The mean absolute deviation is much simpler: It’s the mean of the absolute value of differences between the individual values in a sample and the mean of that sample. In this case it is ((203.2-1) + (203.2-2) + (203.2-3) + (203.2-10) + (1000-203.2))/5 = 318.7.

This has a much simpler interpretation: The mean distance between each value and the mean is 318.7. On average (if we still use that word), each value is about 318.7 away from the mean of 203.2.

When you ask people to interpret a standard deviation, most of them actually reply as if you had asked them about the mean absolute deviation. They say things like “the average distance from the mean”. Only people who know statistics very well and are being very careful would actually say the true answer, “the square root of the sum of squared distances from the mean”.

But there is an even more fundamental reason to prefer the mean absolute deviation, and that is that sometimes the standard deviation doesn’t exist!

For very fat-tailed distributions, the sum that would give you the standard deviation simply fails to converge. You could say the standard deviation is infinite, or that it’s simply undefined. Either way we know it’s fat-tailed, but that’s about all. Any finite sample would have a well-defined standard deviation, but that will keep changing as your sample grows, and never converge toward anything in particular.

But usually the mean still exists, and if the mean exists, then the mean absolute deviation also exists. (In some rare cases even they fail, such as the Cauchy distribution—but actually even then there is usually a way to recover what the mean and mean absolute deviation “should have been” even though they don’t technically exist.)

  1. Standard error

The standard error is even more important for statistical inference than the standard deviation, and frankly even harder to intuitively understand.

The actual definition of the standard error is this: The standard deviation of the distribution of sample means, provided that the null hypothesis is true and the distribution is a normal distribution.

How it is usually used is something more like this: “A good guess of the margin of error on my estimates, such that I’m probably not off by more than 2 standard errors in either direction.”

You may notice that those two things aren’t the same, and don’t even seem particularly closely related. You are correct in noticing this, and I hope that you never forget it. One thing that extensive training in statistics (especially frequentist statistics) seems to do to people is to make them forget that.

In particular, the standard error strictly only applies if the value you are trying to estimate is zero, which usually means that your results aren’t interesting. (To be fair, not always; finding zero effect of minimum wage on unemployment was a big deal.) Using it as a margin of error on your actual nonzero estimates is deeply dubious, even though almost everyone does it for lack of an uncontroversial alternative.
Application of standard errors typically also relies heavily on the assumption of a normal distribution, even though plenty of real-world distributions aren’t normal and don’t even approach a normal distribution in quite large samples. The Central Limit Theorem says that the sampling distribution of the mean of any non-fat-tailed distribution will approach a normal distribution eventually as sample size increases, but it doesn’t say how large a sample needs to be to do that, nor does it apply to fat-tailed distributions.

Therefore, the standard error is really a very conservative estimate of your margin of error; it assumes essentially that the only kind of error you had was random sampling error from a normal distribution in an otherwise perfect randomized controlled experiment. All sorts of other forms of error and bias could have occurred at various stages—and typically, did—making your error estimate inherently too small.

This is why you should never believe a claim that comes from only a single study or a handful of studies. There are simply too many things that could have gone wrong. Only when there are a large number of studies, with varying methodologies, all pointing to the same core conclusion, do we really have good empirical evidence of that conclusion. This is part of why the journalistic model of “A new study shows…” is so terrible; if you really want to know what’s true, you look at large meta-analyses of dozens or hundreds of studies, not a single study that could be completely wrong.

  1. Linear regression and its limits

Finally, I come to linear regression, the workhorse of statistical social science. Almost everything in applied social science ultimately comes down to variations on linear regression.

There is the simplest kind, ordinary least-squares or OLS; but then there is two-stage least-squares 2SLS, fixed-effects regression, clustered regression, random-effects regression, heterogeneous treatment effects, and so on.
The basic idea of all regressions is extremely simple: We have an outcome Y, a variable we are interested in D, and some other variables X.

This might be an effect of education D on earnings Y, or minimum wage D on unemployment Y, or eating strawberries D on getting cancer Y. In our X variables we might include age, gender, race, or whatever seems relevant to Y but can’t be affected by D.

We then make the incredibly bold (and typically unjustifiable) assumption that all the effects are linear, and say that:

Y = A + B*D + C*X + E

A, B, and C are coefficients we estimate by fitting a straight line through the data. The last bit, E, is a random error that we allow to fill in any gaps. Then, if the standard error of B is less than half the size of B itself, we declare that our result is “statistically significant”, and we publish our paper “proving” that D has an effect on Y that is proportional to B.

No, really, that’s pretty much it. Most of the work in econometrics involves trying to find good choices of X that will make our estimates of B better. A few of the more sophisticated techniques involve breaking up this single regression into a few pieces that are regressed separately, in the hopes of removing unwanted correlations between our variable of interest D and our error term E.

What about nonlinear effects, you ask? Yeah, we don’t much talk about those.

Occasionally we might include a term for D^2:

Y = A + B1*D + B2*D^2 + C*X + E

Then, if the coefficient B2 is small enough, which is usually what happens, we say “we found no evidence of a nonlinear effect”.

Those who are a bit more sophisticated will instead report (correctly) that they have found the linear projection of the effect, rather than the effect itself; but if the effect was nonlinear enough, the linear projection might be almost meaningless. Also, if you’re too careful about the caveats on your research, nobody publishes your work, because there are plenty of other people competing with you who are willing to upsell their research as far more reliable than it actually is.

If this process seems rather underwhelming to you, that’s good. I think people being too easily impressed by linear regression is a much more widespread problem than people not having enough trust in linear regression.

Yes, it is possible to go too far the other way, and dismiss even dozens of brilliant experiments as totally useless because they used linear regression; but I don’t actually hear people doing that very often. (Maybe occasionally: The evidence that gun ownership increases suicide and homicide and that corporal punishment harms children is largely based on linear regression, but it’s also quite strong at this point, and I do still hear people denying it.)

Far more often I see people point to a single study using linear regression to prove that blueberries cure cancer or eating aspartame will kill you or yoga cures back pain or reading Harry Potter makes you hate Donald Trump or olive oil prevents Alzheimer’s or psychopaths are more likely to enjoy rap music. The more exciting and surprising a new study is, the more dubious you should be of its conclusions. If a very surprising result is unsupported by many other studies and just uses linear regression, you can probably safely ignore it.

A really good scientific study might use linear regression, but it would also be based on detailed, well-founded theory and apply a proper experimental (or at least quasi-experimental) design. It would check for confounding influences, look for nonlinear effects, and be honest that standard errors are a conservative estimate of the margin of error. Most scientific studies probably should end by saying “We don’t actually know whether this is true; we need other people to check it.” Yet sadly few do, because the publishers that have a strangle-hold on the industry prefer sexy, exciting, “significant” findings to actual careful, honest research. They’d rather you find something that isn’t there than not find anything, which goes against everything science stands for. Until that changes, all I can really tell you is to be skeptical when you read about linear regressions.

Building a wider tent, revisited

Sep 17, JDN 2458014

At a reader’s suggestion, I am expanding upon the argument I made a few weeks ago that political coalitions are strongest when they are willing to accept some disagreement. I made that argument with numbers, which is likely to convince someone like me; but I know that many other people don’t really think that way, so it may help to provide some visuals as well.

60% of this rectangle is filled in red.

Rectangle_1

This represents the proportion of the population that agrees with you on some issue. For concreteness but to avoid making this any more political than it already is, I’m going to pick silly issues. So let’s have this first issue be about which side of the road we should drive on. Let’s say your view is that we should drive on the right. 60% of people agree that we should drive on the right. The other 40% think we should drive on the left.

Now let’s consider another issue. Let’s say this one is about putting pineapples on pizza. You, and 60% of people, agree that pineapples should not be put on pizza. The other 40% think we should put pineapples on pizza.

For now, let’s assume those two issues are independent, that someone’s opinions on driving and pizza are unrelated. Then we can fill 60% of the rectangle in blue, but it should be a perpendicular portion because the two issues aren’t related:

Rectangle_2

Those who agree with you on driving but not pizza (that would include me, by the way) are in red, those who agree with you on pizza but not driving are in blue, those who agree with you on both are in purple, and those who disagree with you on both are in white. You should already be able to see that less than half the population agrees with you on both issues, even though more than half agrees on each.

Let’s add a third issue, which we will color in green. This one can be the question of whether Star Trek is better than Star Wars. Let’s say that 60% of the population agrees with you that Star Trek is better, while 40% think that Star Wars is better. Let’s also assume that this is independent of opinions on both driving and pizza.

Rectangle_3

This is already starting to get unwieldy; there are now eight distinct regions. The white region (8) is comprised of people who disagree with you on everything. The red (6), blue (4), and green (7) regions each have people agree with you on exactly one issue. The blue-green (3), purple (2), and brown (5) regions have people agree with you on two issues. Only those in the dark-green region (1) agree with you on everything.

As you can see, the proportion of people who agree with you on all issues is fairly small, even though the majority of the population agrees with you on any given issue.

If we keep adding issues, this effect gets even stronger. I’m going to change the color-coding now to simplify things. Now, blue will indicate the people who agree with you on all issues, green the people who agree on all but one issue, yellow the people who agree on all but two issues, and red the people who disagree with you on three or more issues.

For three issues, that looks like this, which you can compare to the previous diagram:

Rectangle_4

Now let’s add a fourth issue. Let’s say 60% of people agree with you that socks should not be worn with sandals, but 40% think that socks should be worn with sandals. The blue region gets smaller:

Rectangle_5

How about a fifth issue? Let’s say 60% of people agree with you that cats are better than dogs, while 40% think that dogs are better than cats. The blue region continues to shrink:

Rectangle_6

How about a sixth issue?

Rectangle_7

And finally, a seventh issue?

Rectangle_8

Now the majority of the space is covered by red, meaning that most of the population disagrees with you on at least three issues.

To recap:

By the time there were two issues, the majority of the population disagreed with you on at least one issue.

By the time there were four issues, the majority of the population disagreed with you on at least two issues.

By the time there were seven issues, the majority of the population disagreed with you on at least three issues.

This despite the fact that the majority of the population always agrees with you on any given issue!

If you only welcomed people into your coalition who agree on every single issue (the blue region), you wouldn’t win election if there were even two issues. If you only welcomed those who disagree on at most one (blue or green), you’d stop winning if there were at least four issues. And if there were at least seven issues, you couldn’t even win by allowing those who disagree on at most two issues (blue, green, yellow).

Now, this argument very much does rely upon the different opinions being independent, which in real politics is not the case. So let’s introduce some correlations and see how this changes the result.

Suppose that once someone agrees with you about driving on the right side of the road, they are 90% likely to agree on pizza, Star Trek, sandals, and cats.

Rectangle_9
That makes things look a lot better for you; by including one level of disagreement, you could dominate every election. But notice that even in this case, if you exclude all disagreement, you will continue to lose elections.

With enough issues, even with very strong correlations you can get the same effect. Suppose there are 20 issues, and if you agree on the first one, there is a 99% chance you’ll agree on each of the others. You are still only getting about half the electorate if you don’t allow any disagreement! Due to the very high correlation, if someone disagrees with you on a few things, they usually disagree with you on many things; yet you’re still better off including some disagreement in your coalition.

Rectangle_10

Obviously, you shouldn’t include people in your coalition who actively oppose its core mission. Even if they aren’t actively trying to undermine you, at some point, the disagreement becomes so large that you’ve got to cut them loose. But in a pluralistic democracy, ideological purism is a surefire recipe for electoral failure. You need to allow at least some disagreement.

This isn’t even getting into the possibility that you might be wrong about some issues, and by including those who disagree with you, you may broaden your horizons and correct your mistakes. I’ve thus far assumed you are completely correct and in the majority on every single issue, and yet you still can’t win elections with complex policy mixes unless you include people who disagree with you.

Think of this as a moral recession

August 27, JDN 2457993

The Great Depression was, without doubt, the worst macroeconomic event of the last 200 years. Over 30 million people became unemployed. Unemployment exceeded 20%. Standard of living fell by as much as a third in the United States. Political unrest spread across the world, and the collapsing government of Germany ultimately became the Third Reich and triggered the Second World War If we ignore the world war, however, the effect on mortality rates was surprisingly small. (“Other than that, Mrs. Lincoln, how was the play?”)

And yet, how long do you suppose it took for economic growth to repair the damage? 80 years? 50 years? 30 years? 20 years? Try ten to fifteen. By 1940, the US, US, Germany, and Japan all had a per-capita GDP at least as high as in 1930. By 1945, every country in Europe had a per-capita GDP at least as high as they did before the Great Depression.

The moral of this story is this: Recessions are bad, and can have far-reaching consequences; but ultimately what really matters in the long run is growth.

Assuming the same growth otherwise, a country that had a recession as large as the Great Depression would be about 70% as rich as one that didn’t.

But over 100 years, a country that experienced 3% growth instead of 2% growth would be over two and a half times richer.

Therefore, in terms of standard of living only, if you were given the choice between having a Great Depression but otherwise growing at 3%, and having no recessions but growing at 2%, your grandchildren will be better off if you chose the former. (Of course, given the possibility of political unrest or even war, the depression could very well end up worse.)

With that in mind, I want you to think of the last few years—and especially the last few months—as a moral recession. Donald Trump being President of the United States is clearly a step backward for human civilization, and it seems to have breathed new life into some of the worst ideologies our society has ever harbored, from extreme misogyny, homophobia, right-wing nationalism, and White supremacism to outright Neo-Nazism. When one of the central debates in our public discourse is what level of violence is justifiable against Nazis under what circumstances, something has gone terribly, terribly wrong.

But much as recessions are overwhelmed in the long run by economic growth, there is reason to be confident that this moral backslide is temporary and will be similarly overwhelmed by humanity’s long-run moral progress.

What moral progress, you ask? Let’s remind ourselves.

Just 100 years ago, women could not vote in the United States.

160 years ago, slavery was legal in 15 US states.

Just 3 years ago, same-sex marriage was illegal in 14 US states. Yes, you read that number correctly. I said three. There are gay couples graduating high school and getting married now who as freshmen didn’t think they would be allowed to get married.

That’s just the United States. What about the rest of the world?

100 years ago, almost all of the world’s countries were dictatorships. Today, half of the world’s countries are democracies. Indeed, thanks to India, the majority of the world’s population now lives under democracy.

35 years ago, the Soviet Union still ruled most of Eastern Europe and Northern Asia with an iron fist (or should I say “curtain”?).

30 years ago, the number of human beings in extreme poverty—note I said number, not just rate; the world population was two-thirds what it is today—was twice as large as it is today.

Over the last 65 years, the global death rate due to war has fallen from 250 per million to just 10 per million.

The global literacy rate has risen from 40% to 80% in just 50 years.

World life expectancy has increased by 6 years in just the last 20 years.

We are living in a golden age. Do not forget that.

Indeed, if there is anything that could destroy all these astonishing achievements, I think it would be our failure to appreciate them.

If you listen to what these Neo-Nazi White supremacists say about their grievances, they sound like the spoiled children of millionaires (I mean, they elected one President, after all). They are outraged because they only get 90% of what they want instead of 100%—or even outraged not because they didn’t get what they wanted but because someone else they don’t know also did.

If you listen to the far left, their complaints don’t make much more sense. If you didn’t actually know any statistics, you’d think that life is just as bad for Black people in America today as it was under Jim Crow or even slavery. Well, it’s not even close. I’m not saying racism is gone; it’s definitely still here. But the civil rights movement has made absolutely enormous strides, from banning school segregation and housing redlining to reforming prison sentences and instituting affirmative action programs. Simply the fact that “racist” is now widely considered a terrible thing to be is a major accomplishment in itself. A typical Black person today, despite having only about 60% of the income of a typical White person, is still richer than a typical White person was just 50 years ago. While the 71% high school completion rate Black people currently have may not sound great, it’s much higher than the 50% rate that the whole US population had as recently as 1950.

Yes, there are some things that aren’t going very well right now. The two that I think are most important are climate change and income inequality. As both the global mean temperature anomaly and the world top 1% income share continue to rise, millions of people will suffer and die needlessly from diseases of poverty and natural disasters.

And of course if Neo-Nazis manage to take hold of the US government and try to repeat the Third Reich, that could be literally the worst thing that ever happened. If it triggered a nuclear war, it unquestionably would be literally the worst thing that ever happened. Both these events are unlikely—but not nearly as unlikely as they should be. (Five Thirty Eight interviewed several nuclear experts who estimated a probability of imminent nuclear war at a horrifying five percent.) So I certainly don’t want to make anyone complacent about these very grave problems.

But I worry also that we go too far the other direction, and fail to celebrate the truly amazing progress humanity has made thus far. We hear so often that we are treading water, getting nowhere, or even falling backward, that we begin to feel as though the fight for moral progress is utterly hopeless. If all these centuries of fighting for justice really had gotten us nowhere, the only sensible thing to do at this point would be to give up. But on the contrary, we have made enormous progress in an incredibly short period of time. We are on the verge of finally winning this fight. The last thing we want to do now is give up.