What is the point of democracy?

Apr 9, JDN 2457853

[This topic was chosen by Patreon vote.]

“Democracy” is the sort of word that often becomes just an Applause Light (indeed it was the original example Less Wrong used). Like “freedom” and “liberty” (and for much the same reasons), it’s a good thing, that much we know; but it’s often unclear what is even meant by the word, much less why it should be so important to us.

From another angle, it is strangely common for economists and political scientists to argue that democracy is not all that important; they at least tend to use a precise formal definition of “democracy”, but are oddly quick to dismiss it as pointless or even harmful when it doesn’t line up precisely with their models of an efficient economy or society. I think the best example of this is the so-called “Downs paradox”, where political scientists were so steeped in the tradition of defining all rationality as psychopathic self-interest that they couldn’t even explain why it would occur to anyone to vote. (And indeed, rumor has it that most economists don’t bother to vote, much less campaign politically—which perhaps begins to explain why our economic policy is so terrible.)

Yet especially for Americans in the Trump era, I think it is vital to understand what “democracy” is supposed to mean, and why it is so important.

So, first of all, what is democracy? It is nothing more or less than government by popular vote.

This comes in degrees, of course: The purest direct democracy would have the entire population vote on even the most mundane policies and decisions. You could actually manage something like a monastery or a social club in such a fashion, but this is clearly unworkable on any large scale. Even once you get to hundreds of people, much less thousands or millions, it becomes unviable. The closest example I’ve seen is Switzerland, where there are always numerous popular referenda on ballots that are voted on by entire regions or the entire country—and even then, Switzerland does have representatives that make many of the day-to-day decisions.

So in practice all large-scale democratic systems are some degree of representative democracy, or republic, where some especially decisions may be made by popular vote, but most policies are made by elected representatives, staff appointed by those representatives, or even career civil servants who are appointed in a nominally apolitical process not so different from private-sector hiring. In the most extreme cases such civil servants can become so powerful that you get a deep state, where career bureaucrats exercise more power than elected officials—at that point I think you have actually lost the right to really call yourself a “democracy” and have become something more like a technocracy.
Yet of course a country can get even more undemocratic than that, and many are, governed by an aristocracy or oligarchy that vests power in a small number of wealthy and powerful individuals, or monarchy or autocracy that gives near-absolute power to a single individual.

Thus, there is a continuum of most to least democratic, with popular vote at one end, followed by elected representatives, followed by appointed civil servants, followed by a handful of oligarchs, and ultimately the most undemocratic system is an autocracy controlled by a single individual.

I also think it’s worth mentioning that constitutional monarchies with strong parliamentary systems, like the United Kingdom and Norway, are also “democracies” in the sense I intend. Yes, technically they have these hereditary monarchs—but in practice, the vast majority of the state’s power is vested in the votes of its people. Indeed, if we separate out parliamentary constitutional monarchy from presidential majoritarian democracy and compare them, the former might actually turn out to be better. Certainly, some of the world’s most prosperous nations are governed that way.

As I’ve already acknowledge, the very far extreme of pure direct democracy is unfeasible. But why would we want to get closer to that end? Why be like Switzerland or Denmark rather than like Turkey or Russia—or for that matter why be like California rather than like Mississippi?
Well, if you know anything about the overall welfare of these states, it almost seems obvious—Switzerland and Denmark are richer, happier, safer, healthier, more peaceful, and overall better in almost every way than Turkey and Russia. The gap between California and Mississippi is not as large, but it is larger than most people realize. Median household income in California is $64,500; in Mississippi it is only $40,593. Both are still well within the normal range of a highly-developed country, but that effectively makes California richer than Luxembourg but Mississippi poorer than South Korea. But perhaps the really stark comparison to make is life expectancy: Life expectancy at birth in California is almost 81 years, while in Mississippi it’s only 75.

Of course, there are a lot of other differences between states besides how much of their governance is done by popular referendum. Simply making Mississippi decide more things by popular vote would not turn it into California—much less would making Turkey more democratic turn it into Switzerland. So we shouldn’t attribute these comparisons entirely to differences in democracy. Indeed, a pair of two-way comparisons is only in the barest sense a statistical argument; we should be looking at dozens if not hundreds of comparisons if we really want to see the effects of democracy. And we should of course be trying to control for other factors, adjust for country fixed-effects, and preferably use natural experiments or instrumental variables to tease out causality.

Yet such studies have in fact been done. Stronger degrees of democracy appear to improve long-run economic growth, as well as reduce corruption, increase free trade, protect peace, and even improve air quality.

Subtler analyses have compared majoritarian versus proportional systems (where proportional seems, to me, at least, more democratic), as well as different republican systems with stronger or weaker checks and balances (stronger is clearly better, though whether that is “more democratic” is at least debatable). The effects of democracy on income distribution are more complicated, probably because there have been some highly undemocratic socialist regimes.

So, the common belief that democracy is good seems to be pretty well supported by the data. But why is democracy good? Is it just a practical matter of happening to get better overall results? Could it one day be overturned by some superior system such as technocracy or a benevolent autocratic AI?

Well, I don’t want to rule out the possibility of improving upon existing systems of government. Clearly new systems of government have in fact emerged over the course of history—Greek “democracy” and Roman “republic” were both really aristocracy, and anything close to universal suffrage didn’t really emerge on a large scale until the 20th century. So the 21st (or 22nd) century could well devise a superior form of government we haven’t yet imagined.
However, I do think there is good reason to believe that any new system of government that actually manages to improve upon democracy will still resemble democracy, because there are three key features democracy has that other systems of government simply can’t match. It is these three features that make democracy so important and so worth fighting for.

1. Everyone’s interests are equally represented.

Perhaps no real system actually manages to represent everyone’s interests equally, but the more democratic a system is, the better it will conform to this ideal. A well-designed voting system can aggregate the interests of an entire population and choose the course of action that creates the greatest overall benefit.

Markets can also be a good system for allocating resources, but while markets represent everyone’s interests, they do so highly unequally. Rich people are quite literally weighted more heavily in the sum.

Most systems of government do even worse, by completely silencing the voices of the majority of the population. The notion of a “benevolent autocracy” is really a conceit; what makes you think you could possibly keep the autocrat benevolent?

This is also why any form of disenfranchisement is dangerous and a direct attack upon democracy. Even if people are voting irrationally, against their own interests and yours, by silencing their voice you are undermining the most fundamental tenet of democracy itself. All voices must be heard, no exceptions. That is democracy’s fundamental strength.

2. The system is self-correcting.

This may more accurately describe a constitutional republican system with strong checks and balances, but that is what most well-functioning democracies have and it is what I recommend. If you conceive of “more democracy” as meaning that people can vote their way into fascism by electing a sufficiently charismatic totalitarian, then I do not want us to have “more democracy”. But just as contracts and regulations that protect you can make you in real terms more free because you can now safely do things you otherwise couldn’t risk, I consider strong checks and balances that maintain the stability of a republic against charismatic fascists to be in a deeper sense more democratic. This is ultimately semantic; I think I’ve made it clear enough that I want strong checks and balances.

With such checks and balances in place, democracies may move slower than autocracies; they may spend more time in deliberation or even bitter, polarized conflict. But this also means that their policies do not lurch from one emperor’s whim to another, and they are stable against being overtaken by corruption or fascism. Their policies are stable and predictable; their institutions are strong and resilient.

No other system of government yet devised by humans has this kind of stability, which may be why democracies are gradually taking over the world. Charismatic fascism fails when the charismatic leader dies; hereditary monarchy collapses when the great-grandson of the great king is incompetent; even oligarchy and aristocracy, which have at least some staying power, ultimately fall apart when the downtrodden peasants ultimately revolt. But democracy abides, for where monarchy and aristocracy are made of families and autocracy and fascism are made of a single man, democracy is made of principles and institutions. Democracy is evolutionarily stable, and thus in Darwinian terms we can predict it will eventually prevail.

3. The coercion that government requires is justified.

All government is inherently coercive. Libertarians are not wrong about this. Taxation is coercive. Regulation is coercive. Law is coercive. (The ones who go on to say that all government is “death threats” or “slavery” are bonkers, mind you. But it is in fact coercive.)

The coercion of government is particularly terrible if that coercion is coming from a system like an autocracy, where the will of the people is minimally if at all represented in the decisions of policymakers. Then that is a coercion imposed from outside, a coercion in the fullest sense, one person who imposes their will upon another.

But when government coercion comes from a democracy, it takes on a fundamentally different meaning. Then it is not they who coerce us—it is we who coerce ourselves. Now, why in the world would you coerce yourself? It seems ridiculous, doesn’t it?

Not if you know any game theory. There are in fall all sorts of reasons why one might want to coerce oneself, and two in particular become particularly important for the justification of democratic government.

The first and most important is collective action: There are many situations in which people all working together to accomplish a goal can be beneficial to everyone, but nonetheless any individual person who found a way to shirk their duty and not contribute could benefit even more. Anyone who has done a group project in school with a couple of lazy students in it will know this experience: You end up doing all the work, but they still get a good grade at the end. If everyone had taken the rational, self-interested action of slacking off, everyone in the group would have failed the project.

Now imagine that the group project we’re trying to achieve is, say, defending against an attack by Imperial Japan. We can’t exactly afford to risk that project falling through. So maybe we should actually force people to support it—in the form of taxes, or even perhaps a draft (as ultimately we did in WW2). Then it is no longer rational to try to shirk your duty, so everyone does their duty, the project gets done, and we’re all better off. How do we decide which projects are important enough to justify such coercion? We vote, of course. This is the most fundamental justification of democratic government.

The second that is relevant for government is commitment. There are many circumstances in which we want to accomplish something in the future, and from a long-run perspective it makes sense to achieve that goal—but then when the time comes to take action, we are tempted to procrastinate or change our minds. How can we resolve such a dilemma? Well, one way is to tie our own hands—to coerce ourselves into carrying out the necessary task we are tempted to avoid or delay.

This applies to many types of civil and criminal law, particularly regarding property ownership. Murder is a crime that most people would not commit even if it were completely legal. But shoplifting? I think if most people knew there would be no penalty for petty theft and retail fraud they would be tempted into doing it at least on occasion. I doubt it would be frequent enough to collapse our entire economic system, but it would introduce a lot of inefficiency, and make almost everything more expensive. By having laws in place that punish us for such behavior, we have a way of defusing such temptations, at least for most people most of the time. This is not as important for the basic functioning of government as is collective action, but I think it is still important enough to be worthy of mention.

Of course, there will always be someone who disagrees with any given law, regardless of how sensible and well-founded that law may be. And while in some sense “we all” agreed to pay these taxes, when the IRS actually demands that specific dollar amount from you, it may well be an amount that you would not have chosen if you’d been able to set our entire tax system yourself. But this is a problem of aggregation that I think may be completely intractable; there’s no way to govern by consensus, because human beings just can’t achieve consensus on the scale of millions of people. Governing by popular vote and representation is the best alternative we’ve been able to come up with. If and when someone devises a system of government that solves that problem and represents the public will even better than voting, then we will have a superior alternative to democracy.

Until then, it is as Churchill said: “Democracy is the worst form of government, except for all the others.”

Selling debt goes against everything the free market stands for

JDN 2457555

I don’t think most people—or even most economists—have any concept of just how fundamentally perverse and destructive our financial system has become, and a large chunk of it ultimately boils down to one thing: Selling debt.

Certainly collateralized debt obligations (CDOs), and their meta-form, CDO2s (pronounced “see-dee-oh squareds”), are nothing more than selling debt, and along with credit default swaps (CDS; they are basically insurance, but without those pesky regulations against things like fraud and conflicts of interest) they were directly responsible for the 2008 financial crisis and the ensuing Great Recession and Second Depression.

But selling debt continues in a more insidious way, underpinning the entire debt collection industry which raises tens of billions of dollars per year by harassment, intimidation and extortion, especially of the poor and helpless. Frankly, I think what’s most shocking is how little money they make, given the huge number of people they harass and intimidate.

John Oliver did a great segment on debt collections (with a very nice surprise at the end):

But perhaps most baffling to me is the number of people who defend the selling of debt on the grounds that it is a “free market” activity which must be protected from government “interference in personal liberty”. To show this is not a strawman, here’s the American Enterprise Institute saying exactly that.

So let me say this in no uncertain terms: Selling debt goes against everything the free market stands for.

One of the most basic principles of free markets, one of the founding precepts of capitalism laid down by no less than Adam Smith (and before him by great political philosophers like John Locke), is the freedom of contract. This is the good part of capitalism, the part that makes sense, the reason we shouldn’t tear it all down but should instead try to reform it around the edges.

Indeed, the freedom of contract is so fundamental to human liberty that laws can only be considered legitimate insofar as they do not infringe upon it without a compelling public interest. Freedom of contract is right up there with freedom of speech, freedom of the press, freedom of religion, and the right of due process.

The freedom of contract is the right to make agreements, including financial agreements, with anyone you please, and under conditions that you freely and rationally impose in a state of good faith and transparent discussion. Conversely, it is the right not to make agreements with those you choose not to, and to not be forced into agreements under conditions of fraud, intimidation, or impaired judgment.

Freedom of contract is the basis of my right to take on debt, provided that I am honest about my circumstances and I can find a lender who is willing to lend to me. So taking on debt is a fundamental part of freedom of contract.

But selling debt is something else entirely. Far from exercising the freedom of contract, it violates it. When I take out a loan from bank A, and then they turn around and sell that loan to bank B, I suddenly owe money to bank B, but I never agreed to do that. I had nothing to do with their decision to work with bank B as opposed to keeping the loan or selling it to bank C.

Current regulations prohibit banks from “changing the terms of the loan”, but in practice they change them all the time—they can’t change the principal balance, the loan term, or the interest rate, but they can change the late fees, the payment schedule, and lots of subtler things about the loan that can still make a very big difference. Indeed, as far as I’m concerned they have changed the terms of the loan—one of the terms of the loan was that I was to pay X amount to bank A, not that I was to pay X amount to bank B. I may or may not have good reasons not to want to pay bank B—they might be far less trustworthy than bank A, for instance, or have a far worse social responsibility record—and in any case it doesn’t matter; it is my choice whether or not I want anything to do with bank B, whatever my reasons might be.

I take this matter quite personally, for it is by the selling of debt that, in moral (albeit not legal) terms, a British bank stole my parents’ house. Indeed, not just any British bank; it was none other than HSBC, the money launderers for terrorists.

When they first obtained their mortgage, my parents did not actually know that HSBC was quite so evil as to literally launder money for terrorists, but they did already know that they were involved in a great many shady dealings, and even specifically told their lender that they did not want the loan sold, and if it was to be sold, it was absolutely never to be sold to HSBC in particular. Their mistake (which was rather like the “mistake” of someone who leaves their car unlocked and has it stolen, or forgets to arm the home alarm system and suffers a burglary) was not to get this written into the formal contract, rather than simply made as a verbal agreement with the bankers. Such verbal contracts are enforceable under the law, at least in theory; but that would require proof of the verbal contract (and what proof could we provide?), and also probably have cost as much as the house in litigation fees.

Oh, by the way, they were given a subprime interest rate of 8% despite being middle-class professionals with good credit, no doubt to maximize the broker’s closing commission. Most banks reserved such behavior for racial minorities, but apparently this one was equal-opportunity in the worst way.Perhaps my parents were naive to trust bankers any further than they could throw them.

As a result, I think you know what happened next: They sold the loan to HSBC.

Now, had it ended there, with my parents unwittingly forced into supporting a bank that launders money for terrorists, that would have been bad enough. But it assuredly did not.

By a series of subtle and manipulative practices that poked through one loophole after another, HSBC proceeded to raise my parents’ payments higher and higher. One particularly insidious tactic they used was to sit on the checks until just after the due date passed, so they could charge late fees on the payments, then they recapitalized the late fees. My parents caught on to this particular trick after a few months, and started mailing the checks certified so they would be date-stamped; and lo and behold, all the payments were suddenly on time! By several other similarly devious tactics, all of which were technically legal or at least not provable, they managed to raise my parents’ monthly mortgage payments by over 50%.

Note that it was a fixed-rate, fixed-term mortgage. The initial payments—what should have been always the payments, that’s the point of a fixed-rate fixed-term mortgage—were under $2000 per month. By the end they were paying over $3000 per month. HSBC forced my parents to overpay on a mortgage an amount equal to the US individual poverty line, or the per-capita GDP of Peru.

They tried to make the payments, but after being wildly over budget and hit by other unexpected expenses (including defects in the house’s foundation that they had to pay to fix, but because of the “small” amount at stake and the overwhelming legal might of the construction company, no lawyer was willing to sue over), they simply couldn’t do it anymore, and gave up. They gave the house to the bank with a deed in lieu of foreclosure.

And that is the story of how a bank that my parents never agreed to work with, never would have agreed to work with, indeed specifically said they would not work with, still ended up claiming their house—our house, the house I grew up in from the age of 12. Legally, I cannot prove they did anything against the law. (I mean, other than laundered money for terrorists.) But morally, how is this any less than theft? Would we not be victimized less had a burglar broken into our home, vandalized the walls and stolen our furniture?

Indeed, that would probably be covered under our insurance! Where can I buy insurance against the corrupt and predatory financial system? Where are my credit default swaps to pay me when everything goes wrong?

And all of this could have been prevented, if banks simply weren’t allowed to violate our freedom of contract by selling their loans to other banks.

Indeed, the Second Depression could probably have been likewise prevented. Without selling debt, there is no securitization. Without securitization, there is far less leverage. Without leverage, there are not bank failures. Without bank failures, there is no depression. A decade of global economic growth was lost because we allowed banks to sell debt whenever they please.

I have heard the counter-arguments many times:

“But what if banks need the liquidity?” Easy. They can take out their own loans with those other banks. If bank A finds they need more cashflow, they should absolutely feel free to take out a loan from bank B. They can even point to their projected revenues from the mortgage payments we owe them, as a means of repaying that loan. But they should not be able to involve us in that transaction. If you want to trust HSBC, that’s your business (you’re an idiot, but it’s a free country). But you have no right to force me to trust HSBC.

“But banks might not be willing to make those loans, if they knew they couldn’t sell or securitize them!” THAT’S THE POINT. Banks wouldn’t take on all these ridiculous risks in their lending practices that they did (“NINJA loans” and mortgages with payments larger than their buyers’ annual incomes), if they knew they couldn’t just foist the debt off on some Greater Fool later on. They would only make loans they actually expect to be repaid. Obviously any loan carries some risk, but banks would only take on risks they thought they could bear, as opposed to risks they thought they could convince someone else to bear—which is the definition of moral hazard.

“Homes would be unaffordable if people couldn’t take out large loans!” First of all, I’m not against mortgages—I’m against securitization of mortgages. Yes, of course, people need to be able to take out loans. But they shouldn’t be forced to pay those loans to whoever their bank sees fit. If indeed the loss of subprime securitized mortgages made it harder for people to get homes, that’s a problem; but the solution to that problem was never to make it easier for people to get loans they can’t afford—it is clearly either to reduce the price of homes or increase the incomes of buyers. Subsidized housing construction, public housing, changes in zoning regulation, a basic income, lower property taxes, an expanded earned-income tax credit—these are the sort of policies that one implements to make housing more affordable, not “go ahead and let banks exploit people however they want”.

Remember, a regulation against selling debt would protect the freedom of contract. It would remove a way for private individuals and corporations to violate that freedom, like regulations against fraud, intimidation, and coercion. It should be uncontroversial that no one has any right to force you to do business with someone you would not voluntarily do business with, certainly not in a private transaction between for-profit corporations. Maybe that sort of mandate makes sense in rare circumstances by the government, but even then it should really be implemented as a tax, not a mandate to do business with a particular entity. The right to buy what you choose is the foundation of a free market—and implicit in it is the right not to buy what you do not choose.

There are many regulations on debt that do impose upon freedom of contract: As horrific as payday loans are, if someone really honestly knowingly wants to take on short-term debt at 400% APR I’m not sure it’s my business to stop them. And some people may really be in such dire circumstances that they need money that urgently and no one else will lend to them. Insofar as I want payday loans regulated, it is to ensure that they are really lending in good faith—as many surely are not—and ultimately I want to outcompete them by providing desperate people with more reasonable loan terms. But a ban on securitization is like a ban on fraud; it is the sort of law that protects our rights.

The surprising honesty of politicians

JDN 2457509

The stereotype that politicians are dishonest is so strong that many people use “honest politician” as an example of an oxymoron. There is a sense that politicians never keep their campaign promises, so what they say is basically just meaningless noise.

This impression could scarcely be further from the truth. Politicians are quite honest, and they usually try to keep their campaign promises. On average, about 2/3 of campaign promises are kept. Most of those that aren’t are largely given up under heavy opposition, not simply ignored because they weren’t real objectives. Politicians are distrusted, while clergy are trusted—despite the fact that clergy quite literally make their entire career out of selling beliefs that are demonstrably false and in most cases outright absurd.

Along similar lines, most people seem to have an impression that democracy is largely a show, and powerful oligarchs make most of the real decisions behind the scenes—even Jimmy Carter has been saying this recently. While there is evidence that the rich have disproportionate power over politicians, this is largely only true of Republicans; and furthermore the theory that democracy is meaningless can’t explain two rather important facts:

1. Economic prosperity is strongly correlated with democracy—more strongly correlated than most economists believed until quite recently. Even the “Miracle of Chile” didn’t actually occur when Pinochet reformed the economy—it occurred in the 1990s, after Pinochet ceded power to a democratic government. Stronger democracy is also strongly linked to better education, though surprisingly has little correlation with inequality.

2. Democratic states almost never go to war with one another. Democracies go to war with non-democracies, and non-democracies go to war with one another; but with a few exceptions (and largely limited to young, unstable democracies), democracies do not go to war with other democracies.

If democracy meant nothing, and were all just a sideshow that the elites use to manipulate us, these results would simply be impossible. If voting did not actually shape policy in some fashion, policy outcomes for democracies and non-democracies would have to be identical. In fact they are wildly different, so different it’s actually kind of hard to explain. Apparently similar policies simply seem to work better when they are implemented by democracies—perhaps because in order to be passed in the first place they must have a certain amount of buy-in from the population.

In fact, politicians are more honest than we’d expect them to be based on the incentives provided by elections—they seem to either be acting out of genuine altruism or to advance their reputation in other ways.

Neoclassical economic theory actually has trouble explaining why politicians are so honest—which may have something to do with the fact that politicians who were trained as neoclassical economists are more likely to be corrupt. A similar effect holds for undergraduate students in experiments. Teaching people that human beings are infinite identical psychopaths seems to make them behave a bit more like psychopaths! (Though some of this may also be selection bias: Psychopaths may find economics appealing either because the ideology justifies their behavior or because it’s a pretty lucrative field.)

Part of this false impression clearly comes from the media, and from politicians slandering each other. Hillary Clinton has an almost impeccable fact-check rating—comparable to or arguably even better than Bernie Sanders and John Kasich, both of whom have majority “Mostly True” or “True” ratings. All three are miles ahead of Donald Trump and Ted Cruz, both of whom are over 60% “Mostly False”, “False”, or “Pants on Fire” (the latter is 18% of what Donald Trump says). And yet, Hillary Clinton is widely perceived as dishonest and Donald Trump is widely perceived as “speaking his mind”. Maybe people think Trump is honest because he keeps saying he is. Or maybe it’s because he’s honest about his horrible motivations, even though he gets most of the facts wrong.

These facts should give us hope! Our votes are not meaningless, and our voices do make a difference. We are right to be obsessed with keeping our politicians honest—but it’s time we recognize that it’s working. We are doing something right. If we can figure out what it is, maybe we can do even better.The last thing we want to do right now is throw up our hands and give up.

What really happened in Greece

JDN 2457506

I said I’d get back to this issue, so here goes.

Let’s start with what is uncontroversial: Greece is in trouble.

Their per-capita GDP PPP has fallen from a peak of over $32,000 in 2007 to a trough of just over $24,000 in 2013, and only just began to recover over the last 2 years. That’s a fall of 29 log points. Put another way, the average person in Greece has about the same real income now that they had in the year 2000—a decade and a half of economic growth disappeared.

Their unemployment rate surged from about 7% in 2007 to almost 28% in 2013. It remains over 24%. That is, almost one quarter of all adults in Greece are seeking jobs and not finding them. The US has not seen an unemployment rate that high since the Great Depression.

Most shocking of all, over 40% of the population in Greece is now below the national poverty line. They define poverty as 60% of the inflation-adjusted average income in 2009, which works out to 665 Euros per person ($756 at current exchange rates) per month, or about $9000 per year. They also have an absolute poverty line, which 14% of Greeks now fall below, but only 2% did before the crash.

So now, let’s talk about why.

There’s a standard narrative you’ve probably heard many times, which goes something like this:

The Greek government spent too profligately, heaping social services on the population without the tax base to support them. Unemployment insurance was too generous; pensions were too large; it was too hard to fire workers or cut wages. Thus, work incentives were too weak, and there was no way to sustain a high GDP. But they refused to cut back on these social services, and as a result went further and further into debt until it finally became unsustainable. Now they are cutting spending and raising taxes like they needed to, and it will eventually allow them to repay their debt.

Here’s a fellow of the Cato Institute spreading this narrative on the BBC. Here’s ABC with a five bullet-point list: Pension system, benefits, early retirement, “high unemployment and work culture issues” (yes, seriously), and tax evasion. Here the Telegraph says that Greece “went on a spending spree” and “stopped paying taxes”.

That story is almost completely wrong. Almost nothing about it is true. Cato and the Telegraph got basically everything wrong. The only one ABC got right was tax evasion.

Here’s someone else arguing that Greece has a problem with corruption and failed governance; there is something to be said for this, as Greece is fairly corrupt by European standards—though hardly by world standards. For being only a generation removed from an authoritarian military junta, they’re doing quite well actually. They’re about as corrupt as a typical upper-middle income country like Libya or Botswana; and Botswana is widely regarded as the shining city on a hill of transparency as far as Sub-Saharan Africa is concerned. So corruption may have made things worse, but it can’t be the whole story.

First of all, social services in Greece were not particularly extensive compared to the rest of Europe.

Before the crisis, Greece’s government spending was about 44% of GDP.

That was about the same as Germany. It was slightly more than the UK. It was less than Denmark and France, both of which have government spending of about 50% of GDP.

Greece even tried to cut spending to pay down their debt—it didn’t work, because they simply ended up worsening the economic collapse and undermining the tax base they needed to do that.

Europe has fairly extensive social services by world standards—but that’s a major part of why it’s the First World. Even the US, despite spending far less than Europe on social services, still spends a great deal more than most countries—about 36% of GDP.

Second, if work incentives were a problem, you would not have high unemployment. People don’t seem to grasp what the word unemployment actually means, which is part of why I can’t stand it when news outlets just arbitrarily substitute “jobless” to save a couple of syllables. Unemployment does not mean simply that you don’t have a job. It means that you don’t have a job and are trying to get one.

The word you’re looking for to describe simply not having a job is nonemployment, and that’s such a rarely used term my spell-checker complains about it. Yet economists rarely use this term precisely because it doesn’t matter; a high nonemployment rate is not a symptom of a failing economy but a result of high productivity moving us toward the post-scarcity future (kicking and screaming, evidently). If the problem with Greece were that they were too lazy and they retire too early (which is basically what ABC was saying in slightly more polite language), there would be high nonemployment, but there would not be high unemployment. “High unemployment and work culture issues” is actually a contradiction.

Before the crisis, Greece had an employment-to-population ratio of 49%, meaning a nonemployment rate of 51%. If that sounds ludicrously high, you’re not accustomed to nonemployment figures. During the same time, the United States had an employment-to-population ratio of 52% and thus a nonemployment rate of 48%. So the number of people in Greece who were voluntarily choosing to drop out of work before the crisis was just slightly larger than the number in the US—and actually when you adjust for the fact that the US is full of young immigrants and Greece is full of old people (their median age is 10 years older than ours), it begins to look like it’s we Americans who are lazy. (Actually, it’s that we are studious—the US has an extremely high rate of college enrollment and the best colleges in the world. Full-time students are nonemployed, but they are certainly not unemployed.)

But Greece does have an enormously high debt, right? Yes—but it was actually not as bad before the crisis. Their government debt surged from 105% of GDP to almost 180% today. 105% of GDP is about what we have right now in the US; it’s less than what we had right after WW2. This is a little high, but really nothing to worry about, especially if you’ve incurred the debt for the right reasons. (The famous paper by Rogart and Reinhoff arguing that 90% of GDP is a horrible point of no return was literally based on math errors.)

Moreover, Ireland and Spain suffered much the same fate as Greece, despite running primary budget surpluses.

So… what did happen? If it wasn’t their profligate spending that put them in this mess, what was it?

Well, first of all, there was the Second Depression, a worldwide phenomenon triggered by the collapse of derivatives markets in the United States. (You want unsustainable debt? Try 20 to 1 leveraged CDO-squareds and one quadrillion dollars in notional value. Notional value isn’t everything, but it’s a lot.) So it’s mainly our fault, or rather the fault of our largest banks. As far as us voters, it’s “our fault” in the way that if your car gets stolen it’s “your fault” for not locking the doors and installing a LoJack. We could have regulated against this and enforced those regulations, but we didn’t. (Fortunately, Dodd-Frank looks like it might be working.)

Greece was hit particularly hard because they are highly dependent on trade, particularly in services like tourism that are highly sensitive to the business cycle. Before the crash they imported 36% of GDP and exported 23% of GDP. Now they import 35% of GDP and export 33% of GDP—but it’s a much smaller GDP. Their exports have only slightly increased while their imports have plummeted. (This has reduced their “trade deficit”, but that has always been a silly concept. I guess it’s less silly if you don’t control your own currency, but it’s still silly.)

Once the crash happened, the US had sovereign monetary policy and the wherewithal to actually use that monetary policy effectively, so we weathered the crash fairly well, all things considered. Our unemployment rate barely went over 10%. But Greece did not have sovereign monetary policy—they are tied to the Euro—and that severely limited their options for expanding the money supply as a result of the crisis. Raising spending and cutting taxes was the best thing they could do.

But the bank(st?)ers and their derivatives schemes caused the Greek debt crisis a good deal more directly than just that. Part of the condition of joining the Euro was that countries must limit their fiscal deficit to no more than 3% of GDP (which is a totally arbitrary figure with no economic basis in case you were wondering). Greece was unwilling or unable to do so, but wanted to look like they were following the rules—so they called up Goldman Sachs and got them to make some special derivatives that Greece could use to continue borrowing without looking like they were borrowing. The bank could have refused; they could have even reported it to the European Central Bank. But of course they didn’t; they got their brokerage fee, and they knew they’d sell it off to some other bank long before they had to worry about whether Greece could ever actually repay it. And then (as I said I’d get back to in a previous post) they paid off the credit rating agencies to get them to rate these newfangled securities as low-risk.

In other words, Greece is not broke; they are being robbed.

Like homeowners in the US, Greece was offered loans they couldn’t afford to pay, but the banks told them they could, because the banks had lost all incentive to actually bother with the question of whether loans can be repaid. They had “moved on”; their “financial innovation” of securitization and collateralized debt obligations meant that they could collect origination fees and brokerage fees on loans that could never possibly be repaid, then sell them off to some Greater Fool down the line who would end up actually bearing the default. As long as the system was complex enough and opaque enough, the buyers would never realize the garbage they were getting until it was too late. The entire concept of loans was thereby broken: The basic assumption that you only loan money you expect to be repaid no longer held.

And it worked, for awhile, until finally the unpayable loans tried to create more money than there was in the world, and people started demanding repayment that simply wasn’t possible. Then the whole scheme fell apart, and banks began to go under—but of course we saved them, because you’ve got to save the banks, how can you not save the banks?

Honestly I don’t even disagree with saving the banks, actually. It was probably necessary. What bothers me is that we did nothing to save everyone else. We did nothing to keep people in their homes, nothing to stop businesses from collapsing and workers losing their jobs. Precisely because of the absurd over-leveraging of the financial system, the cost to simply refinance every mortgage in America would have been less than the amount we loaned out in bank bailouts. The banks probably would have done fine anyway, but if they didn’t, so what? The banks exist to serve the people—not the other way around.

We can stop this from happening again—here in the US, in Greece, in the rest of Europe, everywhere. But in order to do that we must first understand what actually happened; we must stop blaming the victims and start blaming the perpetrators.

The credit rating agencies to be worried about aren’t the ones you think

JDN 2457499

John Oliver is probably the best investigative journalist in America today, despite being neither American nor officially a journalist; last week he took on the subject of credit rating agencies, a classic example of his mantra “If you want to do something evil, put it inside something boring.” (note that it’s on HBO, so there is foul language):

As ever, his analysis of the subject is quite good—it’s absurd how much power these agencies have over our lives, and how little accountability they have for even assuring accuracy.

But I couldn’t help but feel that he was kind of missing the point. The credit rating agencies to really be worried about aren’t Equifax, Experian, and Transunion, the ones that assess credit ratings on individuals. They are Standard & Poor’s, Moody’s, and Fitch (which would have been even easier to skewer the way John Oliver did—perhaps we can get them confused with Standardly Poor, Moody, and Filch), the agencies which assess credit ratings on institutions.

These credit rating agencies have almost unimaginable power over our society. They are responsible for rating the risk of corporate bonds, certificates of deposit, stocks, derivatives such as mortgage-backed securities and collateralized debt obligations, and even municipal and government bonds.

S&P, Moody’s, and Fitch don’t just rate the creditworthiness of Goldman Sachs and J.P. Morgan Chase; they rate the creditworthiness of Detroit and Greece. (Indeed, they played an important role in the debt crisis of Greece, which I’ll talk about more in a later post.)

Moreover, they are proven corrupt. It’s a matter of public record.

Standard and Poor’s is the worst; they have been successfully sued for fraud by small banks in Pennsylvania and by the State of New Jersey; they have also settled fraud cases with the Securities and Exchange Commission and the Department of Justice.

Moody’s has also been sued for fraud by the Department of Justice, and all three have been prosecuted for fraud by the State of New York.

But in fact this underestimates the corruption, because the worst conflicts of interest aren’t even illegal, or weren’t until Dodd-Frank was passed in 2010. The basic structure of this credit rating system is fundamentally broken; the agencies are private, for-profit corporations, and they get their revenue entirely from the banks that pay them to assess their risk. If they rate a bank’s asset as too risky, the bank stops paying them, and instead goes to another agency that will offer a higher rating—and simply the threat of doing so keeps them in line. As a result their ratings are basically uncorrelated with real risk—they failed to predict the collapse of Lehman Brothers or the failure of mortgage-backed CDOs, and they didn’t “predict” the European debt crisis so much as cause it by their panic.

Then of course there’s the fact that they are obviously an oligopoly, and furthermore one that is explicitly protected under US law. But then it dawns upon you: Wait… US law? US law decides the structure of credit rating agencies that set the bond rates of entire nations? Yes, that’s right. You’d think that such ratings would be set by the World Bank or something, but they’re not; in fact here’s a paper published by the World Bank in 2004 about how rather than reform our credit rating system, we should instead tell poor countries to reform themselves so they can better impress the private credit rating agencies.

In fact the whole concept of “sovereign debt risk” is fundamentally defective; a country that borrows in its own currency should never have to default on debt under any circumstances. National debt is almost nothing like personal or corporate debt. Their fears should be inflation and unemployment—their monetary policy should be set to minimize the harm of these two basic macroeconomic problems, understanding that policies which mitigate one may enflame the other. There is such a thing as bad fiscal policy, but it has nothing to do with “running out of money to pay your debt” unless you are forced to borrow in a currency you can’t control (as Greece is, because they are on the Euro—their debt is less like the US national debt and more like the debt of Puerto Rico, which is suffering an ongoing debt crisis you may not have heard about). If you borrow in your own currency, you should be worried about excessive borrowing creating inflation and devaluing your currency—but not about suddenly being unable to repay your creditors. The whole concept of giving a sovereign nation a credit rating makes no sense. You will be repaid on time and in full, in nominal terms; if inflation or currency exchange has devalued the currency you are repaid in, that’s sort of like a partial default, but it’s a fundamentally different kind of “default” than simply not paying back the money—and credit ratings have no way of capturing that difference.

In particular, it makes no sense for interest rates on government bonds to go up when a country is suffering some kind of macroeconomic problem.

The basic argument for why interest rates go up when risk is higher is that lenders expect to be paid more by those who do pay to compensate for what they lose from those who don’t pay. This is already much more problematic than most economists appreciate; I’ve been meaning to write a paper on how this system creates self-fulfilling prophecies of default and moral hazard from people who pay their debts being forced to subsidize those who don’t. But it at least makes some sense.

But if a country is a “high risk” in the sense of macroeconomic instability undermining the real value of their debt, we want to ensure that they can restore macroeconomic stability. But we know that when there is a surge in interest rates on government bonds, instability gets worse, not better. Fiscal policy is suddenly shifted away from real production into higher debt payments, and this creates unemployment and makes the economic crisis worse. As Paul Krugman writes about frequently, these policies of “austerity” cause enormous damage to national economies and ultimately benefit no one because they destroy the source of wealth that would have been used to repay the debt.

By letting credit rating agencies decide the rates at which governments must borrow, we are effectively treating national governments as a special case of corporations. But corporations, by design, act for profit and can go bankrupt. National governments are supposed to act for the public good and persist indefinitely. We can’t simply let Greece fail as we might let a bank fail (and of course we’ve seen that there are serious downsides even to that). We have to restructure the sovereign debt system so that it benefits the development of nations rather than detracting from it. The first step is removing the power of private for-profit corporations in the US to decide the “creditworthiness” of entire countries. If we need to assess such risks at all, they should be done by international institutions like the UN or the World Bank.

But right now people are so stuck in the idea that national debt is basically the same as personal or corporate debt that they can’t even understand the problem. For after all, one must repay one’s debts.

We all know lobbying is corrupt. What can we do about it?

JDN 2457439

It’s so well-known as to almost seem cliche: Our political lobbying system is clearly corrupt.

Juan Cole, a historian and public intellectual from the University of Michigan, even went so far as to say that the United States is the most corrupt country in the world. He clearly went too far, or else left out a word; the US may well be the most corrupt county in the First World, though most rankings say Italy. In any case, the US is definitely not the most corrupt country in the whole world; no, that title goes to Somalia and/or North Korea.

Still, lobbying in the US is clearly a major source of corruption. Indeed, economists who study corruption often have trouble coming up with a sound definition of “corruption” that doesn’t end up including lobbying, despite the fact that lobbying is quite legal. Bribery means giving politicians money to get them to do things for you. Lobbying means giving politicians money and asking them to do things. In the letter of the law, that makes all the difference.

One thing that does make a difference is that lobbyists are required to register who they are and record their campaign contributions (unless of course they launder—I mean reallocate—them through a Super PAC of course). Many corporate lobbyists claim that it’s not that they go around trying to find politicians to influence, but rather politicians who call them up demanding money.

One of the biggest problems with lobbying is what’s called the revolving doorpoliticians are often re-hired as lobbyists, or lobbyists as politicians, based on the personal connections formed in the lobbying process—or possibly actual deals between lobbying companies over legislation, though if done explicitly that would be illegal. Almost 400 lobbyists working right now used to be legislators; almost 3,000 more worked as Congressional staff. Many lobbyists will do a tour as a Congressional staffer as a resume-builder, like an internship.

Studies have shown that lobbying does have an impact on policy—in terms of carving out tax loopholes it offers a huge return on investment.

Our current systems to disinventize the revolving door are not working. While there is reason to think that establishing a “cooling-off period” of a few years could make a difference, under current policy we already have some cooling-off periods and it’s clearly not enough.

So, now that we know the problem, let’s start talking about solutions.

Option 1: Ban campaign contributions

One possibility would be to eliminate campaign contributions entirely, which we could do by establishing a law that nobody can ever give money or in-kind favors to politicians ever under any circumstances. It would still be legal to meet with politicians and talk to them about issues, but if you take a Senator out for dinner we’d have to require that the Senator pay for their own food and transportation, lest wining-and-dining still be an effective means of manipulation. Then all elections would have to be completely publicly financed. This is a radical solution, but it would almost certainly work. MoveOn has a petition you can sign if you like this solution, and there’s a site called public-campaign-financing.org that will tell you how it could realistically be implemented (beware, their webmaster appears to be a time traveler from the 1990s who thinks that automatic music and tiled backgrounds constitute good web design).

There are a couple of problems with this solution, however:

First, it would be declared Unconstitutional by the Supreme Court. Under the (literally Orwellian) dicta that “corporations are people” and “money is speech” established in Citizens United vs. FEC, any restrictions on donating money to politicians constitute restrictions on free speech, and are therefore subject to strict scrutiny.

Second, there is actually a real restriction on freedom here, not because money is speech, but because money facilitates speech. Since eliminating all campaign donations would require total public financing of elections, we would need some way of deciding which candidates to finance publicly, because obviously you can’t give the same amount of money to everyone in the country or even everyone who decides to run. It simply doesn’t make sense to provide the same campaign financing for Hillary Clinton that you would for Vermin Supreme. But then, however this mechanism works, it could readily be manipulated to give even more advantages to the two major parties (not that they appear to need any more). If you’re fine with having exactly two parties to choose from, then providing funding for their, say, top 5 candidates in each primary, and then for their nominee in the general election, would work. But I for one would like to have more options than that, and that means devising some mechanism for funding third parties that have a realistic shot (like Ralph Nader or Ross Perot) but not those who don’t (like the aforementioned Vermin Supreme)—but at the same time we need to make sure that it’s not biased or self-fulfilling.

So let’s suppose we don’t eliminate campaign contributions completely. What else could we do that would curb corruption?

Option 2: Donation caps and “Democracy Credits”

I particularly like this proposal, self-titled the American Anti-Corruption Act (beware self-titled laws: USA PATRIOT ACT, anyone?), which would require full transparency—yes, even you, Super PACs—and place reasonable caps on donations so that large amounts of funds must be raised from large numbers of people rather than from a handful of people with a huge amount of money. It also includes an interesting proposal called “Democracy Credits” (again, the titles are a bit heavy-handed), which are basically an independent monetary system, used only to finance elections, and doled out exactly equally to all US citizens to spend on the candidates they like. The credits would then be exchangeable for real money, but only by the candidates themselves. This is a great idea, but sadly I doubt anyone in our political system is likely to go for it.

Actually, I would like to see these “Democracy Credits” used as votes—whoever gets the most credits wins the election, automatically. This is not quite as good as range voting, because it is not cloneproof or independent of irrelevant alternatives (briefly put, if you run two candidates that are exactly alike, their votes get split and they both lose, even if everyone likes them; and similarly, if you add a new candidate that doesn’t win you can still affect who does end up winning. Range voting is basically the only system that doesn’t have these problems, aside from a few really weird “voting” systems like “random ballot”). But still, it would be much better than our current plurality “first past the post” system, and would give third-party candidates a much fairer shot at winning elections. Indeed, it is very similar to CTT monetary voting, which is provably optimal in certain (idealized) circumstances. Of course, that’s even more of a pipe dream.

The donation caps are realistic, however; we used to have them, in fact, before Citizens United vs. FEC. Perhaps future Supreme Court decisions can overturn it and restore some semblance of balance in our campaign finance system.

Option 3: Treat campaign contributions as a conflict of interest

Jack Abramoff, a former lobbyist who was actually so corrupt he got convicted for it, has somewhat ironically made another proposal for how to reduce corrupting in the lobbying system. I suppose he would know, though I must wonder what incentives he has to actually do this properly (and corrupt people are precisely the sort of people with whom you should definitely be looking at the monetary incentives).

Abramoff would essentially use Option 1, but applied only to individuals and corporations with direct interests in the laws being made. As Gawker put it, “If you get money or perks from elected officials, […] you shouldn’t be permitted to give them so much as one dollar.” The way it avoids requiring total public financing is by saying that if you don’t get perks, you can still donate.

His plan would also extend the “cooling off” idea to its logical limit—once you work for Congress, you can never work for a lobbying organization for the rest of your life, and vice versa. That seems like a lot of commitment to ask of twentysomething Congressional interns (“If you take this job, unemployed graduate, you can never ever take that other job!”), but I suppose if it works it might be worth it.

He also wants to establish term limits for Congress, which seems pretty reasonable to me. If we’re going to have term limits for the Executive branch, why not the other branches as well? They could be longer, but if term limits are necessary at all we should use them consistently.

Abramoff also says we should repeal the 17th Amendment, because apparently making our Senators less representative of the population will somehow advance democracy. Best I can figure, he’s coming from an aristocratic attitude here, this notion that we should let “better people” make the important decisions if we want better decisions. And this sounds seductive, given how many really bad decisions people make in this world. But of course which people were the better people was precisely the question representative democracy was intended to answer. At least if Senators are chosen by state legislatures there’s a sort of meta-representation going on, which is obviously better than no representation at all; but still, adding layers of non-democracy by definition cannot make a system more democratic.

But Abramoff really goes off the rails when he proposes making it a conflict of interest to legislate about your own state.Pork-barrel spending”, as it is known, or earmarks as they are formally called, are actually a tiny portion of our budget (about 0.1% of our GDP) and really not worth worrying about. Sure, sometimes a Senator gets a bridge built that only three people will ever use, but it’s not that much money in the scheme of things, and there’s no harm in keeping our construction workers at work. The much bigger problem would be if legislators could no longer represent their own constituents in any way, thus defeating the basic purpose of having a representative legislature. (There is a thorny question about how much a Senator is responsible for their own state versus the country as a whole; but clearly their responsibility to their own state is not zero.)

Even aside from that ridiculous last part, there’s a serious problem with this idea of “no contributions from anyone who gets perks”: What constitutes a “perk”? Is a subsidy for solar power a perk for solar companies, or a smart environmental policy (can it be both?)? Does paying for road construction “affect” auto manufacturers in the relevant sense? What about policies that harm particular corporations? Since a carbon tax would hurt oil company profits, are oil companies allowed to lobby against it on the ground that it is the opposite of a “perk”?

Voting for representatives who will do things you want is kind of the point of representative democracy. (No, New York Post, it is not “pandering” to support women’s rights and interestswomen are the majority of our population. If there is one group of people that our government should represent, it is women.) Taken to its logical extreme, this policy would mean that once the government ever truly acts in the public interest, all campaign contributions are henceforth forever banned. I presume that’s not actually what Abramoff intends, but he offers no clear guidelines on how we would distinguish a special interest to be excluded from donations as opposed to a legitimate public interest that creates no such exclusion. Could we flesh this out in the actual legislative process? Is this something courts would decide?

In all, I think the best reform right now is to put the cap back on campaign contributions. It’s simple to do, and we had it before and it seemed to work (mostly). We could also combine that with longer cooling-off periods, perhaps three or five years instead of only one, and potentially even term limits for Congress. These reforms would certainly not eliminate corruption in the lobbying system, but they would most likely reduce it substantially, without stepping on fundamental freedoms.

Of course I’d really like to see those “Democracy Credits”; but that’s clearly not going to happen.

What does correlation have to do with causation?

JDN 2457345

I’ve been thinking of expanding the topics of this blog into some basic statistics and econometrics. It has been said that there are “Lies, damn lies, and statistics”; but in fact it’s almost the opposite—there are truths, whole truths, and statistics. Almost everything in the world that we know—not merely guess, or suppose, or intuit, or believe, but actually know, with a quantifiable level of certainty—is done by means of statistics. All sciences are based on them, from physics (when they say the Higgs discovery is a “5-sigma event”, that’s a statistic) to psychology, ecology to economics. Far from being something we cannot trust, they are in a sense the only thing we can trust.

The reason it sometimes feels like we cannot trust statistics is that most people do not understand statistics very well; this creates opportunities for both accidental confusion and willful distortion. My hope is therefore to provide you with some of the basic statistical knowledge you need to combat the worst distortions and correct the worst confusions.

I wasn’t quite sure where to start on this quest, but I suppose I have to start somewhere. I figured I may as well start with an adage about statistics that I hear commonly abused: “Correlation does not imply causation.”

Taken at its original meaning, this is definitely true. Unfortunately, it can be easily abused or misunderstood.

In its original meaning, the formal sense of the word “imply” meaning logical implication, to “imply” something is an extremely strong statement. It means that you logically entail that result, that if the antecedent is true, the consequent must be true, on pain of logical contradiction. Logical implication is for most practical purposes synonymous with mathematical proof. (Unfortunately, it’s not quite synonymous, because of things like Gödel’s incompleteness theorems and Löb’s theorem.)

And indeed, correlation does not logically entail causation; it’s quite possible to have correlations without any causal connection whatsoever, simply by chance. One of my former professors liked to brag that from 1990 to 2010 whether or not she ate breakfast had a statistically significant positive correlation with that day’s closing price for the Dow Jones Industrial Average.

How is this possible? Did my professor actually somehow influence the stock market by eating breakfast? Of course not; if she could do that, she’d be a billionaire by now. And obviously the Dow’s price at 17:00 couldn’t influence whether she ate breakfast at 09:00. Could there be some common cause driving both of them, like the weather? I guess it’s possible; maybe in good weather she gets up earlier and people are in better moods so they buy more stocks. But the most likely reason for this correlation is much simpler than that: She tried a whole bunch of different combinations until she found two things that correlated. At the usual significance level of 0.05, on average you need to try about 20 combinations of totally unrelated things before two of them will show up as correlated. (My guess is she used a number of different stock indexes and varied the starting and ending year. That’s a way to generate a surprisingly large number of degrees of freedom without it seeming like you’re doing anything particularly nefarious.)

But how do we know they aren’t actually causally related? Well, I suppose we don’t. Especially if the universe is ultimately deterministic and nonlocal (as I’ve become increasingly convinced by the results of recent quantum experiments), any two data sets could be causally related somehow. But the point is they don’t have to be; you can pick any randomly-generated datasets, pair them up in 20 different ways, and odds are, one of those ways will show a statistically significant correlation.

All of that is true, and important to understand. Finding a correlation between eating grapefruit and getting breast cancer, or between liking bitter foods and being a psychopath, does not necessarily mean that there is any real causal link between the two. If we can replicate these results in a bunch of other studies, that would suggest that the link is real; but typically, such findings cannot be replicated. There is something deeply wrong with the way science journalists operate; they like to publish the new and exciting findings, which 9 times out of 10 turn out to be completely wrong. They never want to talk about the really important and fascinating things that we know are true because we’ve been confirming them over hundreds of different experiments, because that’s “old news”. The journalistic desire to be new and first fundamentally contradicts the scientific requirement of being replicated and confirmed.

So, yes, it’s quite possible to have a correlation that tells you absolutely nothing about causation.

But this is exceptional. In most cases, correlation actually tells you quite a bit about causation.

And this is why I don’t like the adage; “imply” has a very different meaning in common speech, meaning merely to suggest or evoke. Almost everything you say implies all sorts of things in this broader sense, some more strongly than others, even though it may logically entail none of them.

Correlation does in fact suggest causation. Like any suggestion, it can be overridden. If we know that 20 different combinations were tried until one finally yielded a correlation, we have reason to distrust that correlation. If we find a correlation between A and B but there is no logical way they can be connected, we infer that it is simply an odd coincidence.

But when we encounter any given correlation, there are three other scenarios which are far more likely than mere coincidence: A causes B, B causes A, or some other factor C causes A and B. These are also not mutually exclusive; they can all be true to some extent, and in many cases are.

A great deal of work in science, and particularly in economics, is based upon using correlation to infer causation, and has to be—because there is simply no alternative means of approaching the problem.

Yes, sometimes you can do randomized controlled experiments, and some really important new findings in behavioral economics and development economics have been made this way. Indeed, much of the work that I hope to do over the course of my career is based on randomized controlled experiments, because they truly are the foundation of scientific knowledge. But sometimes, that’s just not an option.

Let’s consider an example: In my master’s thesis I found a strong correlation between the level of corruption in a country (as estimated by the World Bank) and the proportion of that country’s income which goes to the top 0.01% of the population. Countries that have higher levels of corruption also tend to have a larger proportion of income that accrues to the top 0.01%. That correlation is a fact; it’s there. There’s no denying it. But where does it come from? That’s the real question.

Could it be pure coincidence? Well, maybe; but when it keeps showing up in several different models with different variables included, that becomes unlikely. A single p < 0.05 will happen about 1 in 20 times by chance; but five in a row should happen less than 1 in 1 million times (assuming they’re independent, which, to be fair, they usually aren’t).

Could it be some artifact of the measurement methods? It’s possible. In particular, I was concerned about the possibility of Halo Effect, in which people tend to assume that something which is better (or worse) in one way is automatically better (or worse) in other ways as well. People might think of their country as more corrupt simply because it has higher inequality, even if there is no real connection. But it would have taken a very large halo bias to explain this effect.

So, does corruption cause income inequality? It’s not hard to see how that might happen: More corrupt individuals could bribe leaders or exploit loopholes to make themselves extremely rich, and thereby increase inequality.

Does inequality cause corruption? This also makes some sense, since it’s a lot easier to bribe leaders and manipulate regulations when you have a lot of money to work with in the first place.

Does something else cause both corruption and inequality? Also quite plausible. Maybe some general cultural factors are involved, or certain economic policies lead to both corruption and inequality. I did try to control for such things, but I obviously couldn’t include all possible variables.

So, which way does the causation run? Unfortunately, I don’t know. I tried some clever statistical techniques to try to figure this out; in particular, I looked at which tends to come first—the corruption or the inequality—and whether they could be used to predict each other, a method called Granger causality. Those results were inconclusive, however. I could neither verify nor exclude a causal connection in either direction. But is there a causal connection? I think so. It’s too robust to just be coincidence. I simply don’t know whether A causes B, B causes A, or C causes A and B.

Imagine trying to do this same study as a randomized controlled experiment. Are we supposed to create two societies and flip a coin to decide which one we make more corrupt? Or which one we give more income inequality? Perhaps you could do some sort of experiment with a proxy for corruption (cheating on a test or something like that), and then have unequal payoffs in the experiment—but that is very far removed from how corruption actually works in the real world, and worse, it’s prohibitively expensive to make really life-altering income inequality within an experimental context. Sure, we can give one participant $1 and the other $1,000; but we can’t give one participant $10,000 and the other $10 million, and it’s the latter that we’re really talking about when we deal with real-world income inequality. I’m not opposed to doing such an experiment, but it can only tell us so much. At some point you need to actually test the validity of your theory in the real world, and for that we need to use statistical correlations.

Or think about macroeconomics; how exactly are you supposed to test a theory of the business cycle experimentally? I guess theoretically you could subject an entire country to a new monetary policy selected at random, but the consequences of being put into the wrong experimental group would be disastrous. Moreover, nobody is going to accept a random monetary policy democratically, so you’d have to introduce it against the will of the population, by some sort of tyranny or at least technocracy. Even if this is theoretically possible, it’s mind-bogglingly unethical.

Now, you might be thinking: But we do change real-world policies, right? Couldn’t we use those changes as a sort of “experiment”? Yes, absolutely; that’s called a quasi-experiment or a natural experiment. They are tremendously useful. But since they are not truly randomized, they aren’t quite experiments. Ultimately, everything you get out of a quasi-experiment is based on statistical correlations.

Thus, abuse of the adage “Correlation does not imply causation” can lead to ignoring whole subfields of science, because there is no realistic way of running experiments in those subfields. Sometimes, statistics are all we have to work with.

This is why I like to say it a little differently:

Correlation does not prove causation. But correlation definitely can suggest causation.