Marriage and matching

Oct 10 JDN 2459498

When this post goes live, I will be married. We already had a long engagement, but it was made even longer by the pandemic: We originally planned to be married in October 2020, but then rescheduled for October 2021. Back then, we naively thought that the pandemic would be under control by now and we could have a wedding without COVID testing and masks. As it turns out, all we really accomplished was having a wedding where everyone is vaccinated—and the venue still required testing and masks. Still, it should at least be safer than it was last year, because everyone is vaccinated.

Since marriage is on my mind, I thought I would at least say a few things about the behavioral economics of marriage.

Now when I say the “economics of marriage” you likely have in mind things like tax laws that advantage (or disadvantage) marriage at different incomes, or the efficiency gains from living together that allow you to save money relative to each having your own place. That isn’t what I’m interested in.

What I want to talk about today is something a bit less economic, but more directly about marriage: the matching process by which one finds a spouse.

Economists would refer to marriage as a matching market. Unlike a conventional market where you can buy and sell arbitrary quantities, marriage is (usually; polygamy notwithstanding) a one-to-one arrangement. And unlike even the job market (which is also a one-to-one matching market), marriage usually doesn’t involve direct monetary payments (though in cultures with dowries it arguably does).

The usual model of a matching market has two separate pools: Employers and employees, for example. Typical heteronormative analyses of marriage have done likewise, separating men and women into different pools. But it turns out that sometimes men marry men and women marry women.

So what happens to our matching theory if we allow the pools to overlap?

I think the most sensible way to do it, actually, is to have only one pool: people who want to get married. Then, the way we capture the fact that most—but not all—men only want to marry women, and most—but not all—women only want to marry men is through the utililty function: Heterosexuals are simply those for whom a same-sex match would have very low utility. This would actually mean modeling marriage as a form of the stable roommates problem. (Oh my god, they were roommates!)

The stable roommates problem actually turns out to be harder than the conventional (heteronormative) stable marriage problem; in fact, while the hetero marriage problem (as I’ll henceforth call it) guarantees at least one stable matching for any preference ordering, the queer marriage problem can fail to have any stable solutions. While the hetero marriage problem ensures that everyone will eventually be matched to someone (if the number of men is equal to the number of women), sadly, the queer marriage problem can result in some people being forever rejected and forever alone. (There. Now you can blame the gays for ruining something: We ruined marriage matching.)

The queer marriage problem is actually more general than the hetero marriage problem: The hetero marriage problem is just the queer marriage problem with a particular utility function that assigns everyone strictly gendered preferences.

The best known algorithm for the queer marriage problem is an extension of the standard Gale-Shapley algorithm for the hetero marriage problem, with the same O(n^2) complexity in theory but a considerably more complicated implementation in practice. Honestly, while I can clearly grok the standard algorithm well enough to explain it to someone, I’m not sure I completely follow this one.

Then again, maybe preference orderings aren’t such a great approach after all. There has been a movement in economics toward what is called ordinal utility, where we speak only of preference orderings: You can like A more than B, but there’s no way to say how much more. But I for one am much more inclined toward cardinal utility, where differences have magnitudes: I like Coke more than Pepsi, and I like getting massaged more than being stabbed—and the difference between Coke and Pepsi is a lot smaller than the difference between getting massaged and being stabbed. (Many economists make much of the notion that even cardinal utility is “equivalent up to an affine transformation”, but I’ve got some news for you: So are temperature and time. All you are really doing by making an “affine transformation” is assigning a starting point and a unit of measurement. Temperature has a sensible absolute zero to use as a starting point, you say? Well, so does utility—not existing. )

With cardinal utility, I can offer you a very simple naive algorithm for finding an optimal match: Just try out every possible set of matchings and pick the one that has the highest total utility.

There are up to n!/((n/2)! 2^n) possible matchings to check, so this could take a long time—but it should work. I’m sure there’s a more efficient algorithm out there, but I don’t have the mental energy to figure it out at the moment. It might still be NP-hard, but I doubt it’s that hard.

Moreover, even once we find a utility-maximizing matching, that doesn’t guarantee a stable matching: Some people might still prefer to change even if it would end up reducing total utility.

Here’s a simple set of preferences for which that becomes an issue. In this table, the row is the person making the evaluation, and the columns are how much utility they assign to a match with each person. The total utility of a match is just the sum of utility from the two partners. The utility of “matching with yourself” is the utility of not being matched at all.


ABCD
A0321
B2031
C3201
D3210

Since everyone prefers every other person to not being matched at all (likely not true in real life!), the optimal matchings will always match everyone with someone. Thus, there are actually only 3 matchings to compare:

AB, CD: (3+2)+(1+1) = 7

AC, BD: (2+3)+(1+2) = 8

AD, BC: (1+3)+(3+2) = 9

The optimal matching, in utilitarian terms, is to match A with D and B with C. This yields total utility of 9.

But that’s not stable, because A prefers C over D, and C prefers A over B. So A and C would choose to pair up instead.

In fact, this set of preferences yields no stable matching at all. For anyone who is partnered with D, another member will rate them highest, and D’s partner will prefer that person over D (because D is everyone’s last choice).

There is always a nonempty set of utility-maximizing matchings. (There must be at least one, and could in principle have as many as there are possible matchings.) This actually just follows from the well-ordering property of the real numbers: Any finite set of reals has a maximum.

As this counterexample shows, there isn’t always a stable matching.

So here are a couple of interesting theoretical questions that this gives rise to:
1. If there is a stable matching, must it be in the set of utility-maximizing matchings?

2. If there is a stable matching, must all utility-maximizing matchings be stable?

Question 1 asks whether being stable implies being utility-maximizing.
Question 2 asks whether being utility-maximizing implies being stable—conditional on there being at least one stable possibility.

So, what is the answer to these questions? I don’t know! I’m actually not sure anyone does! We may have stumbled onto cutting-edge research!

I found a paper showing that these properties do not hold when you are doing the hetero marriage problem and you use multiplicative utility for matchings, but this is the queer marriage problem, and moreover I think multiplicative utility is the wrong approach. It doesn’t make sense to me to say that a marriage where one person is extremely happy and the other is indifferent to leaving is equivalent to a marriage where both partners are indifferent to leaving, but that’s what you’d get if you multiply 1*0 = 0. And if you allow negative utility from matchings (i.e. some people would prefer to remain single than to be in a particular match—which seems sensible enough, right?), since -1*-1 = 1, multiplicative utility yields the incredibly perverse result that two people who despise each other constitute a great match. Additive utility solves both problems: 1+0 = 1 and -1+-1 = -2, so, as we would hope, like + indifferent = like, and hate + hate = even more hate.

There is something to be said for the idea that two people who kind of like each other is better than one person ecstatic and the other miserable, but (1) that’s actually debatable, isn’t it? And (2) I think that would be better captured by somehow penalizing inequality in matches, not by using multiplicative utility.

Of course, I haven’t done a really thorough literature search, so other papers may exist. Nor have I spent a lot of time just trying to puzzle through this problem myself. Perhaps I should; this is sort of my job, after all. But even if I had the spare energy to invest heavily in research at the moment (which I sadly do not), I’ve been warned many times that pure theory papers are hard to publish, and I have enough trouble getting published as it is… so perhaps not.

My intuition is telling me that 2 is probably true but 1 is probably false. That is, I would guess that the set of stable matchings, when it’s not empty, is actually larger than the set of utility-maximizing matchings.

I think where I’m getting that intuition is from the properties of Pareto-efficient allocations: Any utility-maximizing allocation is necessarily Pareto-efficient, but many Pareto-efficient allocations are not utility-maximizing. A stable matching is sort of a strengthening of the notion of a Pareto-efficient allocation (though the problem of finding a Pareto-efficient matching for the general queer marriage problem has been solved).

But it is interesting to note that while a Pareto-efficient allocation must exist (typically there are many, but there must be at least one, because it’s impossible to have a cycle of Pareto improvements as long as preferences are transitive), it’s entirely possible to have no stable matchings at all.

Against “doing your best”

Oct 3 JDN 2459491

It’s an appealing sentiment: Since we all have different skill levels, rather than be held to some constant standard which may be easy for some but hard for others, we should each do our best. This will ensure that we achieve the best possible outcome.

Yet it turns out that this advice is not so easy to follow: What is “your best”?

Is your best the theoretical ideal of what your performance could be if all obstacles were removed and you worked at your greatest possible potential? Then no one in history has ever done their best, and when people get close, they usually end up winning Nobel Prizes.

Is your best the performance you could attain if you pushed yourself to your limit, ignored all pain and fatigue, and forced yourself to work at maximum effort until you literally can’t anymore? Then doing your best doesn’t sound like such a great thing anymore—and you’re certainly not going to be able to do it all the time.

Is your best the performance you would attain by continuing to work at your usual level of effort? Then how is that “your best”? Is it the best you could attain if you work at a level of effort that is considered standard or normative? Is it the best you could do under some constraint limiting the amount of pain or fatigue you are willing to bear? If so, what constraint?

How does “your best” change under different circumstances? Does it become less demanding when you are sick, or when you have a migraine? What if you’re depressed? What if you’re simply not feeling motivated? What if you can’t tell whether this demotivation is a special circumstance, a depression system, a random fluctuation, or a failure to motivate yourself?

There’s another problem: Sometimes you really aren’t good at something.

A certain fraction of performance in most tasks is attributable to something we might call “innate talent”; be it truly genetic or fixed by your early environment, it nevertheless is something that as an adult you are basically powerless to change. Yes, you could always train and practice more, and your performance would thereby improve. But it can only improve so much; you are constrained by your innate talent or lack thereof. No amount of training effort will ever allow me to reach the basketball performance of Michael Jordan, the painting skill of Leonardo Da Vinci, or the mathematical insight of Leonhard Euler. (Of the three, only the third is even visible from my current horizon. As someone with considerable talent and training in mathematics, I can at least imagine what it would be like to be as good as Euler—though I surely never will be. I can do most of the mathematical methods that Euler was famous for; but could I have invented them?)

In fact it’s worse than this; there are levels of performance that would be theoretically possible for someone of your level of talent, yet would be so costly to obtain as to be clearly not worth it. Maybe, after all, there is some way I could become as good a mathematician as Euler—but if it would require me to work 16-hour days doing nothing but studying mathematics for the rest of my life, I am quite unwilling to do so.

With this in mind, what would it mean for me to “do my best” in mathematics? To commit those 16-hour days for the next 30 years and win my Fields Medal—if it doesn’t kill me first? If that’s not what we mean by “my best”, then what do we mean, after all?

Perhaps we should simply abandon the concept, and ask instead what successful people actually do.

This will of course depend on what they were successful at; the behavior of basketball superstars is considerably different from the behavior of Nobel Laureate physicists, which is in turn considerably different from the behavior of billionaire CEOs. But in theory we could each decide for ourselves which kind of success we actually would desire to emulate.

Another pitfall to avoid is looking only at superstars and not comparing them with a suitable control group. Every Nobel Laureate physicist eats food and breathes oxygen, but eating food and breathing oxygen will not automatically give you good odds of winning a Nobel (though I guess your odds are in fact a lot better relative to not doing them!). It is likely that many of the things we observe successful people doing—even less trivial things, like working hard and taking big risks—are in fact the sort of thing that a great many people do with far less success.

Upon making such a comparison, one of the first things that we would notice is that the vast majority of highly-successful people were born with a great deal of privilege. Most of them were born rich or at least upper-middle-class; nearly all of them were born healthy without major disabilities. Yes, there are exceptions to any particular form of privilege, and even particularly exceptional individuals who attained superstar status with more headwinds than tailwinds; but the overwhelming pattern is that people who get home runs in life tend to be people who started the game on third base.

But setting that aside, or recalibrating one’s expectations to try to attain a level of success often achieved by people with roughly the same level of privilege as oneself, we must ask: How often? Should you aspire to the median? The top 20%? The top 10%? The top 1%? And what is your proper comparison group? Should I be comparing against Americans, White male Americans, economists, queer economists, people with depression and chronic migraines, or White/Native American male queer economists with depression and chronic migraines who are American expatriates in Scotland? Make the criteria too narrow, and there won’t be many left in your sample. Make them instead too broad, and you’ll include people with very different circumstances who may not be a fair comparison. Perhaps some sort of weighted average of different groups could work—but with what weighting?

Or maybe it’s right to compare against a very broad group, since this is what ultimately decides our life prospects. What it would take to write the best novel you (or someone “like you” in whatever sense that means) can write may not be the relevant question: What you really needed to know was how likely it is that you could make a living as a novelist.


The depressing truth in such a broad comparison is that you may in fact find yourself faced with so many obstacles that there is no realistic path toward the level of success you were hoping for. If you are reading this, I doubt matters are so dire for you that you’re at serious risk of being homeless and starving—but there definitely are people in this world, millions of people, for whom that is not simply a risk but very likely the best they can hope for.

The question I think we are really trying to ask is this: What is the right standard to hold ourselves against?

Unfortunately, I don’t have a clear answer to this question. I have always been an extremely ambitious individual, and I have inclined toward comparisons with the whole world, or with the superstars of my own fields. It is perhaps not surprising, then, that I have consistently failed to live up to my own expectations for my own achievement—even as I surpass what many others expected for me, and have long-since left behind what most people expect for themselves and each other.

I would thus not exactly recommend my own standards. Yet I also can’t quite bear to abandon them, out of a deep-seated fear that it is only by holding myself to the patently unreasonable standard of trying to be the next Einstein or Schrodinger or Keynes or Nash that I have even managed what meager achievements I have made thus far.

Of course this could be entirely wrong: Perhaps I’d have achieved just as much if I held myself to a lower standard—or I could even have achieved more, by avoiding the pain and stress of continually failing to achieve such unattainable heights. But I also can’t rule out the possibility that it is true. I have no control group.

In general, what I think I want to say is this: Don’t try to do your best. You have no idea what your best is. Instead, try to find the highest standard you can consistently meet.

Realistic open borders

Sep 5 JDN 2459463

In an earlier post I lamented the tight restrictions on border crossings that prevail even between allied First World countries. (On a personal note, you’ll be happy to know that our visas have cleared and we are now moved into Edinburgh, cat and all, though we are still in temporary housing and our official biometric residence permits haven’t yet arrived.)

In this post I’d like to speculate on how we might get from our current regime to something more like open borders.

Obviously we can’t simply remove all border restrictions immediately. That would be a political non-starter, and even ethically or economically it wouldn’t make very much sense. There are sensible reasons behind some of our border regulations—just not most of them.

Instead we would want to remove a few restrictions at a time, starting with the most onerous or ridiculous ones.

High on my list in the UK in particular would be the requirement that pets must fly as cargo. I literally can’t think of a good reason for this; it seems practically designed to cost travelers more money and traumatize as many pets as possible. If it’s intended to support airlines somehow, please simply subsidize airlines. (But really, why are you doing that? You should be taxing airlines because of their high carbon emissions. Subsidize boats and trains.) If it’s intended to somehow prevent the spread of rabies, it’s obviously unnecessary, since every pet moved to the UK already has to document a recent rabies vaccine. But this particular rule seems to be a quirk of the UK in particular, hence not very generalizable.

But here’s one that actually seems quite common: Financial requirements for visas. Even tourist visas in most countries cost money, in amounts that seem to vary according to some sort of occult ritual. I can see no sensible economic reason why a visa would be $130 in Vietnam but only $20 in neighboring Cambodia, or why Kazakhstan can be visited for $25 but Azerbaijan costs $100, or why Myanmar costs only $30 but Bhutan will run you over $200.

Work visas are considerably more demanding still.

Financial requirements in the UK are especially onerous; you have to make above a certain salary and have a certain amount of savings in the bank, based on your family size. This was no problem for me personally, but it damn well shouldn’t be; I have a PhD in economics. My salary is now twice what it was as a grad student, and honestly that’s a good deal less than I was hoping for (and would have gotten on the tenure track at an R1 university).

All the countries in the Schengen Area have their own requirements for “financial subsistence” for visa applications, ranging from a trivial €3 in Hungary (not per day, just total; why do they even bother?) or manageable €14 per day in Latvia, through the more demanding amounts of €45 per day in Germany and Italy, to €92 per day in Switzerland and Liechtenstein, all the way up to the utterly unreasonable €120 per day in France. That would be €43,800 per year, or $51,700. Apparently you must be at least middle class to enter France.

Canada has a similar requirement known as “proof of funds”, but it’s considerably more reasonable, since you can substitute proof of employment and there are no wage minimums for such employment. Even if you don’t already have a job you can still apply and the minimum requirement is actually lower than the poverty line in Canada.

The United States doesn’t require financial requirements for most visas, but it does have a $160 visa fee. And the H1-B visa in particular (the nearest equivalent to the Skilled Worker visa I’ve got in the UK) requires that your wage or salary be at least the “prevailing wage” in your industry—meaning it is nearly impossible for a company to save money by hiring people on H1-B visas and hence they have very little incentive to hire H1-B workers. If you are of above-average talent and being paid only average wages, I guess they can save some money that way. But this is not how trade is supposed to work—nobody requires that you pay US prices for goods shipped from China, and if they did, nobody would ever buy anything from China. This is blatant, naked protectionism—but we’re apparently okay with it as long as it’s trade in labor instead of goods.

I wasn’t able to quickly find whether there are similar financial requirements in other countries. Perhaps there aren’t; these are the countries most people actually want to move to anyway. Permanent migration is overwhelminginly toward OECD (read: First World) countries, and is actually helping us sustain our populations in the face of low birth rates.

I must admit, I can see some fiscal benefits for a country not allowing poor people in, but this practice raises some very deep ethical problems: What right do we have to do this?

If someone is born poor in Laredo, Texas, we take responsibility for them as a US citizen. Maybe we don’t treat them particularly well (that is Texas, after all), but we do give them access to certain basic services, such as emergency services, Medicaid, TANF and SNAP. They are allowed to vote, own property, and even hold office in the United States. But if that same person were born in Nuevo Laredo, Tamaulipas—literally less than a mile away, right across the river—they would receive none of these benefits. They would not even be allowed to cross the river without a passport and a visa.

In some ways the contrast is even more dire if we consider a more liberal US state. A poor person born in Chula Vista, California has access to the full array of California services; Medi-Cal is honestly something close to a single-payer healthcare system, though the full morass of privatized US healthcare is layered on top of us. Then there is CalWORKS, CalFresh, and so on. But the same person born in Tijuana, Baja California would get none of these benefits.

They could be the same person. They could look the same and have essentially the same culture—even the same language, given how many Californians speak Spanish and how many Mexicans speak English. But if they were born on the other side of a river (in Texas) or even an arbitrary line (in California), we treat them completely differently. And then to add insult to injury, we won’t even let them across, not in spite, but because of how poor and desperate they are. If they were rich and educated, we’d let them come across—but then why would they need to?

“Give me your tired, your poor, your huddled masses yearning to breathe free”?

Some restrictions may apply.

Economists talk often of “trade barriers”, but in real terms we have basically removed all trade barriers in goods. Yes, there are still some small tariffs, and the occasional quota here and there—and these should go away too, especially the quotas, because they don’t even raise revenue—but in general we have an extremely globalized economy in terms of goods. The same complex product, like a car or a smartphone, is often made of parts from a dozen countries.

But when it comes to labor, we are still living in a protectionist world. Crossing borders to work is difficult, time-consuming, and above all, expensive. This dramatically reduces opportunities for workers to move where their labor is most valued—which hurts not only them, but also anyone who would employ them or buy products made by them. The poorest people are those who stand to gain the most from crossing borders, and they are precisely the ones that we work hardest to forbid.

So let’s start with that, shall we? We can keep all this nonsense about passports, visas, background checks, and customs inspections. It’s probably all unnecessary and wasteful and unfair, but politically it’s clearly too popular to remove. Let’s just remove this: No more financial requirements or fees for work visas. If you want to come to another country to work, you have to go through an application and all that; fine. But you shouldn’t have to prove you aren’t poor. Poor people have just as much right to live here as anybody else—and if we let them do so, they’d be a lot less poor.

Why are borders so strict?

Aug 15 JDN 2459442

Most of us don’t cross borders all that often, and when we do it’s generally only for brief visits; so we don’t often experience just how absurdly difficult it is to move to another country. I have received a crash course in the subject for the past couple of months, in trying to arrange my move to Edinburgh.

Certain portions of the move would be inherently difficult: Moving a literal ton of stuff across an entire ocean is no mean feat, and really the impressive thing is that our civilization has reached the point where we can do it so quickly and reliably. (I do mean a literal ton: We estimated we have about 350 cubic feet and 2300 pounds of items, or 10 cubic meters and 1040 kilograms.)

But most of the real headaches have been the results of institutional policies.

First of all, there’s the fact that the university gave me so little notice. This is not entirely their fault; my understanding is that the position opened up during the spring, and they scrambled to fill it as fast as they could for the fall. Still, this has made everything that much more difficult.

More importantly, there is the matter of moving across borders.

In order to get visas to live in the UK, my fiance and I had to complete an application documenting basically our whole lives (I had to track down three parking tickets and a speeding ticket from as far back as 2011), maintain bank balances of a sufficient amount for at least 30 days (evidently poor people need not apply), and pay exorbitant fees (over $5000 in all for the two of us, which, gratefully, the university is supposed to reimburse me for). We had to upload not only our passports, but also financial documents as well as housing records to prove our relationship (in lieu of a marriage license, since we had to delay the wedding to this year due to the pandemic). But this was not enough; we had to pay even more fees to get expedited processing, and then travel to a US government office in the LA area to get our fingerprints done, and then mail our passports to another office in New York for further processing. We started this process the first week of August; we still haven’t heard back on our final approval.

Then there is the matter of moving our cat, Tootsie. UK regulations for importing a cat require an ISO-compliant microchip and certain vaccinations; this is perfectly reasonable. But they also require that you bring the cat with you when you move (within at most 5 days of your arrival), or else the cat will be legally considered livestock and subject to a tariff of over $1000.

This would be inconvenient enough, but then there is the fact that current regulations do not allow cats to be transported into the UK in the cabin of an aircraft. If they are to be flown in, they must be brought in the cargo hold. Since we did not want to subject our cat to several hours alone in a cargo hold on a transatlantic flight, we will instead be flying to Amsterdam, because the Netherlands has more lenient regulations. But then of course we still need to get her to Edinburgh; our current plan involves taking a ferry from Amsterdam to Newcastle and then a train from there to Edinburgh. In all the whole process will take at least a day longer (and cost a few hundred dollars more) than it would have without the utterly pointless rule forbidding cats from flying into the UK in the cabin.

All of this for, and I really cannot emphasize this enough, a routine move between two NATO allied First World countries.

The alliance between the US and the UK is one of the most tightly-knit in the world, and dates back generations. Our trade networks are thoroughly interconnected, and we even share most of our media and culture back and forth. There’s honestly no particular reason we couldn’t simply be the same country. (Indeed the one thing we did fight with them about in the last 250 years was over precisely that.)

There is probably less difference culturally and economically between New York and London than there is between New York and rural Texas or between London and rural Scotland. Yet a move within each country requires basically none of this extra hassle and paperwork—you basically just physically move yourself, register your car, maybe a few other minor things. You certainly don’t need to get a passport, apply for a visa, or pay exorbitant fees.

What purpose does all of this extra regulation serve? Are we safer, or richer, or healthier, because we make it so difficult to move across borders?

I can understand the need to hve some sort of security at border crossings: We want to make sure people aren’t smuggling contraband or planning acts of terrorism. (There is, by the way, a series of questions on the UK visa application asking things like this:”Have you ever committed terrorism?” “Have you ever been implicated in genocide?” One wonders if anyone has ever answered “yes”.) It even makes sense to have some kind of registration process and background check for people who plan to move permanently. But what we actually do goes far, far beyond these sensible requirements; the goal seems to be to ensure that only the finest upstanding citizens may be allowed to move to a country, while anyone who is born on the opposite side of that line need not meet any standard whatsoever in order to remain.

In my view, the most sensible standard would be this: You should only exclude someone from entering your country for actions that you’d be willing to imprison them for if they were already there. Clearly, smuggling and terrorism qualify. Indeed, any felony would do. But would you lock someone in prison for not having enough money in their bank account? Or for failing to disclose a parking ticket from ten years ago? Or for filling out paperwork incorrectly? Yet visas are denied for this sort of reason all the time.

I think most economists would agree with me: The free movement of people across borders is one of the most vital principles of free trade—and the one that the world has least lived up to so far.

Yet it seems we are in the minority. Most people seem to think it’s perfectly sensible to have completely different rules for moving from Detroit to Toledo than from Detroit to Windsor.

The reason for this is apparent enough: Once again, the tribal paradigm looms large. Human beings divide themselves into groups, and form their identities around those groups. Those inside the group are good, while those outside are bad. Actions which benefit our own group are right, while actions which benefit other groups are wrong. The group you belong to is an inherent part of who you are, and can never be changed.

We have defined these groups in many different ways throughout human history, and our scale of group identification has gradually expanded over time. First, it was families and tribes. For centuries, it was feudal kingdoms. Now, it is nation-states. Perhaps, someday, it will enlarge to encompass all of humanity.

But until that day comes, people are going to make it as hard as possible to cross from one group to another.

Love the disabled, hate the disability

Aug 1 JDN 2459428

There is a common phrase Christians like to say: “Love the sinner, hate the sin.” This seems to be honored more in the breach than the observance, and many of the things that most Christians consider “sins” are utterly harmless or even good; but the principle is actually quite sound. You can disagree with someone or even believe that what they are doing is wrong while still respecting them as a human being. Indeed, my attitude toward religion is very much “Love the believer, hate the belief.” (Though somehow they don’t seem to like that one so much….)

Yet while ethically this is often the correct attitude, psychologically it can be very difficult for people to maintain. The Halo Effect is a powerful bias, and most people recoil instinctively from saying anything good about someone bad or anything bad about someone good. This can make it uncomfortable to simply state objective facts like “Hitler was a charismatic leader” or “Stalin was a competent administrator”—how dare you say something good about someone so evil? Yet in fact Hitler and Stalin could never have accomplished so much evil if they didn’t have these positive attributes—if we want to understand how such atrocities can occur and prevent them in the future, we need to recognize that evil people can also be charismatic and competent.

Halo Effect also makes it difficult for people to understand the complexities of historical figures who have facets of both great good and great evil: Thomas Jefferson led the charge on inventing modern democracy—but he also owned and raped slaves. Lately it seems like the left wants to deny the former and the right wants to deny the latter; but both are historical truths that important to know.

Halo Effect is the best explanation I have for why so many disability activists want to deny that disabilities are inherently bad. They can’t keep in their head the basic principle of “Love the disabled, hate the disability.”

There is a large community of deaf people who say that being deaf isn’t bad. There are even some blind people who say that being blind isn’t bad—though they’re considerably rarer.

Is music valuable? Is art valuable? Is the world better off because Mozart’s symphonies and the Mona Lisa exist? Yes. It follows that being unable to experience these things is bad. Therefore blindness and deafness are bad. QED.


No human being is made better of by not being able to do something. More capability is better than less capability. More freedom is better than less freedom. Less pain is better than more pain.

(Actually there are a few exceptions to “less pain is better than more pain”: People with CIPA are incapable of feeling pain even when injured, which is very dangerous.)

From this, it follows immediately that disabilities are bad and we should be trying to fix them.

And frankly this seems so utterly obvious to me that it’s hard for me to understand why anyone could possibly disagree. Maybe people who are blind or deaf simply don’t know what they’re missing? Even that isn’t a complete explanation, because I don’t know what it would be like to experience four dimensions or see ultraviolet—yet I still think that I’d be better off if I could. If there were people who had these experiences telling me how great they are, I’d be certain of it.

Don’t get me wrong: A lot of ableist discrimination does exist, and much of it seems to come from the same psychological attitude: Since being disabled is bad, they think that disabled people must be bad and we shouldn’t do anything to make them better off because they are bad. Stated outright this sounds ludicrous; but most people who think this way don’t consciously reflect on it. They just have a general sense of badness related to disability which then rubs off on their attitudes toward disabled people as well.

Yet it makes hardly any more sense to go the other way: Disabled people are human beings of value, they are good; therefore their disabilities are good? Therefore this thing that harms and limits them is good?

It’s certainly true that most disabilities would be more manageable with better accommodations, and many of those accommodations would be astonishingly easy and cheap to implement. It’s terrible that we often fail to do this. Yet the fact remains: The best-case scenario would be not needing accommodations because we can simply cure the disability.

It never ceases to baffle me that disability activists will say things like this:

“A wheelchair user isn’t disabled because of the impairment that interferes with her ability to walk, but because society refuses to make spaces wheelchair-accessible.”

No, the problem is pretty clearly the fact that she can’t walk. There are various ways that we could make society more accessible to people in wheelchairs—and we should do those things—but there are inherently certain things you simply cannot do if you can’t walk, and that has nothing to do with anything society does. You would be better off if society were more accommodating, but you’d be better off still if you could simply walk again.

Perhaps my perspective on this is skewed, because my major disability—chronic migraine—involves agonizing, debilitating chronic pain. Perhaps people whose disabilities don’t cause them continual agony can convince themselves that there’s nothing wrong with them. But it seems pretty obvious to me that I would be better off without migraines.

Indeed, it’s utterly alien to my experience to hear people say things like this: “We’re not suffering. We’re just living our lives in a different way.” I’m definitely suffering, thank you very much. Maybe not everyone with disabilities is suffering—but a lot of us definitely are. Every single day I have to maintain specific habits and avoid triggers, and I still get severe headaches twice a week. I had a particularly nasty one just this morning.

There are some more ambiguous cases, to be sure: Neurodivergences like autism and ADHD that exist on a spectrum, where the most extreme forms are utterly debilitating but the mildest forms are simply ordinary variation. It can be difficult to draw the line at when we should be willing to treat and when we shouldn’t; but this isn’t fundamentally different from the sort of question psychiatrists deal with all the time, regarding the difference between normal sadness and nervousness versus pathological depression and anxiety disorders.

Of course there is natural variation in almost all human traits, and one can have less of something good without it being pathological. Some things we call disabilities could just be considered below-average capabilities within ordinary variation. Yet even then, if we could make everyone healthier, stronger, faster, tougher, and smarter than they currently are, I have trouble seeing why we wouldn’t want to do that. I don’t even see any particular reason to think that the current human average—or even the current human maximum—is in any way optimal. Better is better. If we have the option to become transhuman gods, why wouldn’t we?

Another way to see this is to think about how utterly insane it would be to actively try to create disabilities. If there’s nothing wrong with being deaf, why not intentionally deafen yourself? If being bound to a wheelchair is not a bad thing, why not go get your legs paralyzed? If being blind isn’t so bad, why not stare into a welding torth? In these cases you’d even have consented—which is absolutely not the case for an innate disability. I never consented to these migraines and never would have.

I respect individual autonomy, so I would never force someone to get treatment for their disability. I even recognize that society can pressure people to do things they wouldn’t want to, and so maybe occasionally people really are better off being unable to do something so that nobody can pressure them into it. But it still seems utterly baffling to me that there are people who argue that we’d be better off not even having the option to make our bodies work better.

I think this is actually a major reason why disability activism hasn’t been more effective; the most vocal activists are the ones saying ridiculous things like “the problem isn’t my disability, it’s your lack of accommodations” or “there’s nothing wrong with being unable to hear”. If there is anything you’d be able to do if your disability didn’t exist that you can’t do even with accommodations, that isn’t true—and there basically always is.

Escaping the wrong side of the Yerkes-Dodson curve

Jul 25 JDN 2459421

I’ve been under a great deal of stress lately. Somehow I ended up needing to finish my dissertation, get married, and move overseas to start a new job all during the same few months—during a global pandemic.

A little bit of stress is useful, but too much can be very harmful. On complicated tasks (basically anything that involves planning or careful thought), increased stress will increase performance up to a point, and then decrease it after that point. This phenomenon is known as the Yerkes-Dodson law.

The Yerkes-Dodson curve very closely resembles the Laffer curve, which shows that since extremely low tax rates raise little revenue (obviously), and extremely high tax rates also raise very little revenue (because they cause so much damage to the economy), the tax rate that maximizes government revenue is actually somewhere in the middle. There is a revenue-maximizing tax rate (usually estimated to be about 70%).

Instead of a revenue-maximizing tax rate, the Yerkes-Dodson law says that there is a performance-maximizing stress level. You don’t want to have zero stress, because that means you don’t care and won’t put in any effort. But if your stress level gets too high, you lose your ability to focus and your performance suffers.

Since stress (like taxes) comes with a cost, you may not even want to be at the maximum point. Performance isn’t everything; you might be happier choosing a lower level of performance in order to reduce your own stress.

But once thing is certain: You do not want to be to the right of that maximum. Then you are paying the cost of not only increased stress, but also reduced performance.

And yet I think many of us spent a great deal of our time on the wrong side of the Yerkes-Dodson curve. I certainly feel like I’ve been there for quite awhile now—most of grad school, really, and definitely this past month when suddenly I found out I’d gotten an offer to work in Edinburgh.

My current circumstances are rather exceptional, but I think the general pattern of being on the wrong side of the Yerkes-Dodson curve is not.

Over 80% of Americans report work-related stress, and the US economy loses about half a trillion dollars a year in costs related to stress.

The World Health Organization lists “work-related stress” as one of its top concerns. Over 70% of people in a cross-section of countries report physical symptoms related to stress, a rate which has significantly increased since before the pandemic.

The pandemic is clearly a contributing factor here, but even without it, there seems to be an awful lot of stress in the world. Even back in 2018, over half of Americans were reporting high levels of stress. Why?

For once, I think it’s actually fair to blame capitalism.

One thing capitalism is exceptionally good at is providing strong incentives for work. This is often a good thing: It means we get a lot of work done, so employment is high, productivity is high, GDP is high. But it comes with some important downsides, and an excessive level of stress is one of them.

But this can’t be the whole story, because if markets were incentivizing us to produce as much as possible, that ought to put us near the maximum of the Yerkes-Dodson curve—but it shouldn’t put us beyond it. Maximizing productivity might not be what makes us happiest—but many of us are currently so stressed that we aren’t even maximizing productivity.

I think the problem is that competition itself is stressful. In a capitalist economy, we aren’t simply incentivized to do things well—we are incentivized to do them better than everyone else. Often quite small differences in performance can lead to large differences in outcome, much like how a few seconds can make the difference between an Olympic gold medal and an Olympic “also ran”.

An optimally productive economy would be one that incentivizes you to perform at whatever level maximizes your own long-term capability. It wouldn’t be based on competition, because competition depends too much on what other people are capable of. If you are not especially talented, competition will cause you great stress as you try to compete with people more talented than you. If you happen to be exceptionally talented, competition won’t provide enough incentive!

Here’s a very simple model for you. Your total performance p is a function of two components, your innate ability aand your effort e. In fact let’s just say it’s a sum of the two: p = a + e

People are randomly assigned their level of capability from some probability distribution, and then they choose their effort. For the very simplest case, let’s just say there are two people, and it turns out that person 1 has less innate ability than person 2, so a1 < a2.

There is also a certain amount of inherent luck in any competition. As it says in Ecclesiastes (by far the best book of the Old Testament), “The race is not to the swift or the battle to the strong, nor does food come to the wise or wealth to the brilliant or favor to the learned; but time and chance happen to them all.” So as usual I’ll model this as a contest function, where your probability of winning depends on your total performance, but it’s not a sure thing.

Let’s assume that the value of winning and cost of effort are the same across different people. (It would be simple to remove this assumption, but it wouldn’t change much in the results.) The value of winning I’ll call y, and I will normalize the cost of effort to 1.


Then this is each person’s expected payoff ui:

ui = (ai + ei)/(a1+e1+a2 + e2) V – ei

You choose effort, not ability, so maximize in terms of ei:

(a2 + e2) V = (a1 +e1+a2 + e2)2 = (a1 + e1) V

a1 + e1 = a2 + e2

p1 = p2

In equilibrium, both people will produce exactly the same level of performance—but one of them will be contributing more effort to compensate for their lesser innate ability.

I’ve definitely had this experience in both directions: Effortlessly acing math tests that I knew other people barely passed despite hours of studying, and running until I could barely breathe to keep up with other people who barely seemed winded. Clearly I had too little incentive in math class and too much in gym class—and competition was obviously the culprit.

If you vary the cost of effort between people, or make it not linear, you can make the two not exactly equal; but the overall pattern will remain that the person who has more ability will put in less effort because they can win anyway.

Yet presumably the amount of effort we want to incentivize isn’t less for those who are more talented. If anything, it may be more: Since an hour of work produces more when done by the more talented person, if the cost to them is the same, then the net benefit of that hour of work is higher than the same hour of work by someone less talented.

In a large population, there are almost certainly many people whose talents are similar to your own—but there are also almost certainly many below you and many above you as well. Unless you are properly matched with those of similar talent, competition will systematically lead to some people being pressured to work too hard and others not pressured enough.

But if we’re all stressed, where are the people not pressured enough? We see them on TV. They are celebrities and athletes and billionaires—people who got lucky enough, either genetically (actors who were born pretty, athletes who were born with more efficient muscles) or environmentally (inherited wealth and prestige), to not have to work as hard as the rest of us in order to succeed. Indeed, we are constantly bombarded with images of these fantastically lucky people, and by the availability heuristic our brains come to assume that they are far more plentiful than they actually are.

This dramatically exacerbates the harms of competition, because we come to feel that we are competing specifically with the people who were handed the world on a silver platter. Born without the innate advantages of beauty or endurance or inheritance, there’s basically no chance we could ever measure up; and thus we feel utterly inadequate unless we are constantly working as hard as we possibly can, trying to catch up in a race in which we always fall further and further behind.

How can we break out of this terrible cycle? Well, we could try to replace capitalism with something like the automated luxury communism of Star Trek; but this seems like a very difficult and long-term solution. Indeed it might well take us a few hundred years as Roddenberry predicted.

In the shorter term, we may not be able to fix the economic problem, but there is much we can do to fix the psychological problem.

By reflecting on the full breadth of human experience, not only here and now, but throughout history and around the world, you can come to realize that you—yes, you, if you’re reading this—are in fact among the relatively fortunate. If you have a roof over your head, food on your table, clean water from your tap, and ibuprofen in your medicine cabinet, you are far more fortunate than the average person in Senegal today; your television, car, computer, and smartphone are things that would be the envy even of kings just a few centuries ago. (Though ironically enough that person in Senegal likely has a smartphone, or at least a cell phone!)

Likewise, you can reflect upon the fact that while you are likely not among the world’s most very most talented individuals in any particular field, there is probably something you are much better at than most people. (A Fermi estimate suggests I’m probably in the top 250 behavioral economists in the world. That’s probably not enough for a Nobel, but it does seem to be enough to get a job at the University of Edinburgh.) There are certainly many people who are less good at many things than you are, and if you must think of yourself as competing, consider that you’re also competing with them.

Yet perhaps the best psychological solution is to learn not to think of yourself as competing at all. So much as you can afford to do so, try to live your life as if you were already living in a world that rewards you for making the best of your own capabilities. Try to live your life doing what you really think is the best use of your time—not your corporate overlords. Yes, of course, we must do what we need to in order to survive, and not just survive, but indeed remain physically and mentally healthy—but this is far less than most First World people realize. Though many may try to threaten you with homelessness or even starvation in order to exploit you and make you work harder, the truth is that very few people in First World countries actually end up that way (it couldbe brought to zero, if our public policy were better), and you’re not likely to be among them. “Starving artists” are typically a good deal happier than the general population—because they’re not actually starving, they’ve just removed themselves from the soul-crushing treadmill of trying to impress the neighbors with manicured lawns and fancy SUVs.

A prouder year for America, and for me

Jul 4 JDN 2459380

Living under Trump from 2017 to 2020, it was difficult to be patriotic. How can we be proud of a country that would put a man like that in charge? And then there was the COVID pandemic, which initially the US handled terribly—largely because of the aforementioned Trump.

But then Biden took office, and almost immediately things started to improve. This is a testament to how important policy can be—and how different the Democrats and Republicans have become.

The US now has one of the best rates of COVID vaccination in the world (though lately progress seems to be stalling and other countries are catching up). Daily cases in the US are now the lowest they have been since March 2020. Even real GDP is almost back up to its pre-pandemic level (even per-capita), and the surge of inflation we got as things began to re-open already seems to be subsiding.

I can actually celebrate the 4th of July with some enthusiasm this year, whereas the last four years involved continually reminding myself that I was celebrating the liberating values of America’s founding, not the current terrible state of its government. Of course our government policy still retains many significant flaws—but it isn’t the utter embarrassment it was just a year ago.

This may be my last 4th of July to celebrate for the next few years, as I will soon be moving to Scotland (more on that in a moment).

2020 was a very bad year, but even halfway through it’s clear that 2021 is going to be a lot better.

This was true for just about everyone. I was no exception.

The direct effects of the pandemic on me were relatively minor.

Transitioning to remote work was even easier than I expected it to be; in fact I was even able to run experiments online using the same research subject pool as we’d previously used for the lab. I not only didn’t suffer any financial hardship from the lockdowns, I ended up better off because of the relief payments (and the freezing of student loan payments as well as the ludicrous stock boom, which I managed to buy in near the trough of). Ordering groceries online for delivery is so convenient I’m tempted to continue it after the pandemic is over (though it does cost more).

I was careful and/or fortunate enough not to get sick (now that I am fully vaccinated, my future risk is negligible), as were most of my friends and family. I am not close to anyone who died from the virus, though I do have some second-order links to some who died (grandparents of a couple of my friends, the thesis advisor of one of my co-authors).

It was other things, that really made 2020 a miserable year for me. Some of them were indirect effects of the pandemic, and some may not even have been related.

For me, 2020 was a year full of disappointments. It was the year I nearly finished my dissertation and went on the job market, applying for over one hundred jobs—and got zero offers. It was the year I was scheduled to present at an international conference—which was then canceled. It was the year my papers were rejected by multiple journals. It was the year I was scheduled to be married—and then we were forced to postpone the wedding.

But now, in 2021, several of these situations are already improving. We will be married on October 9, and most (though assuredly not all) of the preparations for the wedding are now done. My dissertation is now done except for some formalities. After over a year of searching and applying to over two hundred postings in all, I finally found a job, a postdoc position at the University of Edinburgh. (A postdoc isn’t ideal, but on the other hand, Edinburgh is more prestigious than I thought I’d be able to get.) I still haven’t managed to publish any papers, but I no longer feel as desperate a need to do so now that I’m not scrambling to find a job. Now of course we have to plan for a move overseas, though fortunately the university will reimburse our costs for the visa and most of the moving expenses.

Of course, 2021 isn’t over—neither is the COVID pandemic. But already it looks like it’s going to be a lot better than 2020.

When to give up

Jun 6 JDN 2459372

Perseverance is widely regarded as a virtue, and for good reason. Often one of the most important deciding factors in success is the capacity to keep trying after repeated failure. I think this has been a major barrier for me personally; many things came easily to me when I was young, and I internalized the sense that if something doesn’t come easily, it must be beyond my reach.

Yet it’s also worth noting that this is not the only deciding factor—some things really are beyond our capabilities. Indeed, some things are outright impossible. And we often don’t know what is possible and what isn’t.

This raises the question: When should we persevere, and when should we give up?

There is actually reason to think that people often don’t give up when they should. Steven Levitt (of Freakonomics fame)recently published a study that asked people who were on the verge of a difficult decision to flip a coin, and then base their decision on the coin flip: Heads, make a change; tails, keep things as they are. Many didn’t actually follow the coin flip—but enough did that there was a statistical difference between those who saw heads and those who saw tails. The study found that the people who flipped heads and made a change were on average happier a couple of years later than the people who flipped tails and kept things as they were.

This question is particularly salient for me lately, because the academic job market has gone so poorly for me. I’ve spent most of my life believing that academia is where I belong; my intellect and my passion for teaching and research has convinced me and many others that this is the right path for me. But now that I have a taste of what it is actually like to apply for tenure-track jobs and submit papers to journals, I am utterly miserable. I hate every minute of it. I’ve spent the entire past year depressed and feeling like I have accomplished absolutely nothing.

In theory, once one actually gets tenure it’s supposed to get easier. But that could be a long way away—or it might never happen at all. As it is, there’s basically no chance I’ll get a tenure track position this year, and it’s unclear what my chances would be if I tried again next year.

If I could actually get a paper published, that would no doubt improve my odds of landing a better job next year. But I haven’t been able to do that, and each new rejection cuts so deep that I can barely stand to look at my papers anymore, much less actually continue submitting them. And apparently even tenured professors still get their papers rejected repeatedly, which means that this pain will never go away. I simply cannot imagine being happy if this is what I am expected to do for the rest of my life.

I found this list of criteria for when you should give up something—and most of them fit me. I’m not sure I know in my heart it can’t work out, but I increasingly suspect that. I’m not sure I want it anymore, now that I have a better idea of what it’s really like. Pursuing it is definitely making me utterly miserable. I wouldn’t say it’s the only reason, but I definitely do worry what other people will think if I quit; I feel like I’d be letting a lot of people down. I also wonder who I am without it, where I belong if not here. I don’t know what other paths are out there, but maybe there is something better. This constant stream of failure and rejection has definitely made me feel like I hate myself. And above all, when I imagine quitting, I absolutely feel an enormous sense of relief.

Publishing in journals seems to be the thing that successful academics care about most, and it means almost nothing to me anymore. I only want it because of all the pressure to have it, because of all the rewards that come from having it. It has become fully instrumental to me, with no intrinsic meaning or value. I have no particular desire to be lauded by the same system that lauded Fischer Black or Kenneth Rogoff—both of whose egregious and easily-avoidable mistakes are responsible for the suffering of millions people around the world.

I want people to read my ideas. But people don’t actually read journals. They skim them. They read the abstracts. They look at the graphs and regression tables. (You have the meeting that should have been an email? I raise you the paper that should have been a regression table.) They see if there’s something in there that they should be citing for their own work, and if there is, maybe then they actually read the paper—but everyone is so hyper-specialized that only a handful of people will ever actually want to cite any given paper. The vast majority of research papers are incredibly tedious to read and very few people actually bother. As a method for disseminating ideas, this is perhaps slightly better than standing on a street corner and shouting into a megaphone.

I would much rather write books; people sometimes actually read books, especially when they are written for a wide audience and hence not forced into the straitjacket of standard ‘scientific writing’ that no human being actually gets any enjoyment out of writing or reading. I’ve seen a pretty clear improvement in writing quality of papers written by Nobel laureates—after they get their Nobels or similar accolades. Once they establish themselves, they are free to actually write in ways that are compelling and interesting, rather than having to present everything in the most dry, tedious way possible. If your paper reads like something that a normal person would actually find interesting or enjoyable to read, you will be—as I have been—immediately told that you must remove all such dangerous flavor until the result is as tasteless as possible.

No, the purpose of research journals is not to share ideas. Its function is not to share, but to evaluate. And it isn’t even really to evaluate research—it’s to evaluate researchers. It’s to outsource the efforts of academic hiring to an utterly unaccountable and arbitrary system run mostly by for-profit corporations. It may have some secondary effect of evaluating ideas for validity; at least the really awful ideas are usually excluded. But its primary function is to decide the academic pecking order.

I had thought that scientific peer review was supposed to select for truth. Perhaps sometimes it does. It seems to do so reasonably well in the natural sciences, at least. But in the social sciences? That’s far less clear. Peer-reviewed papers are much more likely to be accurate than any randomly-selected content; but there are still a disturbingly large number of peer-reviewed published papers that are utterly wrong, and some unknown but undoubtedly vast number of good papers that have never seen the light of day.

Then again, when I imagine giving up on an academic career, I don’t just feel relief—I also feel regret and loss. I feel like I’ve wasted years of my life putting together a dream that has now crumbled in my hands. I even feel some anger, some sense that I was betrayed by those who told me that this was about doing good research when it turns out it’s actually about being thick-skinned enough that you can take an endless assault of rejections. It feels like I’ve been running a marathon, and I just rounded a curve to discover that the last five miles must be ridden on horseback, when I don’t have a horse, I have no equestrian training, and in fact I’m allergic to horses.

I wish someone had told me it would be like this. Maybe they tried and I didn’t listen. They did say that papers would get rejected. They did say that the tenure track was high-pressure and publish-or-perish was a major source of anxiety. But they never said that it would tear at my soul like this. They never said that I would have to go through multiple rounds of agony, self-doubt, and despair in order to get even the slighest recognition for my years of work. They never said that the whole field would treat me like I’m worthless because I can’t satisfy the arbitrary demands of a handful of anonymous reviewers. They never said that I would begin to feel worthless after several rounds of this.

That’s really what I want to give up on. I want to give up on hitching my financial security, my career, my future, my self-worth to a system as capricious as peer review.

I don’t want to give up on research. I don’t want to give up on teaching. I still believe strongly in discovering new truths and sharing them with others. I’m just increasingly realizing that academia isn’t nearly as good at that as I thought it was.

It isn’t even that I think it’s impossible for me to succeed in academia. I think that if I continued trying to get a tenure-track job, I would land one eventually. Maybe next year. Or maybe I’d spend a few years at a postdoc first. And I’d probably manage to publish some paper in some reasonably respectable journal at some point in the future. But I don’t know how long it would take, or how good a journal it would be—and I’m already past the point where I really don’t care anymore, where I can’t afford to care, where if I really allowed myself to care it would only devastate me when I inevitably fail again. Now that I see what is really involved in the process, how arduous and arbitrary it is, publishing in a journal means almost nothing to me. I want to be validated; I want to be appreciated; I want to be recognized. But the system is set up to provide nothing but rejection, rejection, rejection. If even the best work won’t be recognized immediately and even the worst work can make it with enough tries, then the whole system begins to seem meaningless. It’s just rolls of the dice. And I didn’t sign up to be a gambler.

The job market will probably be better next year than it was this year. But how much better? Yes, there will be more openings, but there will also be more applicants: Everyone who would normally be on the market, plus everyone like me who didn’t make it this year, plus everyone who decided to hold back this year because they knew they wouldn’t make it (as I probably should have done). Yes, in a normal year, I could be fairly confident of getting some reasonably decent position—but this wasn’t a normal year, and next year won’t be one either, and the one after that might still not be. If I can’t get a paper published in a good journal between now and then—and I’m increasingly convinced that I can’t—then I really can’t expect my odds to be greatly improved from what they were this time around. And if I don’t know that this terrible gauntlet is going to lead to something good, I’d really much rather avoid it altogether. It was miserable enough when I went into it being (over)confident that it would work out all right.

Perhaps the most important question when deciding whether to give up is this: What will happen if you do? What alternatives do you have? If giving up means dying, then don’t give up. (“Learn to let go” is very bad advice to someone hanging from the edge of a cliff.) But while it may feel that way sometimes, rarely does giving up on a career or a relationship or a project yield such catastrophic results.

When people are on the fence about making a change and then do so, even based on the flip of a coin, it usually makes them better off. Note that this is different from saying you should make all your decisions randomly; if you are confident that you don’t want to make a change, don’t make a change. This advice is for people who feel like they want a change but are afraid to take the chance, people who find themselves ambivalent about what direction to go next—people like me.

I don’t know where I should go next. I don’t know where I belong. I know it isn’t Wall Street. I’m pretty sure it’s not consulting. Maybe it’s nonprofits. Maybe it’s government. Maybe it’s freelance writing. Maybe it’s starting my own business. I guess I’d still consider working in academia; if Purdue called me back to say they made a terrible mistake and they want me after all, I’d probably take the offer. But since such an outcome is now vanishingly unlikely, perhaps it’s time, after all, to give up.

Social science is broken. Can we fix it?

May 16 JDN 2459349

Social science is broken. I am of course not the first to say so. The Atlantic recently published an article outlining the sorry state of scientific publishing, and several years ago Slate Star Codex published a lengthy post (with somewhat harsher language than I generally use on this blog) showing how parapsychology, despite being obviously false, can still meet the standards that most social science is expected to meet. I myself discussed the replication crisis in social science on this very blog a few years back.

I was pessimistic then about the incentives of scientific publishing be fixed any time soon, and I am even more pessimistic now.

Back then I noted that journals are often run by for-profit corporations that care more about getting attention than getting the facts right, university administrations are incompetent and top-heavy, and publish-or-perish creates cutthroat competition without providing incentives for genuinely rigorous research. But these are widely known facts, even if so few in the scientific community seem willing to face up to them.

Now I am increasingly concerned that the reason we aren’t fixing this system is that the people with the most power to fix it don’t want to. (Indeed, as I have learned more about political economy I have come to believe this more and more about all the broken institutions in the world. American democracy has its deep flaws because politicians like it that way. China’s government is corrupt because that corruption is profitable for many of China’s leaders. Et cetera.)

I know economics best, so that is where I will focus; but most of what I’m saying here would also apply to other social sciences such as sociology and psychology as well. (Indeed it was psychology that published Daryl Bem.)

Rogoff and Reinhart’s 2010 article “Growth in a Time of Debt”, which was a weak correlation-based argument to begin with, was later revealed (by an intrepid grad student! His name is Thomas Herndon.) to be based upon deep, fundamental errors. Yet the article remains published, without any notice of retraction or correction, in the American Economic Review, probably the most prestigious journal in economics (and undeniably in the vaunted “Top Five”). And the paper itself was widely used by governments around the world to justify massive austerity policies—which backfired with catastrophic consequences.

Why wouldn’t the AER remove the article from their website? Or issue a retraction? Or at least add a note on the page explaining the errors? If their primary concern were scientific truth, they would have done something like this. Their failure to do so is a silence that speaks volumes, a hound that didn’t bark in the night.

It’s rational, if incredibly selfish, for Rogoff and Reinhart themselves to not want a retraction. It was one of their most widely-cited papers. But why wouldn’t AER’s editors want to retract a paper that had been so embarrassingly debunked?

And so I came to realize: These are all people who have succeeded in the current system. Their work is valued, respected, and supported by the system of scientific publishing as it stands. If we were to radically change that system, as we would necessarily have to do in order to re-align incentives toward scientific truth, they would stand to lose, because they would suddenly be competing against other people who are not as good at satisfying the magical 0.05, but are in fact at least as good—perhaps even better—actual scientists than they are.

I know how they would respond to this criticism: I’m someone who hasn’t succeeded in the current system, so I’m biased against it. This is true, to some extent. Indeed, I take it quite seriously, because while tenured professors stand to lose prestige, they can’t really lose their jobs even if there is a sudden flood of far superior research. So in directly economic terms, we would expect the bias against the current system among grad students, adjuncts, and assistant professors to be larger than the bias in favor of the current system among tenured professors and prestigious researchers.

Yet there are other motives aside from money: Norms and social status are among the most powerful motivations human beings have, and these biases are far stronger in favor of the current system—even among grad students and junior faculty. Grad school is many things, some good, some bad; but one of them is a ritual gauntlet that indoctrinates you into the belief that working in academia is the One True Path, without which your life is a failure. If your claim is that grad students are upset at the current system because we overestimate our own qualifications and are feeling sour grapes, you need to explain our prevalence of Impostor Syndrome. By and large, grad students don’t overestimate our abilities—we underestimate them. If we think we’re as good at this as you are, that probably means we’re better. Indeed I have little doubt that Thomas Herndon is a better economist than Kenneth Rogoff will ever be.

I have additional evidence that insider bias is important here: When Paul Romer—Nobel laureate—left academia he published an utterly scathing criticism of the state of academic macroeconomics. That is, once he had escaped the incentives toward insider bias, he turned against the entire field.

Romer pulls absolutely no punches: He literally compares the standard methods of DSGE models to “phlogiston” and “gremlins”. And the paper is worth reading, because it’s obviously entirely correct. He pulls no punches and every single one lands on target. It’s also a pretty fun read, at least if you have the background knowledge to appreciate the dry in-jokes. (Much like “Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity.” I still laugh out loud every time I read the phrase “hegemonic Zermelo-Frankel axioms”, though I realize most people would be utterly nonplussed. For the unitiated, these are the Zermelo-Frankel axioms. Can’t you just see the colonialist imperialism in sentences like “\forall x \forall y (\forall z, z \in x \iff z \in y) \implies x = y”?)

In other words, the Upton Sinclair Principle seems to be applying here: “It is difficult to get a man to understand something when his salary depends upon not understanding it.” The people with the most power to change the system of scientific publishing are journal editors and prestigious researchers, and they are the people for whom the current system is running quite swimmingly.

It’s not that good science can’t succeed in the current system—it often does. In fact, I’m willing to grant that it almost always does, eventually. When the evidence has mounted for long enough and the most adamant of the ancien regime finally retire or die, then, at last, the paradigm will shift. But this process takes literally decades longer than it should. In principle, a wrong theory can be invalidated by a single rigorous experiment. In practice, it generally takes about 30 years of experiments, most of which don’t get published, until the powers that be finally give in.

This delay has serious consequences. It means that many of the researchers working on the forefront of a new paradigm—precisely the people that the scientific community ought to be supporting most—will suffer from being unable to publish their work, get grant funding, or even get hired in the first place. It means that not only will good science take too long to win, but that much good science will never get done at all, because the people who wanted to do it couldn’t find the support they needed to do so. This means that the delay is in fact much longer than it appears: Because it took 30 years for one good idea to take hold, all the other good ideas that would have sprung from it in that time will be lost, at least until someone in the future comes up with them.

I don’t think I’ll ever forget it: At the AEA conference a few years back, I went to a luncheon celebrating Richard Thaler, one of the founders of behavioral economics, whom I regard as one of the top 5 greatest economists of the 20th century (I’m thinking something like, “Keynes > Nash > Thaler > Ramsey > Schelling”). Yes, now he is being rightfully recognized for his seminal work; he won a Nobel, and he has an endowed chair at Chicago, and he got an AEA luncheon in his honor among many other accolades. But it was not always so. Someone speaking at the luncheon offhandedly remarked something like, “Did we think Richard would win a Nobel? Honestly most of us weren’t sure he’d get tenure.” Most of the room laughed; I had to resist the urge to scream. If Richard Thaler wasn’t certain to get tenure, then the entire system is broken. This would be like finding out that Erwin Schrodinger or Niels Bohr wasn’t sure he would get tenure in physics.

A. Gary Schilling, a renowned Wall Street economist (read: One Who Has Turned to the Dark Side), once remarked (the quote is often falsely attributed to Keynes): “markets can remain irrational a lot longer than you and I can remain solvent.” In the same spirit, I would say this: the scientific community can remain wrong a lot longer than you and I can extend our graduate fellowships and tenure clocks.

On the quality of matches

Apr 11 JDN 2459316

Many situations in the real world involve matching people to other people: Dating, job hunting, college admissions, publishing, organ donation.

Alvin Roth won his Nobel Prize for his work on matching algorithms. I have nothing to contribute to improving his algorithm; what baffles me is that we don’t use it more often. It would probably feel too impersonal to use it for dating; but why don’t we use it for job hunting or college admissions? (We do use it for organ donation, and that has saved thousands of lives.)

In this post I will be looking at matching in a somewhat different way. Using a simple model, I’m going to illustrate some of the reasons why it is so painful and frustrating to try to match and keep getting rejected.

Suppose we have two sets of people on either side of a matching market: X and Y. I’ll denote an arbitrarily chosen person in X as x, and an arbitrarily chosen person in Y as y. There’s no reason the two sets can’t have overlap or even be the same set, but making them different sets makes the model as general as possible.

Each person in X wants to match with a person in Y, and vice-versa. But they don’t merely want to accept any possible match; they have preferences over which matches would be better or worse.

In general, we could say that people have some kind of utility function: Ux:Y->R and Uy:X->R that maps from possible match partners to the utility of such a match. But that gets very complicated very fast, because it raises the question of when you should keep searching, and when you should stop searching and accept what you have. (There’s a whole literature of search theory on this.)

For now let’s take the simplest possible case, and just say that there are some matches each person will accept, and some they will reject. This can be seen as a special case where the utility functions Ux and Uy always yield a result of 1 (accept) or 0 (reject).

This defines a set of acceptable partners for each person: A(x) is the set of partners x will accept: {y in Y|Ux(y) = 1} and A(y) is the set of partners y will accept: {x in X|Uy(x) = 1}

Then, the set of mutual matches than x can actually get is the set of ys that x wants, which also want x back: M(x) = {y in A(x)|x in A(y)}

Whereas, the set of mutual matches that y can actually get is the set of xs that y wants, which also want y back: M(y) = {x in A(y)|y in A(x)}

This relation is mutual by construction: If x is in M(y), then y is in M(x).

But this does not mean that the sets must be the same size.

For instance, suppose that there are three people in X, x1, x2, x3, and three people in Y, y1, y2, y3.

Let’s say that the acceptable matches are as follows:

A(x1) = {y1, y2, y3}

A(x2) = {y2, y3}

A(x3) = {y2, y3}

A(y1) = {x1,x2,x3}

A(y2) = {x1,x2}

A(y3) = {x1}

This results in the following mutual matches:

M(x1) = {y1, y2, y3}

M(y1) = {x1}

M(x2) = {y2}

M(y2) = {x1, x2}

M(x3) = {}

M(y3) = {x1}

x1 can match with whoever they like; everyone wants to match with them. x2 can match with y2. But x3, despite having the same preferences as x2, and being desired by y3, can’t find any mutual matches at all, because the one person who wants them is a person they don’t want.

y1 can only match with x1, but the same is true of y3. So they will be fighting over x1. As long as y2 doesn’t also try to fight over x1, x2 and y2 will be happy together. Yet x3 will remain alone.

Note that the number of mutual matches has no obvious relation with the number of individually acceptable partners. x2 and x3 had the same number of acceptable partners, but x2 found a mutual match and x3 didn’t. y1 was willing to accept more potential partners than y3, but got the same lone mutual match in the end. y3 was only willing to accept one partner, but will get a shot at x1, the one that everyone wants.

One thing is true: Adding another acceptable partner will never reduce your number of mutual matches, and removing one will never increase it. But often changing your acceptable partners doesn’t have any effect on your mutual matches at all.

Now let’s consider what it must feel like to be x1 versus x3.

For x1, the world is their oyster; they can choose whoever they want and be guaranteed to get a match. Life is easy and simple for them; all they have to do is decide who they want most and that will be it.

For x3, life is an endless string of rejection and despair. Every time they try to reach out to suggest a match with someone, they are rebuffed. They feel hopeless and alone. They feel as though no one would ever actually want them—even though in fact there is someone who wants them, it’s just not someone they were willing to consider.

This is of course a very simple and small-scale model; there are only six people in it, and they each only say yes or no. Yet already I’ve got x1 who feels like a rock star and x3 who feels utterly hopeless if not worthless.

In the real world, there are so many more people in the system that the odds that no one is in your mutual match set are negligible. Almost everyone has someone they can match with. But some people have many more matches than others, and that makes life much easier for the ones with many matches and much harder for the ones with fewer.

Moreover, search costs then become a major problem: Even knowing that in all probability there is a match for you somewhere out there, how do you actually find that person? (And that’s not even getting into the difficulty of recognizing a good match when you see it; in this simple model you know immediately, but in the real world it can take a remarkably long time.)

If we think of the acceptable partner sets as preferences, they may not be within anyone’s control; you want what you want. But if we instead characterize them as decisions, the results are quite differentand I think it’s easy to see them, if nothing else, as the decision of how high to set your standards.

This raises a question: When we are searching and not getting matches, should we lower our standards and add more people to our list of acceptable partners?

This simple model would seem to say that we should always do that—there’s no downside, since the worst that can happen is nothing. And x3 for instance would be much happier if they were willing to lower their standards and accept y1. (Indeed, if they did so, there would be a way to pair everyone off happily: x1 with y3, x2 with y2, and x3 with y1.)

But in the real world, searching is often costly: There is at least the involved, and often a literal application or submission fee; but perhaps worst of all is the crushing pain of rejection. Under those circumstances, adding another acceptable partner who is not a mutual match will actually make you worse off.

That’s pretty much what the job market has been for me for the last six months. I started out with the really good matches: GiveWell, the Oxford Global Priorities Institute, Purdue, Wesleyan, Eastern Michigan University. And after investing considerable effort into getting those applications right, I made it as far as an interview at all those places—but no further.

So I extended my search, applying to dozens more places. I’ve now applied to over 100 positions. I knew that most of them were not good matches, because there simply weren’t that many good matches to be found. And the result of all those 100 applications has been precisely 0 interviews. Lowering my standards accomplished absolutely nothing. I knew going in that these places were not a good fit for me—and it looks like they all agreed.

It’s possible that lowering my standards in some different way might have worked, but even this is not clear: I’ve already been willing to accept much lower salaries than a PhD in economics ought to entitle, and included positions in my search that are only for a year or two with no job security, and applied to far-flung locales across the globe that I don’t know if I’d really be willing to move to.

Honestly at this point I’ve only been using the following criteria: (1) At least vaguely related to my field (otherwise they wouldn’t want me anyway), (2) a higher salary than I currently get as a grad student (otherwise why bother?), (3) a geographic location where homosexuality is not literally illegal and an institution that doesn’t actively discriminate against LGBT employees (this rules out more than you’d think—there are at least three good postings I didn’t apply to on these grounds), (4) in a region that speaks a language I have at least some basic knowledge of (i.e. preferably English, but also allowing Spanish, French, German, or Japanese) (5) working conditions that don’t involve working more than 40 hours per week (which has severely detrimental health effects, even ignoring my disability which would compound the effects), and (6) not working for a company that is implicated in large-scale criminal activity (as a remarkable number of major banks have in fact been implicated). I don’t feel like these are unreasonably high standards, and yet so far I have failed to land a match.

What’s more, the entire process has been emotionally devastating. While others seem to be suffering from pandemic burnout, I don’t think I’ve made it that far; I think I’d be just as burnt out even if there were no pandemic, simply from how brutal the job market has been.

Why does rejection hurt so much? Why does being turned down for a date, or a job, or a publication feel so utterly soul-crushing? When I started putting together this model I had hoped that thinking of it in terms of match-sets might actually help reduce that feeling, but instead what happened is that it offered me a way of partly explaining that feeling (much as I did in my post on Bayesian Impostor Syndrome).

What is the feeling of rejection? It is the feeling of expending search effort to find someone in your acceptable partner set—and then learning that you were not in their acceptable partner set, and thus you have failed to make a mutual match.

I said earlier that x1 feels like a rock star and x3 feels hopeless. This is because being present in someone else’s acceptable partner set is a sign of status—the more people who consider you an acceptable partner, the more you are “worth” in some sense. And when it’s something as important as a romantic partner or a career, that sense of “worth” is difficult to circumscribe into a particular domain; it begins to bleed outward into a sense of your overall self-worth as a human being.

Being wanted by someone you don’t want makes you feel superior, like they are “beneath” you; but wanting someone who doesn’t want you makes you feel inferior, like they are “above” you. And when you are applying for jobs in a market with a Beveridge Curve as skewed as ours, or trying to get a paper or a book published in a world flooded with submissions, you end up with a lot more cases of feeling inferior than cases of feeling superior. In fact, I even applied for a few jobs that I felt were “beneath” my level—they didn’t take me either, perhaps because they felt I was overqualified.

In such circumstances, it’s hard not to feel like I am the problem, like there is something wrong with me. Sometimes I can convince myself that I’m not doing anything wrong and the market is just exceptionally brutal this year. But I really have no clear way of distinguishing that hypothesis from the much darker possibility that I have done something terribly wrong that I cannot correct and will continue in this miserable and soul-crushing fruitless search for months or even years to come. Indeed, I’m not even sure it’s actually any better to know that you did everything right and still failed; that just makes you helpless instead of defective. It might be good for my self-worth to know that I did everything right; but it wouldn’t change the fact that I’m in a miserable situation I can’t get out of. If I knew I were doing something wrong, maybe I could actually fix that mistake in the future and get a better outcome.

As it is, I guess all I can do is wait for more opportunities and keep trying.