NSA

What is the processing power of the human brain?

pnrj cognitive science, futurism , , , , , , , , , , , , , , , , ,

JDN 2457485

Futurists have been predicting that AI will “surpass humans” any day now for something like 50 years. Eventually they’ll be right, but it will be more or less purely by chance, since they’ve been making the same prediction longer than I’ve been alive. (Similarity, whenever someone projects the date at which immortality will be invented, it always seems to coincide with just slightly before the end of the author’s projected life expectancy.) Any technology that is “20 years away” will be so indefinitely.

There are a lot of reasons why this prediction keeps failing so miserably. One is an apparent failure to grasp the limitations of exponential growth. I actually think the most important is that a lot of AI fans don’t seem to understand how human cognition actually works—that it is primarily social cognition, where most of the processing has already been done and given to us as cached results, some of them derived centuries before we were born. We are smart enough to run a civilization with airplanes and the Internet not because any individual human is so much smarter than any other animal, but because all humans together are—and other animals haven’t quite figured out how to unite their cognition in the same way. We’re about 3 times smarter than any other animal as individuals—and several billion times smarter when we put our heads together.

A third reason is that even if you have sufficient computing power, that is surprisingly unimportant; what you really need are good heuristics to make use of your computing power efficiently. Any nontrivial problem is too complex to brute-force by any conceivable computer, so simply increasing computing power without improving your heuristics will get you nowhere. Conversely, if you have really good heuristics like the human brain does, you don’t even need all that much computing power. A chess grandmaster was once asked how many moves ahead he can see on the board, and he replied: “I only see one move ahead. The right one.” In cognitive science terms, people asked him how much computing power he was using, expecting him to say something far beyond normal human capacity, and he replied that he was using hardly any—it was all baked into the heuristics he had learned from years of training and practice.

Making an AI capable of human thought—a true artificial person—will require a level of computing power we can already reach (as long as we use huge supercomputers), but that is like having the right material. To really create the being we will need to embed the proper heuristics. We are trying to make David, and we have finally mined enough marble—now all we need is Michelangelo.

But another reason why so many futurists have failed in their projections is that they have wildly underestimated the computing power of the human brain. Reading 1980s cyberpunk is hilarious in hindsight; Neuromancer actually quite accurately projected the number of megabytes that would flow through the Internet at any given moment, but somehow thought that a few hundred megaflops would be enough to copy human consciousness. The processing power of the human brain is actually on the order of a few petaflops. So, you know, Gibson was only off by a factor of a few million.

We can now match petaflops—the world’s fastest supercomputer is actually about 30 petaflops. Of course, it cost half a month of China’s GDP to build, and requires 24 megawatts to run and cool, which is about the output of a mid-sized solar power station. The human brain consumes only about 400 kcal per day, which is about 20 watts—roughly the consumption of a typical CFL lightbulb. Even if you count the rest of the human body as necessary to run the human brain (which I guess is sort of true), we’re still clocking in at about 100 watts—so even though supercomputers can now process at the same speed, our brains are almost a million times as energy-efficient.

How do I know it’s a few petaflops?

Earlier this year a study was published showing that a conservative lower bound for the total capacity of human memory is about 4 bits per synapse, where previously some scientists thought that each synapse might carry only 1 bit (I’ve always suspected it was more like 10 myself).

So then we need to figure out how many synapses we have… which turns out to be really difficult actually. They are in a constant state of flux, growing, shrinking, and moving all the time; and when we die they fade away almost immediately (reason #3 I’m skeptical of cryonics). We know that we have about 100 billion neurons, and each one can have anywhere between 100 and 15,000 synapses with other neurons. The average seems to be something like 5,000 (but highly skewed in a power-law distribution), so that’s about 500 trillion synapses. If each one is carrying 4 bits to be as conservative as possible, that’s a total storage capacity of about 2 quadrillion bits, which is about 0.2 petabytes.

Of course, that’s assuming that our brains store information the same way as a computer—every bit flipped independently, each bit stored forever. Not even close. Human memory is constantly compressing and decompressing data, using a compression scheme that’s lossy enough that we not only forget things, we can systematically misremember and even be implanted with false memories. That may seem like a bad thing, and in a sense it is; but if the compression scheme is that lossy, it must be because it’s also that efficient—that our brains are compressing away the vast majority of the data to make room for more. Our best lossy compression algorithms for video are about 100:1; but the human brain is clearly much better than that. Our core data format for long-term memory appears to be narrative; more or less we store everything not as audio or video (that’s short-term memory, and quite literally so), but as stories.

How much compression can you get by storing things as narrative? Think about The Lord of the Rings. The extended edition of the films runs to 6 discs of movie (9 discs of other stuff), where a Blu-Ray disc can store about 50 GB. So that’s 300 GB. Compressed into narrative form, we have the books (which, if you’ve read them, are clearly not optimally compressed—no, we do not need five paragraphs about the trees, and I’m gonna say it, Tom Bombadil is totally superfluous and Peter Jackson was right to remove him), which run about 500,000 words altogether. If the average word is 10 letters (normally it’s less than that, but this is Tolkien we’re talking about), each word will take up about 10 bytes (because in ASCII or Unicode a letter is a byte). So altogether the total content of the entire trilogy, compressed into narrative, can be stored in about 5 million bytes, that is, 5 MB. So the compression from HD video to narrative takes us all the way from 300 GB to 5 MB, which is a factor of 60,000. Sixty thousand. I believe that this is the proper order of magnitude for the compression capability of the human brain.

Even more interesting is the fact that the human brain is almost certainly in some sense holographic storage; damage to a small part of your brain does not produce highly selective memory loss as if you had some bad sectors of your hard drive, but rather an overall degradation of your total memory processing as if you in some sense stored everything everywhere—that is, holographically. How exactly this is accomplished by the brain is still very much an open question; it’s probably not literally a hologram in the quantum sense, but it definitely seems to function like a hologram. (Although… if the human brain is a quantum computer that would explain an awful lot—it especially helps with the binding problem. The problem is explaining how a biological system at 37 C can possibly maintain the necessary quantum coherences.) The data storage capacity of holograms is substantially larger than what can be achieved by conventional means—and furthermore has similar properties to human memory in that you can more or less always add more, but then what you had before gradually gets degraded. Since neural nets are much closer to the actual mechanics of the brain as we know them, understanding human memory will probably involve finding ways to simulate holographic storage with neural nets.

With these facts in mind, the amount of information we can usefully take in and store is probably not 0.2 petabytes—it’s probably more like 10 exabytes. The human brain can probably hold just about as much as the NSA’s National Cybersecurity Initiative Data Center in Utah, which is itself more or less designed to contain the Internet. (The NSA is at once awesome and terrifying.)

But okay, maybe that’s not fair if we’re comparing human brains to computers; even if you can compress all your data by a factor of 100,000, that isn’t the same thing as having 100,000 times as much storage.

So let’s use that smaller figure, 0.2 petabytes. That’s how much we can store; how much can we process?

The next thing to understand is that our processing architecture is fundamentally difference from that of computers.

Computers generally have far more storage than they have processing power, because they are bottlenecked through a CPU that can only process 1 thing at once (okay, like 8 things at once with a hyperthreaded quad-core; as you’ll see in a moment this is a trivial difference). So it’s typical for a new computer these days to have processing power in gigaflops (It’s usually reported in gigahertz, but that’s kind of silly; hertz just tells you clock cycles, while what you really wanted to know is calculations—and that you get from flops. They’re generally pretty comparable numbers though.), while they have storage in terabytes—meaning that it would take about 1000 seconds (about 17 minutes) for the computer to process everything in its entire storage once. In fact it would take a good deal longer than that, because there are further bottlenecks in terms of memory access, especially from hard-disk drives (RAM and solid-state drives are faster, but would still slow it down to a couple of hours).

The human brain, by contrast, integrates processing and memory into the same system. There is no clear distinction between “memory synapses” and “processing synapses”, and no single CPU bottleneck that everything has to go through. There is however something like a “clock cycle” as it turns out; synaptic firings are synchronized across several different “rhythms”, the fastest of which is about 30 Hz. No, not 30 GHz, not 30 MHz, not even 30 kHz; 30 hertz. Compared to the blazing speed of billions of cycles per second that goes on in our computers, the 30 cycles per second our brains are capable of may seem bafflingly slow. (Even more bafflingly slow is the speed of nerve conduction, which is not limited by the speed of light as you might expect, but is actually less than the speed of sound. When you trigger the knee-jerk reflex doctors often test, it takes about a tenth of a second for the reflex to happen—not because your body is waiting for anything, but because it simply takes that long for the signal to travel to your spinal cord and back.)

The reason we can function at all is because of our much more efficient architecture; instead of passing everything through a single bottleneck, we do all of our processing in parallel. All of those 100 billion neurons with 500 trillion synapses storing 2 quadrillion bits work simultaneously. So whereas a computer does 8 things at a time, 3 billion times per second, a human brain does 2 quadrillion things at a time, 30 times per second. Provided that the tasks can be fully parallelized (vision, yes; arithmetic, no), a human brain can therefore process 60 quadrillion bits per second—which turns out to be just over 6 petaflops, somewhere around 6,000,000,000,000,000 calculations per second.

So, like I said, a few petaflops.

9/11, 14 years on—and where are our civil liberties?

pnrj public policy , , , , , , , , , , , , , , , , , ,

JDN 2457278 (09/11/2015) EDT 20:53

Today is the 14th anniversary of the 9/11 attacks. A lot has changed since then—yet it’s quite remarkable what hasn’t. In particular, we still don’t have our civil liberties back.

In our immediate panicked response to the attacks, the United States passed almost unanimously the USA PATRIOT ACT, giving unprecedented power to our government in surveillance, searches, and even arrests and detentions. Most of those powers have been renewed repeatedly and remain in effect; the only major change has been a slight weakening of the NSA’s authority to use mass dragnet surveillance on Internet traffic and phone metadata. And this change in turn was almost certainly only made because of Edward Snowden, who is still forced to live in Russia for fear of being executed if he returns to the US. That is, the man most responsible for the only significant improvement in civil liberties in the United States in the last decade is living in Russia because he has been branded a traitor. No, the traitors here are the over one hundred standing US Congress members who voted for an act that is in explicit and direct violation of the Constitution. At the very least every one of them should be removed from office, and we as voters have the power to do that—so why haven’t we? In particular, why are Dan Lipinski and Steny Hoyer, both Democrats from non-southern states who voted every single time to extend provisions of the PATRIOT ACT, still in office? At least Carl Levin had the courtesy to resign after sponsoring the act allowing indefinite detention—I hope we would have voted him out anyway, since I’d much rather have a Republican (and all the absurd economic policy that entails) than someone who apparently doesn’t believe the Fourth and Sixth Amendments have any meaning at all.

We have become inured to this loss of liberty; it feels natural or inevitable to us. But these are not minor inconveniences; they are not small compromises. Giving our government the power to surveil, search, arrest, imprison, torture, and execute anyone they want at any time without the system of due process—and make no mistake, that is what the PATRIOT ACT and the indefinite detention law do—means giving away everything that separates us from tyranny. Bypassing the justice system and the rule of law means bypassing everything that America stands for.

So far, these laws have actually mostly been used against people reasonably suspected of terrorism, that much is true; but it’s also irrelevant. Democracy doesn’t mean you give the government extreme power and they uphold your trust and use it benevolently. Democracy means you don’t give them that power in the first place.

If there’s really sufficient evidence to support an arrest for terrorism, get a warrant. If you don’t have enough evidence for a warrant, you don’t have enough evidence for an arrest. If there’s really sufficient evidence to justify imprisoning someone for terrorism, get a jury to convict. If you don’t have enough evidence to convince a jury, guess what? You don’t have enough evidence to imprison them. These are not negotiable. They are not “political opinions” in any ordinary sense. The protection of due process is so fundamental to democracy that without it political opinions lose all meaning.

People talk about “Big Government” when we suggest increasing taxes on capital gains or expanding Medicare. No, that isn’t Big Government. Searching without warrants is Big Government. Imprisoning people without trial is Big Government. From all the decades of crying wolf in which any policy someone doesn’t like is accused of being “tyranny”, we seem to have lost the ability to recognize actual tyranny. I hope you understand the full force of my meaning when I say that the PATRIOT ACT is literally fascist. Fascism has come to America, and as predicted it was wrapped in the flag and carrying a cross.

In this sort of situation, a lot of people like to quote (or misquote) Benjamin Franklin:

“Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety.”

With the qualifiers “essential” and “temporary”, this quote seems right; but a lot of people forget them and quote him as saying:
“Those would give up liberty to purchase safety, deserve neither liberty nor safety.”

That’s clearly wrong. We do in fact give up liberty to purchase safety, and as well we should. We give up our liberty to purchase weapons-grade plutonium; we give up our liberty to drive at 220 mph. The question we need to be asking is: How much liberty are we giving up to gain how much safety?

Spoken like an economist, the question is not whether you will give up liberty to purchase safety—the question is at what price you’re willing to make the purchase. The price we’ve been paying in response to terrorism is far too high. Indeed, the price we are paying is tantamount to America itself.

As horrific as 9/11 was, it’s important to remember: It only killed 3,000 people.

This statement probably makes you uncomfortable; it may even offend you. How dare I say “only”?

I don’t mean to minimize the harm of those deaths. I don’t mean to minimize the suffering of people who lost friends, colleagues, parents, siblings, children. The death of any human being is the permanent destruction of something irreplaceable, a spark of life that can never be restored; it is always a tragedy and there is never any way to repay it.

But I think people are actually doing the opposite—they are ignoring or minimizing millions of other deaths because those deaths didn’t happen to be dramatic enough. A parent killed by a heart attack is just as lost as a parent who died in 9/11. A friend who died of brain cancer is just as gone as a friend who was killed in a terrorist attack. A child killed in a car accident is just as much a loss as a child killed by suicide bombers. If you really care about human suffering, I contend that you should care about all human suffering, not just the kind that makes the TV news.

Here is a list, from the CDC, of things that kill more Americans per month than terrorists have killed in the last three decades:

Heart disease: 50,900 per month

Cancer: 48,700 per month

Lung disease: 12,400 per month

Accidents: 10,800 per month

Stroke: 10,700 per month

Alzheimer’s: 7,000 per month

Diabetes: 6,300 per month

Influenza: 4,700 per month

Kidney failure: 3,900 per month

Terrorism deaths since 1985: 3,455
Yes, that’s right; influenza kills more Americans per month (on average; flu is seasonal, after all) than terrorism has killed in the last thirty years.
And for comparison, other violent deaths, not quite but almost as many per month as terrorism has killed in my entire life so far:
Suicide: 3,400 per month

Homicide: 1,300 per month

Now, with those figures in mind, I want you to ask yourself the following question: Would you be willing to give up basic, fundamental civil liberties in order to avoid any of these things?

Would you want the government to be able to arrest you and imprison you without trial for eating too many cheeseburgers, so as to reduce the risk of heart disease and stroke?

Would you want the government to monitor your phone calls and Internet traffic to make sure you don’t smoke, so as to avoid lung disease? Or to watch for signs of depression, to reduce the rate of suicide?

Would you want the government to be able to use targeted drone strikes, ordered directly by the President, pre-emptively against probable murderers (with a certain rate of collateral damage, of course), to reduce the rate of homicide?

I presume that the answer to all the above questions is “no”. Then now I have to ask you: Why are you willing to give up those same civil liberties to prevent a risk that is three hundred times smaller?

And then of course there’s the Iraq War, which killed 4,400 Americans and at least 100,000 civilians, and the Afghanistan War, which killed 3,400 allied soldiers and over 90,000 civilians.

In response to the horrific murder of 3,000 people, we sacrificed another 7,800 soldiers and killed another 190,000 innocent civilians. What exactly did that accomplish? What benefit did we get for such an enormous cost?

The people who sold us these deadly wars and draconian policies did so based on the threat that terrorism could somehow become vastly worse, involving the release of some unstoppable bioweapon or the detonation of a full-scale nuclear weapon, killing millions of people—but that has never happened, has never gotten close to happening, and would be thousands of times worse than the worst terrorist attacks that have ever actually happened.

If we’re worried about millions of people dying, it is far more likely that there would be a repeat of the 1918 influenza pandemic, or an accidental detonation of a nuclear weapon, or a flashpoint event with Russia or China triggering World War III; it’s probably more likely that there would be an asteroid impact large enough to kill a million people than there would be a terrorist attack large enough to do the same.

As it is, heart disease is already killing millions of people—about a million every two years—and we aren’t so panicked about that as to give up civil liberties. Elsewhere in the world, malnutrition kills over 3 million children per year, essentially all of it due to extreme poverty, which we could eliminate by spending between a quarter ($150 billion) and a half ($300 billion) of our current military budget ($600 billion); but we haven’t even done that even though it would require no loss of civil liberties at all.

Why is terrorism different? In short, the tribal paradigm.

There are in fact downsides to not being infinite identical psychopaths, and this is one of them. An infinite identical psychopath would simply maximize their own probability of survival; but finite diverse tribalists such as we underreact to some threats (such as heart disease) and overreact to others (such as terrorism). We’ll do almost anything to stop the latter—and almost nothing to stop the former.

Terrorists are perceived as a threat not just to our individual survival like heart disease or stroke, but a threat to our tribe from another tribe. This triggers a deep, instinctual sense of panic and hatred that makes us willing to ignore principles we would otherwise uphold and commit acts of violence we would otherwise find unimaginable.

Indeed, it’s precisely that instinct which motivates the terrorists in the first place. From their perspective, we are the other tribe that threatens their tribe, and they are therefore willing to stop at nothing until we are destroyed.

In a fundamental way, when we respond to terrorism in this way we do not defeat them—we become them.
If you ask people who support the PATRIOT ACT, it’s very clear that they don’t see themselves as imposing upon the civil liberties of Americans. Instead, they see themselves as protecting Americans (our tribe), and they think the impositions upon civil liberties will only harm those who don’t count as Americans (other tribes). This is a pretty bizarre notion if you think about it carefully—if you don’t need a warrant or probable cause to imprison people, then what stops you from imprisoning people who aren’t terrorists?—but people don’t think about it carefully. They act on emotion, on instinct.

The odds of terrorists actually destroying America by killing people are basically negligible. Even the most deadly terrorist attack in recorded history—9/11—killed fewer Americans than die every month from diabetes, or every week from heart disease. Even the most extreme attacks feared (which are extremely unlikely) wouldn’t be any worse than World War II, which of course we won.

But the odds of terrorists destroying America by making us give up the rights and freedoms that define us as a nation? That’s well underway.