7
$\begingroup$

How many flops my brain can process, or how many GHz is a human brain capable of? Is it valid to think that each celular brain is like a small cpu? (like cuda architecture). Our brains works in parallel, right?

$\endgroup$
  • 4
    $\begingroup$ Both FLOPS and GHz are technical terms that are tailored to CPUs as we build them. They don't apply to human brains in a meaningful way. (I guess we'd come out at $\approx 10$ FLOPS while we can outperform server farms in image processing and pattern recognition.) It's similar with the term "parallel"; I'm not sure whether this applies in the same way as it does to computers. Do we have more than one "thread" of conscious thought? Probably not, for most people. Do we have hundreds (thousands?) of subconscious "subroutines"? Hell yea. $\endgroup$ – Raphael Jan 28 '14 at 8:24
6
$\begingroup$

Here is what we know for sure. The largest neuronal network simulation to date was achieved using the K Japanese supercomputer (see the Top500) during summer 2013, a few months ago.

Using the open-source software NEST, the scientists simulated a network consisting of 1.73 billion nerve cells connected by 10.4 trillion synapses. To realize this feat, the program recruited 82,944 processors of the K Computer and used 1 petabyte of memory. The process took 40 minutes, to complete the simulation of 1 second of neuronal network activity in real, biological, time. Although the simulated network is huge, it only represents 1% of the neuronal network in the brain.

K is a peta-scale supercomputer, so, we need an exa-scale machine to completely simulate the whole human brain. These machines will be (probably) available between 2017-2020.

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ Of course, any simulation adds significant overhead so this does not allow a reasonable estimate of a brain's processing power. $\endgroup$ – Raphael Jan 28 '14 at 8:25
  • 4
    $\begingroup$ On the one hand, you need a huge computer to simulate a human brain; on the other hand, you need a huge number of human brains to simulate a desktop computer. $\endgroup$ – David Richerby Jan 28 '14 at 9:52
  • $\begingroup$ an even more interesting related question is what did it do or learn? ... and its telling that reports on the subject say little except that the simulation "ran" ... related research in this area deep learning $\endgroup$ – vzn Jan 29 '14 at 23:17
  • $\begingroup$ This represents a lower bound for the brain's estimated "processing power" because of efficiency losses. $\endgroup$ – Awn Oct 31 '18 at 2:31
3
$\begingroup$

This is an apples and oranges comparison. There is no known way to measure the CPU power of a neuron because it is "apparently" not computing in a way that computers do. However, there are some scientists who are willing to extrapolate somewhat using estimates based on the retina processing images vs computer algorithms doing the same, and so on. Of course this should be taken as a "SWAG". For example:

The 1,500 cubic centimeter human brain is about 100,000 times as large as the retina, suggesting that matching overall human behavior will take about 100 million MIPS of computer power.

–Moravec, When will computer hardware match the human brain, page 2, 1997

Also, the trends of Moore's law can be used to estimate the arrival of such computing capacity (also with the counterissue that some think it may be slowing).

Abstract of the Moravec paper:

This paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain are estimated. Based on extrapolation of past trends and on examination of technologies under development, it is predicted that the required hardware will be available in cheap machines in the 2020s.

However, this is evading the key issue because while the picture may be somewhat understandable for hardware it is not known what (or, strictly, whether) algorithms (software) can "create" or "exhibit" intelligence. In fact, it is quite conceivable that Google or some other company (say Amazon) right now has the magic threshold of 100 M-MIPS cloud computing capacity. For this reason it is quite plausible to conjecture that if machine intelligence is ever constructed it will first be exhibited in a supercomputer or a computing cloud.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ by the way, it was an offhand comment about google's/amazon's actual computing capacity. would like to know estimates of roughly how much computing capacity they have; some of this detail is not openly shared. might research that sometime. $\endgroup$ – vzn Jan 29 '14 at 23:19
3
$\begingroup$

Here's my 2 cents on the question, please take it with many grains of salt as I'm not a neuro scientist.

If we make the assumption that every change in neuronal state is equal to a floating point operation then we get:

  • For full nervous system: $200 \times 86e9 / 1e9 = 17200$ GFLOPS
  • For the cerebral cortex alone: $200 \times 23e9 / 1e9 = 4600$ GFLOPS

This simplification might be grossly underestimating the amount of computation that is going on in the brain. If you think about the very abstracted model of neurons that is commonly used in machine learning, where a neuron performs an operation of the form $$out = f(\sum_i w_i in_i)$$, you see that a biological neuron performs a very large number of floating point operations to arrive at it's output. This number seems to be roughly proportional to the number of synapses ($10e4$ to $10e5$ on average). If we take that into account then:

  • For full nervous system: $172e6$ to $1720e6$ GFLOPS
  • For the cerebral cortex alone: $46e6$ to $460e6$ GFLOPS

Just for comparison the K80 GPU from NVIDIA can do 8740 GFLOPS while the current fastest supercomputer clocks at 33 PFLOPS (33e6 GFLOPS).

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

Ralph Merkle, a computer scientist at Xerox PARC, published a paper in 1989 evaluating intellectual processing power. He measured it in three different ways.

Method One: There are about 1 quadrillion synapses in the brain. They process about 10 nerve impulses per second. Therefore the brain carries out about 10 quadrillion synapse operations per second.

Method Two: The human retina (which has its own processing power and is relatively well understood) contains about 100 million nerve cells performing about 10 billion addition operations per second. The brain is bigger than the retina by a factor of somewhere between 100 and 10,000. Therefore the brain must process between 1 and 100 trillion operations per second.

Method Three: The human brain consumes about 25 watts of energy, of which about 10 watts are used directly for mental processes. We know the power consumption of a single synapse and can estimate the average distance between synapses. This means we can figure the maximum number of synapse operations that can be supported by the brain's "power supply." The upper limit turns out to be 2 quadrillion synapse operations per second.

Averaging out these estimates, it looks as if the brain may run at around 1 quadrillion synapse operations per second.

from: http://archive.wired.com/wired/archive/5.01/ffgenome.html?pg=9&topic=

Merkle's paper: http://www.merkle.com/brainLimits.html

| cite | improve this answer | |
$\endgroup$
-1
$\begingroup$

[Since I can't add a comment above...]

http://www.wsj.com/articles/amazons-spending-leads-to-another-loss-1414095239 suggests that Amazon is spending on the order of 5% of revenue on data centers. Revenue was around \$90 billion/year in 2014, so \$4.5 billion. Their servers will be replaced every two or three years, so let's use two years of revenue to estimate the cost of their data centers and equipment: \$8 billion. Take around half of that for operations (electricity). Then we'll estimate around \$4,000 per server. Assume a server has a couple of hefty cpus, 128GB of ram, 10Gb networking, a few terabyte hard drives. So you get a million servers using that approach.

[This estimate is probably within an order of magnitude or two. Amazon probably doesn't spend \$400 or \$40,000 per server, and probably doesn't spend either \$400 million or \$40 billion on their server hardware, and probably doesn't have either 16GB or 1TB of ram per server.]

Moravec suggests that a typical computer will balance memory and computational capacity to the tune of 1MB/Mips. So a server is 128Gips. Total computation capacity would them be around 128 billion Mips. Using the Moravec estimate, that's about 1,000 humans. Merkle suggests it might be as low as 10 humans.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This has nothing to do with the question. $\endgroup$ – David Richerby Jul 31 '15 at 7:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.