How many flops my brain can process, or how many GHz is a human brain capable of? Is it valid to think that each celular brain is like a small cpu? (like cuda architecture). Our brains works in parallel, right?
Using the open-source software NEST, the scientists simulated a network consisting of 1.73 billion nerve cells connected by 10.4 trillion synapses. To realize this feat, the program recruited 82,944 processors of the K Computer and used 1 petabyte of memory. The process took 40 minutes, to complete the simulation of 1 second of neuronal network activity in real, biological, time. Although the simulated network is huge, it only represents 1% of the neuronal network in the brain.
K is a peta-scale supercomputer, so, we need an exa-scale machine to completely simulate the whole human brain. These machines will be (probably) available between 2017-2020.
This is an apples and oranges comparison. There is no known way to measure the CPU power of a neuron because it is "apparently" not computing in a way that computers do. However, there are some scientists who are willing to extrapolate somewhat using estimates based on the retina processing images vs computer algorithms doing the same, and so on. Of course this should be taken as a "SWAG". For example:
The 1,500 cubic centimeter human brain is about 100,000 times as large as the retina, suggesting that matching overall human behavior will take about 100 million MIPS of computer power.
–Moravec, When will computer hardware match the human brain, page 2, 1997
Also, the trends of Moore's law can be used to estimate the arrival of such computing capacity (also with the counterissue that some think it may be slowing).
Abstract of the Moravec paper:
This paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain are estimated. Based on extrapolation of past trends and on examination of technologies under development, it is predicted that the required hardware will be available in cheap machines in the 2020s.
However, this is evading the key issue because while the picture may be somewhat understandable for hardware it is not known what (or, strictly, whether) algorithms (software) can "create" or "exhibit" intelligence. In fact, it is quite conceivable that Google or some other company (say Amazon) right now has the magic threshold of 100 M-MIPS cloud computing capacity. For this reason it is quite plausible to conjecture that if machine intelligence is ever constructed it will first be exhibited in a supercomputer or a computing cloud.
Here's my 2 cents on the question, please take it with many grains of salt as I'm not a neuro scientist.
- The human brain has roughly $86e9$ neurons for the entire nervous system and about $23e9$ for the cerebral cortex (http://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons).
- Biological neurons operate at peak speed of 200Hz (from the Book "Superintelligence" by Nick Bostrom).
If we make the assumption that every change in neuronal state is equal to a floating point operation then we get:
- For full nervous system: $200 \times 86e9 / 1e9 = 17200$ GFLOPS
- For the cerebral cortex alone: $200 \times 23e9 / 1e9 = 4600$ GFLOPS
This simplification might be grossly underestimating the amount of computation that is going on in the brain. If you think about the very abstracted model of neurons that is commonly used in machine learning, where a neuron performs an operation of the form $$out = f(\sum_i w_i in_i)$$, you see that a biological neuron performs a very large number of floating point operations to arrive at it's output. This number seems to be roughly proportional to the number of synapses ($10e4$ to $10e5$ on average). If we take that into account then:
- For full nervous system: $172e6$ to $1720e6$ GFLOPS
- For the cerebral cortex alone: $46e6$ to $460e6$ GFLOPS
Just for comparison the K80 GPU from NVIDIA can do 8740 GFLOPS while the current fastest supercomputer clocks at 33 PFLOPS (33e6 GFLOPS).
Ralph Merkle, a computer scientist at Xerox PARC, published a paper in 1989 evaluating intellectual processing power. He measured it in three different ways.
Method One: There are about 1 quadrillion synapses in the brain. They process about 10 nerve impulses per second. Therefore the brain carries out about 10 quadrillion synapse operations per second.
Method Two: The human retina (which has its own processing power and is relatively well understood) contains about 100 million nerve cells performing about 10 billion addition operations per second. The brain is bigger than the retina by a factor of somewhere between 100 and 10,000. Therefore the brain must process between 1 and 100 trillion operations per second.
Method Three: The human brain consumes about 25 watts of energy, of which about 10 watts are used directly for mental processes. We know the power consumption of a single synapse and can estimate the average distance between synapses. This means we can figure the maximum number of synapse operations that can be supported by the brain's "power supply." The upper limit turns out to be 2 quadrillion synapse operations per second.
Averaging out these estimates, it looks as if the brain may run at around 1 quadrillion synapse operations per second.
Merkle's paper: http://www.merkle.com/brainLimits.html
[Since I can't add a comment above...]
http://www.wsj.com/articles/amazons-spending-leads-to-another-loss-1414095239 suggests that Amazon is spending on the order of 5% of revenue on data centers. Revenue was around \$90 billion/year in 2014, so \$4.5 billion. Their servers will be replaced every two or three years, so let's use two years of revenue to estimate the cost of their data centers and equipment: \$8 billion. Take around half of that for operations (electricity). Then we'll estimate around \$4,000 per server. Assume a server has a couple of hefty cpus, 128GB of ram, 10Gb networking, a few terabyte hard drives. So you get a million servers using that approach.
[This estimate is probably within an order of magnitude or two. Amazon probably doesn't spend \$400 or \$40,000 per server, and probably doesn't spend either \$400 million or \$40 billion on their server hardware, and probably doesn't have either 16GB or 1TB of ram per server.]
Moravec suggests that a typical computer will balance memory and computational capacity to the tune of 1MB/Mips. So a server is 128Gips. Total computation capacity would them be around 128 billion Mips. Using the Moravec estimate, that's about 1,000 humans. Merkle suggests it might be as low as 10 humans.