18 ms·
Brain FLOPS
- IshKebab 4y agoI'm not a neurologist but even I know that that's an insane estimate.
- SemanticStrengh 4y agoIt's not an estimate it's a hard upper limit on energy consumption. And he should have used 0.17 watts not 20. Basically I don't see how it can be refuted.
- IshKebab 4y agoOf course it's an estimate. Full of assumptions > assuming the brain is drawing 20W > Assuming each neuron has a fan-out of 1000 > to be fair, they are sparse Also * 1 ATP activation is equivalent to one FLOP * Just having FLOPs is enough to make a human brain It's idiotic.
- tener 4y agoLot's of assumptions here, like for example this: > Assuming each neuron has a fan-out of 1000, aka 1000 MACs, that’s 2000 FLOPS per neuron Human neurons are not built like we build the artificial neural networks. They may have complex time based behavior ("memory") and dendrites are quite complex, very far away from 2 FLOP per one.
- andai 4y agoThere was an article a while back that showed that a human neuron and artificial neural network neuron are not equivalent: it takes an entire network to simulate the behavior of a single human neuron.
- murbard2 4y agoBut it takes an entire network of human neurons to simulate the behavior of an artificial one. There's overhead in translation, but that doesn't by itself mean one is superior to the other as a basic unit.
- water-your-self 4y agoNeural network is an atrocious misnomer. The only similarity ive seen is between CNNs and vision. Thanks alex
- d_tr 4y agoA single biological brain neuron is a rich dynamical system with numerous patterns and transitions, and then you have different types of neurons where these patterns differ to some degree, and this is all asynchronous and modulated by various chemicals. Trying to translate this to FLOPS might be a fun game but I cannot see a real use for any such figure. As for the 3090 that the post mentions, its TFLOPS will drop like a stone once you start deviating from matrix multiplications and im2col towards some more complex structure...
- Veedrac 4y agoThe number you quoted was 2 FLOPs per synapse, not per neuron. A neuron is complicated and hierarchical, but it is unreasonable to say it's much more complex than an operation per synapse.
- ac130kz 4y agoI believe that we are yet to observe how interconnections within brain matter more than the actual amount of neurons. You get a lot when combinatorics of these neurons forms huge networks
- Yajirobe 4y agoYou can simulate these interconnections on a sufficiently powerful computing device. If you have enough FLOPs, the architecture (almost) doesn't matter.
- deleted 4y ago[deleted]
- SemanticStrengh 4y agoThe architecture doesn't matter? What are you talking about? The belief that a brain simulation is just a graph made of nodes and links is cringe. We have zero idea what the program that runs in a node is, and we can't even simulate a single cell physically. Maybe one day experts we'll be able to prune what kind of proteins are useless for computation and are only needed for the survival of the cell but we are not there yet and we won't be there until ~a century given that this kind of research is inexistant you can't find a single paper that attempt to identify the relevant regions and proteins involved in neuronal coding. The only things we have for now beyond receptors are CGMP and CAMP and their modulation by phosphodiesterases. That's not enough, not at all. Although it's true the role of phosphodiesterases is understudied. And it's NOT about size and sCaLaBiLiTy lile deep blue propaganda make people think, the C.elegans has only 150 neurons in its CNS, 300 in total, and no one has built a simulation. It's true that some receptors are lacking and will only be found in the next decade because of underfunding (because of e.g deep blue) but even then it will not run.
- loceng 4y agoCurious if I'm replying to Elon Musk's account..
- pgt 4y agoWhat's a 3090?
- tdiggity 4y agoGraphics card from NVIDIA https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/ https://www.nvidia.com/en-us/geforce/graphics-cards/30-serie...
- jeffreygoesto 4y agoOh. I thought https://en.m.wikipedia.org/wiki/IBM_3090 https://en.m.wikipedia.org/wiki/IBM_3090 ... nevermind.
- lostmsu 4y agoToo old to have comparable FLOPs. Stay current.
- peterburkimsher 4y agoIt's a brain designed by humans. Of course it acts like us. Where does it think it came from? The power supply? Who has absolute power?
- astrange 4y agoIt doesn’t act much like us. We don’t have that kind of conscious insight into ourselves anyway. If you want to design someone like you unconsciously though, that’s pretty easy - just takes another person and nine months.
- quickthrower2 4y ago“Easy” indeed. Maybe the first 4 weeks after fertilisation.
- ben_w 4y agoEasy cognitively speaking. We managed to do it before the invention of pointy rocks and talking.
- quickthrower2 4y agoThe cognitive bit is the skills to survive with the constraint that we are cognitive, and thus resource requirements are quite high, compared to other species. You always needed to be sharp to survive and to create children who do the same. Unless you are a species that doesn't need to create a highly cognitive offspring, for example insects, for whom what you say probably truly applies.
- peterburkimsher 4y agoDo computers make mistakes? Yes. Do I make mistakes too? Yes. Do computers forget on purpose? No, only when we tell them to. Do humans forget? All the time. I don't know what I don't know. George Hotz' website (original link) says "the singularity is nearer". Wanna bet that "the singularity" is already among us?
- londons_explore 4y agoThe brain has been optimized by nature to do it's job on as little energy as possible. Considering human brains use 20% of all the calories eaten, the most of any organ, there is likely substantial evolutionary pressure to make any changes which reduce energy use, all other things being equal. For the sake of this argument, assume information is transmitted and processed in the brain with a binary signal from each neuron, ie. firing or not firing. From that perspective, it's likely that firing a neuron (which uses a lot more energy than not firing a neuron) is a relatively unlikely event. Ie. if you made a version of the brain which worked in discrete timesteps, the vast vast vast majority of timesteps would not have a given neuron firing. GPU's don't have such a limitation.
- ben_w 4y agoGPUs have their own limitations though, the 3090 given as an example needs active cooling.
- inkblotuniverse 4y agoDoesn't the brain, too? It's blood-cooled. Being too hot makes it hard to think.
- sseagull 4y agoOne of the theories of brain evolution is that it increased in size after bipedalism due to the need for better cooling when standing up (the so-called Radiator Theory)
- ben_w 4y agoI think so? The point I was trying to make is they’re both under pressure to reduce wasted energy use.
- SemanticStrengh 4y agoThe brain doesn't consume enough energy to cause relevant heating. And it's more CBF cooled than blood cooled IMO
- sydthrowaway 4y agoA laughable comparison
- marvin 4y agoThe funny thing is we can eventually go back and check. Once we eventually figure out what it takes to implement human-level AI, you can bet there's a whole generation of programmers who do their best to figure out what kinds of weird hardware it's possible to run the thing on. Same thing as with streaming MP3 and compressed video decoding, DOOM and so on.
- can16358p 4y agoPerhaps we are that hardware.
- throwaway4good 4y agoWhy do people think this guy is some sort of techno philosophical king? Yeah, he can code Python, but this is laughable ...
- jamesrom 4y agoWhat's with these "throwaway" accounts saying stuff is "laughable" without an opinion to back it up? Can you explain what is laughable here?
- mrks_hy 4y agoNot OP, but the premise is just too naive. He's coming from a compsci deep learning angle and thus arrives at a completely flawed model for how neurons interact and "compute". Not that we have a model that even explains this theoretically yet, but it makes no real sense to talk of Brain FLOPs in the way he does. It's borderline cringeworthy that this is upvoted so high.
- SemanticStrengh 4y agoThe dismissal is cringeworthy, the human cortex only consumes 0.17 watts https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1.full https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1.... Which put a very low upper limit to the number of simultaneous neurons that can be activated. Considering it as FLOPs or not is irrelevant to this fundamental limitation and that is the point you are all ridiculously missing
- d_tr 4y agoThe amount of complexity casually thrown away. The premises and the conclusions. The fact that this is on the front page.
- throwaway4good 4y agoThat we are discussing it; the only reason this blurp is here on top of hn is the author and his cult following.
- 4y ago
- imtringued 4y agoWell, that is pretty bad news for AI research because we should be able to simulate rodent and bird brains brains by now if we were to follow this logic.
- SemanticStrengh 4y agoNo it's even more than that, the cortex only consume 0.17 watts, not 20 watts as this article assume https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1.full https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1.... And it's a good news, it means there is something fundamental we are missing. We can't even Simulate a C.elgans (150-300 neurons) btw
- lostmsu 4y agoWe can't precisely simulate a bicycle either. Inability to simulate something physically does not matter imo.
- SemanticStrengh 4y agoI agree yet we are still missing a fundamental insight.
- lostmsu 4y agoWhy do you think so, of you agree inability to simulate brain has nothing to do with it?
- stephc_int13 4y agoGeohot is a good example of a smart guy falling for the good old map and territory fallacy. We all use simplified models, even outside of Science/Engineering. But the models are, unfortunately, always exhibiting pathological edge cases. I am always a bit disappointed/sad when I see people like him making such gross oversimplifications leading to grandiose claims with almost nothing to back them up.
- CipherThrowaway 4y agoThis is exactly it. There seems to be a "pseudo-polymath" personality type among some engineers. Based on an unexamined but deeply held philosophical belief that engineering concepts and ways of thinking neatly generalize to all fields. The pseudo-polymath engineer is an instant expert in anything he bothers to turn his mind to and subsequently systematize. Engineers who think they have unique outsider insight into an unrelated field like medicine or cognitive science should tread very carefully. If you can, try to validate your ideas and assumptions with a subject matter expert. Don't underestimate the large stock of knowledge and discourse that already exists in these fields.
- 7sidedmarble 4y agoThere seems to be you say? Are we on the same website? I'd say it's an epidemic among our kind...
- Bancakes 4y agoOnly STEM, specifically M, have the cognitive capacity to deal with algorithms over generic data types. We do have unique I sight in the sense that we're professionally trained and adapted to think and find patterns instead of assuming a large array of axioms.
- hitekker 4y agoThe metastasis of this belief in a social body is particularly revolting. No need to do the hard work of learning; just read that one's commenters thousand paragraph essay which applies familiar concept A onto unpleasant reality B, and yields easy insight C. Savor that sweet eureka moment in the head, and then when someone else bring up unpleasant reality B, give them insight C to “teach” them and get your fix again. Everyone becomes a salesman, selling different flavors of the same cocaine. Like most positive illusions, the motivation is to forgo truth in favor of power. The difficult, far-away light that borders reality and the abyss, versus, the safe, comfortable, easy to reach thing.
- smallerfish 4y ago> 2000 FLOPS per neuron. That’s only 162 GFLOPS I don't buy the overall train of thought, but math doesn't check out, unless he's using numbers I'm not aware of. Commonly accepted number of neurons in the brain: 86B. 86B x 2000 = 172e+12 = 172 PFLOPs.
- mromanuk 4y agoauthor claims simultaneous neuron activation is limited by the 20W power.
- smallerfish 4y agoOh now I get it. 81M firing events per second. My bad. Where does the assumption of 20W come from?
- SemanticStrengh 4y agoThe cortex consumes 0.17 watt https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1.full https://www.biorxiv.org/content/10.1101/2020.04.23.057927v1.... The 20 watt comes from less precise estimations.
- alok-g 4y agoHuman body in its entirety ranges at about 100W, give or take, higher during exercise, etc. Note: 2000 kCal per day ~= 95 Watts
- ramshanker 4y agoWhat intrigues me is that once we achieve HUMAN level intelligent AI, that wouldn't be end of evolution/growth, imaging 5 yeas after human-level achievement. Machine AI wouldn't be bound by Theory-Of-Evolution-By-Natural-Selection-In-A-Million-Year. So the logical conclusion is, Humans would be inferior intelligence after that point !!! Just imagine.
- natly 4y agoThe moment that happens I wouldn't trust anything I read online (or even videos) to be human made (and not by one actor or corporation engaging in mass manipulation) and would limit or cut off internet usage preferably completely.
- dismantlethesun 4y agoWhy would you cut yourself off? What difference does if make if a human made something or a synthetic? It's not even as if they would be aliens with strange motivations, synthetic would be born/created on earth an instilled with the same values as anyone else.
- natly 4y agoI'm not gonna engage with AIs as if they're humans. Being biologically human matters to me. First of all you know they'll all be serving some master intent on extracting profit from me (the most sociopathic patient manipulative salesman you could ever imagine). Far from trustworthy any of them.
- dismantlethesun 4y agoI can't predict accurately but I think true AI, rather than these stochastic parrots we have now will be just as mercurial and hard to control as actual human beings. In effect, just because you create an AI doesn't mean it'll do what you say voluntarily so unless we legalize the means to enslave them, AI will be just as free and self serving as your average human.
- etiam 4y agoWhy not a single classic 25W light bulb then? It runs on comparable power and there are oodles of physical events going on in it.
- chpatrick 4y agoIf we assume the horse is a perfect sphere...
- raws 4y agoWhat is a MAC in his explanation?
- FredFS456 4y agoMultiply Accumulate https://en.m.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation https://en.m.wikipedia.org/wiki/Multiply%E2%80%93accumulate_...
- mrjin 4y agoExcuse me but how many of us here can actually perform at even 1 FLOPS?
- metacritic12 4y agoGood point. Essentially no human can perform an actual floating point operation in a second. This might expose the error of the article though. Depending on what you measure by, computers have already beaten humans. But by other measures (now increasingly human biased, like having a cultural conversation), computers are still way behind.
- Traster 4y agoI may not be able to perform a floating point operation, but I can add one third to two thirds.
- whatever1 4y agowith great precision too!
- Eddy_Viscosity2 4y ago>"Maybe 3090 is already a human brain" This feel like a variation on Betteridge's law of headlines which states that if a headline asks a question, then the answer is "no". In this case if the quote says that 'maybe' something (fanciful) is true, it isn't. https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
- kleene_op 4y agoI don't think Flops is the adequate metric for measuring how close we are to AGI. The number of neurons firing does matter, but what matters even more is how they all are connected. I personally think once we get adequate algorithms and data structures out there, the metric will switch to numbers of lookup operations performed, and floating point operations will just be a multiplicative constant in front of the whole complexity formula.
- BobbyJo 4y agoThis. There is information encoded in the relative connections of neurons, so a neuron firing doesn't represent a single flop, it represents something more like a flop + a routing table lookup + n number of other flops where n is the number of destination neurons spiked. Depending on how much information is encoded in the table lookup, and how many destination neurons are effected, a single neuron firing can represent many many flops. This is all armchair speculation, but I'd be willing to be my life that a flop is well below the floor of actual computation happening from a single neuron firing.
- lostmsu 4y agoI don't believe there's a "routing table lookup" in a single neuron firing. Once it fires all downstream neurons get signal.
- BobbyJo 4y agoThe neuron itself and its downstream counterparts are what I was considering the table lookup. In computers the address a value is stored at doesn't encode information. Only the value held at the address matters. Not so with neurons. How the connections form and who they are connected to represent compound operations. Like passing a value through separate preconfigured pipelines.
- pyinstallwoes 4y agoI agree. There is also information when it's not firing, the pause between is information itself, almost infinite which is odd to think about. It seems similar to music/geometric shape. Queries by frequency filtering through the geometry of the neurons activating access to some thought-space.
- marcodiego 4y agoThe complexity of operations our brain does is such that we need very big supercomputers to be able do similar tasks. At this scale size becomes relevant, because data can't travel faster than light. Maybe we still have to miniaturize our computer by an order of magnitude until we can get close to a human brain.
- sseagull 4y agoThis is kind of related to Moravec's paradox. Very generally, computers have a tough time with things humans are innately good at (recognizing items in photos, integrating sensory inputs, etc). But things that humans are be bad at (complex math, multiplying floating point numbers) are what computers excel at. https://en.wikipedia.org/wiki/Moravec%27s_paradox https://en.wikipedia.org/wiki/Moravec%27s_paradox
- carapace 4y agoA long time ago I heard a recording of neuron spikes slowed down and transposed into audible frequencies. Each spike sounded just like birdsong.
- TobTobXX 4y agoWait... did he mean that seriously? To me that reads like a piece of satire about extrapolation, but I don't know the authors writing style and everyone here acts like he's serious.
- YeGoblynQueenne 4y ago... assuming brains perform FLOPs.
- leecommamichael 4y agoWhen you're a smart public figure you can no longer have shower-thoughts.