12 ms·
When will computer hardware match the human brain? (1998)
- 2genders46305 3y ago[flagged]
- 2genders39738 3y ago[flagged]
- 2genders30497 3y ago[flagged]
- 2genders33556 3y ago[flagged]
- SEXMCNIGGA34288 3y ago[dead]
- 2genders37015 3y ago[flagged]
- SEXMCNIGGA10539 3y ago[flagged]
- SEXMCNIGGA36632 3y ago[flagged]
- SEXMCNIGGA47800 3y ago[flagged]
- cs702 3y agoMoravec's plots (from 1998!) are looking more and more like prophecies: https://www.jetpress.org/volume1/power_075.jpg https://www.jetpress.org/volume1/power_075.jpg https://www.jetpress.org/volume1/All_things_075.jpg https://www.jetpress.org/volume1/All_things_075.jpg
- BitwiseFool 3y agoWhat I find incredible about the progress of computing power is that there isn't anything that actually makes Moore's law a given. Engineers keep discovering advances in materials science and manufacturing that enable advancements on such a consistent pace. Is this a fluke? Luck? What enables this?
- almostnormal 3y agoThe market does. New stuff needs to be sold every year, and needs to be some amount faster to find a buyer. The cost of development is minimized while still reaching that goal, limiting the gain to no more than necessary.
- Teever 3y agoThat's like saying that the storke brings babies. It doesn't really answer the question, it just pushes it off for another day. Why don't we see exponential density growth in other products? What would that even look lke anyways?
- jcranmer 3y agoTens of billions of dollars of investment specifically to keep up with Moore's Law. It's basically akin to something like the Manhattan Project or the Apollo Program, just something initiated by private corporations rather than a government-directed investment program.
- loandbehold 3y agoOne thing driving Moore's law is the every generation of computers is used to design/simulate next generation. It's a positive feedback loop.
- semi-extrinsic 3y agoExcept for the scale on the right. Which suggests that the nematode worm could be simulated reliably in 1998, but here we are in 2024 and still can't do it...
- SEXMCNIGGA21062 3y ago[dead]
- SEXMCNIGGA2481 3y ago[flagged]
- SEXMCNIGGA13091 3y ago[dead]
- jll29 3y agoMoravac (in the linked paper): "In both cases, the evidence for an intelligent mind lies in the machine's performance, not its makeup." Do you agree? I'm much less keen to ascribe "intelligence" to large, pretrained language models given that I know how primitive their training regime is compared to a scenario where I might have been "blended" by their ability to "chat" (double quote here since I know ChatGPT and the likes do not have a memory, so all prior interactions have to be re-submitted with each turn of a conversation). Intuitively, I'd be more prone to ascribe intelligence based on convincing ways of construction that go along with intellignece-like performance, especially if the model also makes human-like errors.
- awesomeMilou 3y agoReducing the capability of the human brain to performance alone is too simplistic, especially when looking at LLM's. Even if we would assign some intelligence to LLM's, they need a 400w GPU at inference time, and several orders of magnitude more of those at training time. The human brain runs constanly at ~20w. I highly doubt you'd be able to get even close to that kind of performance with current manufacturing processes. We'd need something entirely different from laser lithography for that to happen.
- psunavy03 3y ago> We'd need something entirely different from laser lithography for that to happen. Like, you know . . . nerve cells.
- syndicatedjelly 3y agoCare to explain?
- psunavy03 3y agoI was being facetious in that brains already exist.
- ben_w 3y ago
- smusamashah 3y agoQuote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phenomenal growth must soon come to an end. I don't know where to look to confirm that can current computers do 1 billion MIPS (Million Instructions per Second?) as predicted by this article? EDIT: This Wikipedia article puts an AMD CPU at ~2 million MIPS https://en.wikipedia.org/wiki/Instructions_per_second https://en.wikipedia.org/wiki/Instructions_per_second as the highest one.
- kolinko 3y agoM1/M2s are around 2-4 trillion if I’m not mistaken
- foobiekr 3y agonot even within 2 orders of magnitude.
- rerdavies 3y agoJust rack-mount a hundred iphones. :-P
- beautifulfreak 3y ago~2.6 teraflops FP32 (CPU) for the M3 Max.
- ben_w 3y agoWhen the article was written, almost nobody cared about GPUs or similar architectures being really good at parallel computation. So while he wrote about MIPS, and those have indeed basically stopped significantly improving, FLOPS have continued to improve. And for AI in particular, for inference at least, we can get away with 8 bit floats, which is why my phone does 1.58e13/second, only a factor of x60 from a million-billion.
- HarHarVeryFunny 3y agoIt's a tough question since we still don't understand the brain well enough to know what degree of fidelity of copying it is necessary to achieve the same/similar functionality. What is the computational equivalent of a biological neuron? How much of the detailed chemistry is really relevant to it's operation (doing useful computation), or can we just use the ANN model of synapses as weights, and the neuron itself as a summation device plus a non-linearity? Maybe we can functionally model the human brain at a much coarser, more efficient, level than individual neurons - at cortical mini/macro column level perhaps, or as an abstract architecture not directly related to nature's messy implementation?
- vrnvu 3y agoAren't we there yet? George Hotz has a nice series of posts on this: https://geohot.github.io/blog/jekyll/update/2020/08/20/a-silicon-person.html https://geohot.github.io/blog/jekyll/update/2020/08/20/a-sil... https://geohot.github.io/blog/jekyll/update/2022/02/17/brain-flops.html https://geohot.github.io/blog/jekyll/update/2022/02/17/brain... https://geohot.github.io/blog/jekyll/update/2023/04/26/a-person-of-compute.html https://geohot.github.io/blog/jekyll/update/2023/04/26/a-per... I sorted them by date.
- nicklecompte 3y ago[flagged]
- deleted 3y ago[deleted]
- abeppu 3y agoHmm, this reasoning is making a lot of really questionable assumptions: > We’ll use the estimates of 100 billion neurons and 100 trillion synapses for this post. That’s 100 teraweights. ... or maybe actual synapses cannot be described with a single weight. > The max firing rate seems to be 200hz [4]. I really want an estimate of “neuron lag” here, but let’s upper bound it at 5ms. ... but ANNs output float activations per pass. Biological neurons encode values with sequences of spikes which vary in timing, so the firing rate doesn't on its own tell you the rate at which neurons can communicate new values. > Remember also, that the brain is always learning, so it needs to be doing forward and backward passes. I’m not exactly sure why they are different, but [6] and [7] point to the backward pass taking 2x more compute than the forward pass. ... but the brain probably isn't doing backprop, in part because it doesn't get to observe the 'correct' output, compute a loss, etc, and because the brain isn't a DAG.
- vrnvu 3y agoYeah, fore sure. I just share it as a fun read. I think they have been discussed in HN before.
- crispyambulance 3y agoMoravec also wrote a book much along these lines in the late 80's (Mind Children). If I recall correctly, a good part of it was also about the idea of "transferring" human consciousness into a machine "host". The idea being that a sufficiently advanced computer would be able to somehow make sense of a human's neuron mappings and then "continue running" as that individual. It was provocative back then, and still is! The idea of having my consciousness continue its existence indefinitely in somebody's kubernetes cluster, however, seems like a very special vision of hell.
- syntheticnature 3y agoSee https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- ycombinete 3y agoThis is also the premise of Greg Egan’s novel Permutation City. Also a provocative read.
- SEXMCNIGGA15290 3y ago[flagged]
- SEXMCNIGGA43793 3y ago[flagged]
- geysersam 3y agoThe estimate for how many operations per second the brain does is quite a wild guess. It starts out very reasonable, estimating the amount of information the eyes feed to the brain. But from there on it's really just a wild guess. We don't know how the brain processes visual input, we don't know what the fundamental unit of processing in the brain is, or if there is one.
- nyrikki 3y agoIt is more than a wild guess, it is a guess based on the outdated perceptron model of the brain. Active dendrites the that can do operations like xor before anything reaches the soma, or use spike timing, once again before the soma are a couple of examples. SNNs, or spikey artificial NNs have been hitting limits of the computable numbers. Riddled basins as an example, which are sets with no open subsets. Basically any circle you can draw will always contain at least one point on a boundary set. https://arxiv.org/abs/1711.02160 https://arxiv.org/abs/1711.02160 This is a much stronger constraint than even classic chaos. While I am sure we will continue to try to make comparisons, wetwear and hardware are cheese and chalk. Both have things they are better at and neither is a direct replacement for the other.
- hax0ron3 3y agoIt also blows my mind that that, even though DNA seems to just code for proteins and does not store a schematic for a brain in any way that we've been able to decipher so far, human eggs pretty reliably end up growing into people who 9 months after conception already have a bunch of stuff, including visual processing, working. Of course the DNA is not the only input, there is also the mother's body, the whole process of pregnancy, but I don't know how that contributes to the new baby being able to enter the world already intelligent. It's possible that I'm not aware of some breakthroughs on this topic, though.
- HarHarVeryFunny 2y agoDNA does a lot more than coding for proteins. It also controls how genes are expressed (how same gene does different things in different contexts) and how cells differentiate into different types (neurons, skin, muscle, etc) depending on location. In this way, DNA does control the blueprint for the brain, down to the initial generic wiring scheme of the cortex - six layers of differentiated neuron types, with a specific pattern of inter-connectivity. As a rough analogy, you could think of this DNA-controlled initial wiring scheme as something like an untrained transformer... the architecture is there, but won't do anything useful until it has been exposed to a lot of data, which will complete the wiring scheme (synapses = brain's model weights). The brain is going to start to learn as soon as it has sufficiently developed, which is certainly before birth, and newborn babies have been shown to already respond to things like their mothers voice which they were exposed to in the womb. Intelligence - ability to learn - is different from knowledge. A newborn baby has very little knowledge, but it does have intelligence since that comes from the brain architecture, hence ability to learn, that our DNA "encodes".
- freitzkriesler2 3y agoI'd argue the issue isn't hardware anymore but "software" or algorithms that get the hardware to start acting human.
- thangalin 3y ago2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We'll likely have real-time brain analogs by 2064. (Aside, these are the dates I've used in my hard sci-fi novel. See my profile for details.) [1]: https://pubmed.ncbi.nlm.nih.gov/19226510/ https://pubmed.ncbi.nlm.nih.gov/19226510/ [2]: https://historyofinformation.com/detail.php?id=3901 https://historyofinformation.com/detail.php?id=3901 [3]: https://en.wikipedia.org/wiki/Transistor_count https://en.wikipedia.org/wiki/Transistor_count
- hulitu 3y agoToo many presumptions.
- smt88 3y ago400 transistors per synapse is a foundational assumption and is total nonsense
- Terr_ 3y agoAlso, at what rate? If it's not at least one second per second, that changes things quite a bit.
- idiotsecant 3y agoSo list your better assumptions. The exercise is guesstimating the future. Gather your unique perspective and offer up some upper and lower bounds.
- RaftPeople 3y ago> * 86 billion neurons in a brain [1]* > * 400 transistors to simulate a synapse [2]* > * That's 34 trillion, 400 billion transistors to simulate a human brain.* You forgot about: 1-Astrocytes - more common than neurons, computational, have their own style of internal and external signaling, have bi-directional communication with neurons at the synapse (and are thought to control the neurons) 2-Synapse: a single synapse can be both excitatory and inhibitory depending on it's structure and the permeant ions inside and outside the cell at that point in time and space - are your 400 transistors handling all of that dynamic capability? 3-Brain waves: are now shown to be causal (influencing neuron behavior) not just a by-product of activity. Many different types of waves (different frequencies) that operate very locally (high freq), or broadly (low freq). Information is encoded in frequency, phase, space and time. This is the summary, the details are even more interesting and complex. Some of the details are probably not adding computational capabilities, but many clearly are.
- SEXMCNIGGA4855 3y ago[flagged]
- SEXMCNIGGA14371 3y ago[flagged]
- SEXMCNIGGA31239 3y ago[flagged]
- SEXMCNIGGA19130 3y ago[flagged]
- SEXMCNIGGA43340 3y ago[flagged]
- SEXMCNIGGA35773 3y ago[flagged]
- SEXMCNIGGA3278 3y ago[flagged]
- SEXMCNIGGA21622 3y ago[flagged]
- SEXMCNIGGA3871 3y ago[flagged]
- SEXMCNIGGA9706 3y ago[flagged]
- SEXMCNIGGA12960 3y ago[dead]
- SEXMCNIGGA35432 3y ago[flagged]
- SEXMCNIGGA14072 3y ago[flagged]
- SEXMCNIGGA21520 3y ago[flagged]
- SEXMCNIGGA5378 3y ago[dead]
- SEXMCNIGGA9389 3y ago[dead]
- deleted 3y ago[deleted]
- SEXMCNIGGA21266 3y ago[dead]
- SEXMCNIGGA36027 3y ago[dead]
- SEXMCNIGGA34809 3y ago[dead]
- SEXMCNIGGA14342 3y ago[flagged]
- SEXMCNIGGA29742 3y ago[dead]
- SEXMCNIGGA8425 3y ago[dead]
- SEXMCNIGGA19183 3y ago[dead]
- SEXMCNIGGA43928 3y ago[dead]
- SEXMCNIGGA36243 3y ago[dead]
- SEXMCNIGGA39234 3y ago[dead]
- 2genders38990 3y ago[dead]
- 2genders17644 3y ago[dead]
- 2genders24797 3y ago[dead]
- 2genders47384 3y ago[dead]
- 2genders10948 3y ago[dead]
- 2genders33033 3y ago[dead]
- 2genders34744 3y ago[dead]
- 2genders26071 3y ago[dead]
- 2genders43304 3y ago[dead]
- 2genders8774 3y ago[flagged]
- 2genders31778 3y ago[flagged]
- 2genders23289 3y ago[flagged]
- 2genders38144 3y ago[flagged]
- 2genders29226 3y ago[flagged]
- 2genders36912 3y ago[flagged]
- jakeogh 3y agoI bet never. Brains are not computers. https://news.ycombinator.com/item?id=39751509 https://news.ycombinator.com/item?id=39751509 https://news.ycombinator.com/item?id=9885558 https://news.ycombinator.com/item?id=9885558 https://news.ycombinator.com/item?id=21253267 https://news.ycombinator.com/item?id=21253267
- nikolay 3y agoNext century.
- atemerev 3y agoAt my workplace, we are now completing the multiscale simulation of a full mouse brain (neuron/synaptic activity, cellular metabolism, etc). It fits into our small supercomputer first deployed in 2019, and there's a large potential for simplification not yet realized. I believe that a supercomputer from the top 5 can already simulate a human brain. But we don't have the data and the theory yet. We are developing it, though.
- sigmoid10 3y agoAre you talking about real time or faster simulation of complete cellular metabolism incl. neural activity across an entire mouse brain? If not, then I'd say I can run a simulation like this on my home computer. It will just take forever to compute something meaningful. But if yes, that is insanely impressive, even though this won't scale to the human brain any time soon since it has 1000x more neurons and exponentially more synapses.
- obruchez 3y agoHe's working on the Blue Brain Project, apparently. https://en.wikipedia.org/wiki/Blue_Brain_Project https://en.wikipedia.org/wiki/Blue_Brain_Project
- Phiwise_ 3y agoProbably not at least until we know for sure what the hardware of the human brain is architected like. So not for quite a while, still.
- RecycledEle 3y agoNobody has a clue how many operations per second the human brain performs, so the usual comparisons are make up garbage. An intelligent question is: how does one machine compare to the human brain on one dimension of performance? A slide rule easily outperforms the human brain on some mathematical computations.
- deleted 3y ago[deleted]