4 ms·

> A few buckets of sand also contain more elements than there are synapses in the human brain. Actually you'd need about a thousand buckets. https://www.quora

> A few buckets of sand also contain more elements than there are synapses in the human brain. Actually you'd need about a thousand buckets. https://www.quora.com/How-many-grains-of-sand-could-fit-in-a-cylindrical-bucket-1-5-feet-tall-and-1-foot-in-diameter https://www.quora.com/How-many-grains-of-sand-could-fit-in-a... --- Ultimately I think these numbers do matter. As chips approach the size of human brains, only 1,000,000,000x faster, you remove the question of engineering and scale, and are left with just research.

jacquesm 7y ago
Using fine sand I make that out to be about 6 buckets, give or take, besides, this isn't about whether it is 6, 10 or even 100 (or 1000) buckets, it is about how the number itself does not make a huge difference. Before if you wanted 1.2T transistors you needed 50 or so regular large dies, and presumably there have already been dies with more than the 21 billion that I'm referring to here (the NVidia 5K cores chip). If that was all it took then we'd have had 'human brains' in a box long ago, after all, 50 GPUs is something that isn't all that rare in plenty of institutions.
- Veedrac 7y ago
  There are 100 trillion synapses in the brain. I'm not claiming Cerebras is human scale; I think it's a factor of 100-1000 off personally. I'm just saying I don't think the comparison is meaningless.
  - jacquesm 7y ago
    Time will tell. I personally put the time between now and AGI well in the decades, possibly more than a few centuries. This is not a mere matter of engineering, it is one of fundamental understanding of which we likely have achieved only very little. Imagine seeing the wiring diagram of an alien computer with parts on the order of 6 orders of magnitude smaller than the ones that you are familiar with without knowing the contents of its memory and then you are expected to make a working copy. Good luck.
azinman2 7y ago
We don’t know what numbers are the numbers that we need to care about. How much do chemical gradients matter? Inhibitors? Hormones? What other factors are at play? I read so many people that have both simplified views of DNNs as well as real neurons that they get overly excited about the possibility of somehow getting to AGI by just having enough transistors. There was similar thinking in the 1950s when proto-AIs could play chess. Cognition is extraordinary. The theory of mind encompasses many important features that we have no use how to approach. To say all that’s left is research doesn’t really say much at all. It’s like saying for teleportation all that’s left is research as we have lots of bandwidth, or that for month-long battery life all that’s left is research as we have the phones ready to take them. The gap between our knowledge Of the brain and what’s being applied to machine learning is a big leap.
- Veedrac 7y ago
  Brains run on computation. Computation is fungible. The highest bandwidth signal in the brain is through neuron spiking, and there isn't much space for other mechanisms to hide. Take a simple 10 virtual neurons:1 synapse ratio as a rough upper bound of how much computation the brain can be doing, and by necessity the rest is architecture. This is not at all like teleportation. It is absurd to claim numbers much larger than my own. 100:1? Where would the computation be? It is completely unnecessary to claim that the brain must have vast amounts of hidden capacity to do what it does, rather than the secret being in the ‘software’, and it goes against most of what we know about the brain, its ancestry, and computation. This is not like teleportation. The brain isn't magic. I don't have to claim to know how the brain works to point this out.
  - ohum 7y ago
    Please don’t take this as a personal attack. Your comment and certainty sound like religious fanaticism. The belief that consciousness is merely computation is unproven, and a matter of faith. It always seems a little short sighted when people claim to know something about consciousness and the mind based on a belief that the brain amounts to a biological computer. As an example of an alternative possibility to brain as a biological computer: in a holographic universe the brain itself is an emergent system. Or, Buddhists say that the material world as we perceive it is interdependent with mind, which is more fundamental than form. Some interesting phenomena that present problems with mind as programming running on a biological computer are the placebo/*cebo effects and Wim Hof :). Mind over matter, man.

jacquesm 7y ago

Using fine sand I make that out to be about 6 buckets, give or take, besides, this isn't about whether it is 6, 10 or even 100 (or 1000) buckets, it is about how the number itself does not make a huge difference. Before if you wanted 1.2T transistors you needed 50 or so regular large dies, and presumably there have already been dies with more than the 21 billion that I'm referring to here (the NVidia 5K cores chip). If that was all it took then we'd have had 'human brains' in a box long ago, after all, 50 GPUs is something that isn't all that rare in plenty of institutions.

Veedrac 7y ago
There are 100 trillion synapses in the brain. I'm not claiming Cerebras is human scale; I think it's a factor of 100-1000 off personally. I'm just saying I don't think the comparison is meaningless.
- jacquesm 7y ago
  Time will tell. I personally put the time between now and AGI well in the decades, possibly more than a few centuries. This is not a mere matter of engineering, it is one of fundamental understanding of which we likely have achieved only very little. Imagine seeing the wiring diagram of an alien computer with parts on the order of 6 orders of magnitude smaller than the ones that you are familiar with without knowing the contents of its memory and then you are expected to make a working copy. Good luck.

azinman2 7y ago

We don’t know what numbers are the numbers that we need to care about. How much do chemical gradients matter? Inhibitors? Hormones? What other factors are at play? I read so many people that have both simplified views of DNNs as well as real neurons that they get overly excited about the possibility of somehow getting to AGI by just having enough transistors. There was similar thinking in the 1950s when proto-AIs could play chess. Cognition is extraordinary. The theory of mind encompasses many important features that we have no use how to approach. To say all that’s left is research doesn’t really say much at all. It’s like saying for teleportation all that’s left is research as we have lots of bandwidth, or that for month-long battery life all that’s left is research as we have the phones ready to take them. The gap between our knowledge Of the brain and what’s being applied to machine learning is a big leap.

Veedrac 7y ago
Brains run on computation. Computation is fungible. The highest bandwidth signal in the brain is through neuron spiking, and there isn't much space for other mechanisms to hide. Take a simple 10 virtual neurons:1 synapse ratio as a rough upper bound of how much computation the brain can be doing, and by necessity the rest is architecture. This is not at all like teleportation. It is absurd to claim numbers much larger than my own. 100:1? Where would the computation be? It is completely unnecessary to claim that the brain must have vast amounts of hidden capacity to do what it does, rather than the secret being in the ‘software’, and it goes against most of what we know about the brain, its ancestry, and computation. This is not like teleportation. The brain isn't magic. I don't have to claim to know how the brain works to point this out.
- ohum 7y ago
  Please don’t take this as a personal attack. Your comment and certainty sound like religious fanaticism. The belief that consciousness is merely computation is unproven, and a matter of faith. It always seems a little short sighted when people claim to know something about consciousness and the mind based on a belief that the brain amounts to a biological computer. As an example of an alternative possibility to brain as a biological computer: in a holographic universe the brain itself is an emergent system. Or, Buddhists say that the material world as we perceive it is interdependent with mind, which is more fundamental than form. Some interesting phenomena that present problems with mind as programming running on a biological computer are the placebo/*cebo effects and Wim Hof :). Mind over matter, man.