4 ms·

Even human brains, with all their capabilities, get the job done for 20 watts (or something in that order of magnitude.) Who even knows how much power we'd nee

Even human brains, with all their capabilities, get the job done for 20 watts (or something in that order of magnitude.) Who even knows how much power we'd need for equivalent results from the best specialized silicon we have? I think you might burn a hundred gigawatts and still not get comparable results. Something is clearly wrong with our approach. It should be possible to do much more with much less, but there is some piece of the puzzle we're missing.

osmarks 5y ago
The brain has hyperspecialized and hyperoptimized single-purpose hardware. Human AI systems run on more general-purpose machines with a few decades spent improving the hardware. Blaming the software is silly.
- wearywanderer 5y ago
  I'm not talking about running general purpose CPUs. I'm talking about the most high end specialized silicon we have. Throw the best TPUs ever made at the problem and you still won't get anywhere even remotely close to what the squishy pink meat can do. The power gap is so huge, our software simply must be inadequate. We're talking about a power gap wider than the gap between my toaster oven and the Saturn V.
  - osmarks 5y ago
    I rechecked some random internet articles, and the brain contains about 10^15 synapses. They operate at about 10Hz (~10 PFLOPS). If we arbitrarily assume that each synapse operation is equivalent to 1 floating point operation on a computer, then this is about 3 orders of magnitude more than a good GPU (~10 TFLOPS). Which is actually a lot less than I thought. Still, the brain's basic structures have been optimized over ridiculous amounts of time, the training data those get while you exist is years of high-resolution images/sound/etc, and being a smaller in-skull device it has advantages of lower latency than, say, a cluster used to train ML models.
    - jonplackett 5y ago
      I think the link between 1 floating point operation and 1 synapse is probably a bit dubious.

osmarks 5y ago

The brain has hyperspecialized and hyperoptimized single-purpose hardware. Human AI systems run on more general-purpose machines with a few decades spent improving the hardware. Blaming the software is silly.

wearywanderer 5y ago
I'm not talking about running general purpose CPUs. I'm talking about the most high end specialized silicon we have. Throw the best TPUs ever made at the problem and you still won't get anywhere even remotely close to what the squishy pink meat can do. The power gap is so huge, our software simply must be inadequate. We're talking about a power gap wider than the gap between my toaster oven and the Saturn V.
- osmarks 5y ago
  I rechecked some random internet articles, and the brain contains about 10^15 synapses. They operate at about 10Hz (~10 PFLOPS). If we arbitrarily assume that each synapse operation is equivalent to 1 floating point operation on a computer, then this is about 3 orders of magnitude more than a good GPU (~10 TFLOPS). Which is actually a lot less than I thought. Still, the brain's basic structures have been optimized over ridiculous amounts of time, the training data those get while you exist is years of high-resolution images/sound/etc, and being a smaller in-skull device it has advantages of lower latency than, say, a cluster used to train ML models.
  - jonplackett 5y ago
    I think the link between 1 floating point operation and 1 synapse is probably a bit dubious.