5 ms·

When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surp

When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.

Teever 3y ago
Won't that mean that we just change the quest from higher and higher density of digital elements to higher and higher density of analog elements? Like people weren't trying to make computers out of bigger and bigger tubes before the transistor, they were trying to make them out of smaller and smaller ones.
bigyikes 3y ago
Computers are going to increase in size as they consume more and more power, requiring increasingly elaborate cooling systems. Like how the transistor made the big and hot vacuum tubes obsolete, maybe we’ll see some analog breakthrough do the same thing to transistors, at least for AI. I doubt there is a world where we use analog for general purpose computing, but it seems perfect for messy, probabilistic processes like thinking.
- enlyth 3y ago
  What's amazing is that the human brain does it all on the equivalent of like 20 watts of power. That's basically a couple of LED light bulbs.
  - passion__desire 3y ago
    Is there a comparison of power efficiency of human brain doing 50 digits multiplication vs a multiplier circuit doing it?
    - gamacodre 3y ago
      I think the problem here would be figuring out how much of the brain's power draw to attribute to the multiplication. A brain is more akin to a motherboard than a single CPU, with all kinds of I/O, internal regulation, and other ancillary stuff going on all the time.
      - passion__desire 3y ago
        Is the issue then we haven't discovered the magical algorithm run by our brain? If we discover it, then digital circuits will handsomely beat brain.
        gamacodre 3y ago
        We can surely build more efficient and capable hardware than our current evolved wetware, since all of the details of how to build it are generally externalized. If the chips had to fab themselves, it would be a different story. The software is a different story. Sure, the brain does all sorts of things that aren't necessary for $TASK, but we aren't necessarily going to be able to correctly identify which are which. Is your inner experience of your arm motion needed to fully parse the meaning in "raise a glass to toast the bride and groom", or respond meaningfully to someone who says that? Or perhaps it doesn't really matter - language is already a decent tool for bridging disjoint creature realities, maybe it'll stretch to synthetic consciousness too.
        passion__desire 3y ago
        All of computation is realised by very few arithmetic operations. Then test energy efficiency of wetware and hardware on those operations. Then any difference can be attributed to algorithms.
smallmancontrov 3y ago
What? Action potentials are extremely digital, in that small perturbations are suppressed so as to encode information in a higher level conceptual state (the "digit" / "bit" is the presence or absence of an AP) to obtain robustness and integration scale.
jzombie 3y ago
I may have misinterpreted this comment by thinking you meant that as we squeeze more and more transistors into a small amount of space, the resulting waveforms would start to resemble analog more so than digital.
knodi123 3y ago
> I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. I have some theories that this isn't necessary. 1.) Just because the brain is a general-purpose machine great at doing lots of things, doesn't mean it's great at each of those things. Like when two people are playing catch, and one of them sees the first fragments of a parabola and estimates where the ball is going to land- a computer can calculate that way more efficiently than a mind, despite the fact that both are quick enough to get the job done. 2.) While the brain is great at, say, putting names to faces... a good CV machine can do the job almost as well, and can annotate a video stream in real-time. Combining 1.) the fact that some problems are much simpler to solve with classical algorithms instead of neural networking, and 2.) that many brain tasks can be farmed out to a coprocessor/service, my hypothesis is that the number of neurons/resources required to do the "secret sauce" part of agi could be greatly reduced.
- Galaxeblaffer 3y ago
  I'm also in the camp that believes that we won't reach agi without significantly more compute. I think that consciousness is an emergent property, just like i think life itself is an emergent property, Both need a certain set elements/systems to work. The secret sauce may look simple when we recreate the right conditions, but it's not going to be something that makes it possible without the right hardware so to speak.
  - knodi123 3y ago
    But, take for instance, sight. Sighted people use a large percentage of their brain to process visual input. People born congenitally blind are no smarter or dumber for their brains not having to process that input- so clearly that's not the secret sauce. I'm not convinced consciousness is emergent, I don't really have an opinion on _that_- but I'm > 50% convinced that consciousness itself doesn't require a neural network as large as a human brain's.
bee_rider 3y ago
Analogue circuits can be a real pain in the butt, imagine if integer overflows destroyed the whole ALU, haha. Transistors are already much smaller than neurons. And of course the brain doesn’t have a clock. And neurons have more complex behavior than single transistors… The whole system is just very different. So, this doesn’t seem like a strategy to get past a boundary, it is more like a suggestion that we give up on the current path and go in a radically different direction. It… isn’t impossible but it seems like a wild change for the field. If we want something post-cmos, potentially radically more efficient, but still familiar in the sense that it produces digital logic, quantum dot cellular automata with Bennett clocking seems more promising IMO.
- philbin 3y ago
  [dead]
mwbajor 3y ago
I work in analog, 1) Noise is an issue as the system gets complex. You can't get away with counting to 1 anymore, all those levels in between matter. 2) Its hard to make an analog computer reconfigurable. 3) Analog computers exist commercially believe it or not, but for niche applications and essentially as coprocessors.
- jameshart 3y ago
  Quantization of parameters in neural networks is roughly analogous to introducing noise into analog signals. We’ve got good evidence that these architectures are robust to quantization - which implies they could be implemented over noisy analog signals. Not sure who’s working on that but I can’t believe it’s not being examined.
floxy 3y ago
>We need that ONE paper on analogue https://www.analog.com/en/resources/analog-dialogue/articles/considering-multipliers-part-1.html https://www.analog.com/en/resources/analog-dialogue/articles...
marcosdumay 3y ago
I doubt it. Digital just scales better. Our brain has a pretty bounded need of scaling, but once we create some computer equivalent, it would be very counterproductive to make it useless for larger problems for a small gain on smaller ones.
- smallmancontrov 3y ago
  > Digital just scales better. Yes! > Our brain has a pretty bounded need of scaling No! Over aeons our brains scaled from several neurons to 100 billion neurons, each with 1000 synapses. They were able to do it because our brains are digital. They lean on their digital nature even more than computer chips do. Action potentials are so digital it hurts. They aren't just quantized in level, but in the entire shape of the waveform across several milliseconds. Just as in computer chips, this suppresses perturbations. As long as higher level computation only depends on presence/absence of action potentials and timing, it inherits this robustness and allows scale. Rather than errors accumulating and preventing integration beyond a certain threshold, error resilience scales alongside computation. Every neuron "refreshes the signal," allowing arbitrary integration complexity at any scale, even in the face of messy biology problems along the way. Just like every transistor (or at least logic gate) "refreshes the signal" so that you can stack billions on a chip and quadrillions in sequential computation, even though each transistor is imperfect. Digital computation is the way. Always has been, always will be.

Teever 3y ago

Won't that mean that we just change the quest from higher and higher density of digital elements to higher and higher density of analog elements? Like people weren't trying to make computers out of bigger and bigger tubes before the transistor, they were trying to make them out of smaller and smaller ones.

bigyikes 3y ago

Computers are going to increase in size as they consume more and more power, requiring increasingly elaborate cooling systems. Like how the transistor made the big and hot vacuum tubes obsolete, maybe we’ll see some analog breakthrough do the same thing to transistors, at least for AI. I doubt there is a world where we use analog for general purpose computing, but it seems perfect for messy, probabilistic processes like thinking.

enlyth 3y ago
What's amazing is that the human brain does it all on the equivalent of like 20 watts of power. That's basically a couple of LED light bulbs.
- passion__desire 3y ago
  Is there a comparison of power efficiency of human brain doing 50 digits multiplication vs a multiplier circuit doing it?
  - gamacodre 3y ago
    I think the problem here would be figuring out how much of the brain's power draw to attribute to the multiplication. A brain is more akin to a motherboard than a single CPU, with all kinds of I/O, internal regulation, and other ancillary stuff going on all the time.
    - passion__desire 3y ago
      Is the issue then we haven't discovered the magical algorithm run by our brain? If we discover it, then digital circuits will handsomely beat brain.
      - gamacodre 3y ago
        We can surely build more efficient and capable hardware than our current evolved wetware, since all of the details of how to build it are generally externalized. If the chips had to fab themselves, it would be a different story. The software is a different story. Sure, the brain does all sorts of things that aren't necessary for $TASK, but we aren't necessarily going to be able to correctly identify which are which. Is your inner experience of your arm motion needed to fully parse the meaning in "raise a glass to toast the bride and groom", or respond meaningfully to someone who says that? Or perhaps it doesn't really matter - language is already a decent tool for bridging disjoint creature realities, maybe it'll stretch to synthetic consciousness too.
        passion__desire 3y ago
        All of computation is realised by very few arithmetic operations. Then test energy efficiency of wetware and hardware on those operations. Then any difference can be attributed to algorithms.

smallmancontrov 3y ago

What? Action potentials are extremely digital, in that small perturbations are suppressed so as to encode information in a higher level conceptual state (the "digit" / "bit" is the presence or absence of an AP) to obtain robustness and integration scale.

jzombie 3y ago

I may have misinterpreted this comment by thinking you meant that as we squeeze more and more transistors into a small amount of space, the resulting waveforms would start to resemble analog more so than digital.

knodi123 3y ago

> I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. I have some theories that this isn't necessary. 1.) Just because the brain is a general-purpose machine great at doing lots of things, doesn't mean it's great at each of those things. Like when two people are playing catch, and one of them sees the first fragments of a parabola and estimates where the ball is going to land- a computer can calculate that way more efficiently than a mind, despite the fact that both are quick enough to get the job done. 2.) While the brain is great at, say, putting names to faces... a good CV machine can do the job almost as well, and can annotate a video stream in real-time. Combining 1.) the fact that some problems are much simpler to solve with classical algorithms instead of neural networking, and 2.) that many brain tasks can be farmed out to a coprocessor/service, my hypothesis is that the number of neurons/resources required to do the "secret sauce" part of agi could be greatly reduced.

Galaxeblaffer 3y ago
I'm also in the camp that believes that we won't reach agi without significantly more compute. I think that consciousness is an emergent property, just like i think life itself is an emergent property, Both need a certain set elements/systems to work. The secret sauce may look simple when we recreate the right conditions, but it's not going to be something that makes it possible without the right hardware so to speak.
- knodi123 3y ago
  But, take for instance, sight. Sighted people use a large percentage of their brain to process visual input. People born congenitally blind are no smarter or dumber for their brains not having to process that input- so clearly that's not the secret sauce. I'm not convinced consciousness is emergent, I don't really have an opinion on _that_- but I'm > 50% convinced that consciousness itself doesn't require a neural network as large as a human brain's.

bee_rider 3y ago

Analogue circuits can be a real pain in the butt, imagine if integer overflows destroyed the whole ALU, haha. Transistors are already much smaller than neurons. And of course the brain doesn’t have a clock. And neurons have more complex behavior than single transistors… The whole system is just very different. So, this doesn’t seem like a strategy to get past a boundary, it is more like a suggestion that we give up on the current path and go in a radically different direction. It… isn’t impossible but it seems like a wild change for the field. If we want something post-cmos, potentially radically more efficient, but still familiar in the sense that it produces digital logic, quantum dot cellular automata with Bennett clocking seems more promising IMO.

philbin 3y ago
[dead]

mwbajor 3y ago

I work in analog, 1) Noise is an issue as the system gets complex. You can't get away with counting to 1 anymore, all those levels in between matter. 2) Its hard to make an analog computer reconfigurable. 3) Analog computers exist commercially believe it or not, but for niche applications and essentially as coprocessors.

jameshart 3y ago
Quantization of parameters in neural networks is roughly analogous to introducing noise into analog signals. We’ve got good evidence that these architectures are robust to quantization - which implies they could be implemented over noisy analog signals. Not sure who’s working on that but I can’t believe it’s not being examined.

floxy 3y ago

>We need that ONE paper on analogue https://www.analog.com/en/resources/analog-dialogue/articles/considering-multipliers-part-1.html https://www.analog.com/en/resources/analog-dialogue/articles...

marcosdumay 3y ago

I doubt it. Digital just scales better. Our brain has a pretty bounded need of scaling, but once we create some computer equivalent, it would be very counterproductive to make it useless for larger problems for a small gain on smaller ones.

smallmancontrov 3y ago
> Digital just scales better. Yes! > Our brain has a pretty bounded need of scaling No! Over aeons our brains scaled from several neurons to 100 billion neurons, each with 1000 synapses. They were able to do it because our brains are digital. They lean on their digital nature even more than computer chips do. Action potentials are so digital it hurts. They aren't just quantized in level, but in the entire shape of the waveform across several milliseconds. Just as in computer chips, this suppresses perturbations. As long as higher level computation only depends on presence/absence of action potentials and timing, it inherits this robustness and allows scale. Rather than errors accumulating and preventing integration beyond a certain threshold, error resilience scales alongside computation. Every neuron "refreshes the signal," allowing arbitrary integration complexity at any scale, even in the face of messy biology problems along the way. Just like every transistor (or at least logic gate) "refreshes the signal" so that you can stack billions on a chip and quadrillions in sequential computation, even though each transistor is imperfect. Digital computation is the way. Always has been, always will be.