4 ms·

one thing a friend was suggesting to me was that he thinks the difference is that human intelligence is analogue while computing intelligence is digital. there

one thing a friend was suggesting to me was that he thinks the difference is that human intelligence is analogue while computing intelligence is digital. there are nuances in our thinking and language that current digital systems cannot compete with until the become fine grained enough to appear analogue.

wegfawefgawefg 3y ago
Ive heard this before. Im not trying to be offensive here, but it just doesn't make any sense. It's like being concerned that planes can't fly because they don't flap their wings. If you want to escape this state of ignorance you're gonna have to implement some neural networks, and learn about neurology. At the very least consider that double precision floating points are extremely insanely precise. Definitely orders of magnitude more precise than real biological neurons can measure over the noise floor of activity in a brain.
- ravetcofx 3y ago
  The analogue/digital isn't the main distinction I think, it's a distinction of highly optimized higher order multi-modality. Our brains have Nth degree multi modality input and output that run on a few watts (calories) while nueral networks & LLMs so far take kilowatts or more. A bird can fly 10s-100s of km with flapping with a few watts, an airplane, megawatts for fixed wing flight of the same distance. Evolution was an unintentional search algorithm over hundreds of millions of years to find resilient, optimized, and efficient-ish systems. Our toys are so new on that search tree and are thus not the most complex, resilient or efficient.
  - wegfawefgawefg 3y ago
    I don't think that was his point. I think he just vaguely meant digital computers could never compute the same functions as analogue computers, particularly whatever the "alive function" is. To answer your point simply, machines don't have the same energy constraints as small animals. A bird brain gets more compute per watt than my desktop pc, but humanity can (and does. often) hook up a power plant to a building sized computer. If you believe the function a brain is computing is computeable on von neuman, efficiency isnt relevant, in so far as you build a big enough machine, and get it running the right program.
gkbrk 3y ago
Computers use binary logic and circuits, but with neural networks you can and pretty much always use floating point numbers. That's very analog. There are around 2^52 doubles between 0 and 1. Surely that's enough to represent any analog signal you might care about.
nicklecompte 3y ago
Turing completeness says this isn't true: any Turing-complete digital system can simulate any physical system, including a human brain. What Turing completeness doesn't address is efficiency - how much energy would it take an artificial neural network to simulate a human brain? If it's "approximately one nuclear reactor" then maybe this will work with some efficiency improvements. If it's "approximately one sun" then it won't work at all. There is a very good Quanta article about this[1] detailing researchers attempt to simulate a rat cortex neuron with an ANN: it took 700 artificial neurons to simulate a single rat neuron, and that was just simulating the "voltmeter" measurements of the synapses, ignoring any genetic effects. If you included genetic effects[2] I would guess it would take several thousand artificial neurons, if not more. And that's just a rat! Even at the level of individual neurons, primates seem to be more sophisticated than other mammals, with apes more sophisticated than other primates. So your friend is onto something: biological computers are astonishingly sophisticated compared to 21st-century transistor technology, and this probably has implications for our ability to emulate biological intelligence even on supercomputers. Think of a neural network that was trained to draw physically-sensible spiderweb patterns across a wide range of geometries. Now think about running this neural network in a computer the size of a spider. And that's ignoring the difficult real-world planning an actual spider would have to do to create a physical spiderweb. [1] https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... [2] AI is not even close to this level of computational sophistication: https://www.universityofcalifornia.edu/news/biologists-transfer-memory-between-snails https://www.universityofcalifornia.edu/news/biologists-trans...
- somewhereoutth 3y ago
  No, a Turing complete system can perfectly simulate any other discreet (digital) computation system. What it cannot do is perfectly simulate any analogue system. By Cantor's diagonal argument, there are vastly more real numbers than natural numbers. Any system that is completely definable in the natural numbers (like digital computers) will not be able to 'reach' the vast majority of real numbers (in case you are concerned about the existence of real numbers, what is the square root of 2? What is the ratio of diameter to circumference?). Throw in some Chaos Theory and you can see how far away any Turing complete system is from a biological brain.
somewhereoutth 3y ago
Your friend is correct. I call it the Cardinality Barrier - Cantor showed that there are incomprehensibly more real numbers than natural (whole) numbers. Any digital system can therefore be only an approximation. It is likely that the secret of intelligence lies beyond any approximation we might make in silicon. Of course AGI advocates won't like this idea, as it dooms their entire project from the off. It may be that the only realistic way to make a human like brain, is to make a human (which is quite easy actually).
layer8 3y ago
You’d have to explain why the analog couldn’t be sufficiently approximated by the digital. Slightly disturbing the brain by electromagnetic waves doesn’t cause it to cease to function, so it’s unclear why the deviation caused by numeric approximation would cause that.

wegfawefgawefg 3y ago

Ive heard this before. Im not trying to be offensive here, but it just doesn't make any sense. It's like being concerned that planes can't fly because they don't flap their wings. If you want to escape this state of ignorance you're gonna have to implement some neural networks, and learn about neurology. At the very least consider that double precision floating points are extremely insanely precise. Definitely orders of magnitude more precise than real biological neurons can measure over the noise floor of activity in a brain.

ravetcofx 3y ago
The analogue/digital isn't the main distinction I think, it's a distinction of highly optimized higher order multi-modality. Our brains have Nth degree multi modality input and output that run on a few watts (calories) while nueral networks & LLMs so far take kilowatts or more. A bird can fly 10s-100s of km with flapping with a few watts, an airplane, megawatts for fixed wing flight of the same distance. Evolution was an unintentional search algorithm over hundreds of millions of years to find resilient, optimized, and efficient-ish systems. Our toys are so new on that search tree and are thus not the most complex, resilient or efficient.
- wegfawefgawefg 3y ago
  I don't think that was his point. I think he just vaguely meant digital computers could never compute the same functions as analogue computers, particularly whatever the "alive function" is. To answer your point simply, machines don't have the same energy constraints as small animals. A bird brain gets more compute per watt than my desktop pc, but humanity can (and does. often) hook up a power plant to a building sized computer. If you believe the function a brain is computing is computeable on von neuman, efficiency isnt relevant, in so far as you build a big enough machine, and get it running the right program.

gkbrk 3y ago

Computers use binary logic and circuits, but with neural networks you can and pretty much always use floating point numbers. That's very analog. There are around 2^52 doubles between 0 and 1. Surely that's enough to represent any analog signal you might care about.

nicklecompte 3y ago

Turing completeness says this isn't true: any Turing-complete digital system can simulate any physical system, including a human brain. What Turing completeness doesn't address is efficiency - how much energy would it take an artificial neural network to simulate a human brain? If it's "approximately one nuclear reactor" then maybe this will work with some efficiency improvements. If it's "approximately one sun" then it won't work at all. There is a very good Quanta article about this[1] detailing researchers attempt to simulate a rat cortex neuron with an ANN: it took 700 artificial neurons to simulate a single rat neuron, and that was just simulating the "voltmeter" measurements of the synapses, ignoring any genetic effects. If you included genetic effects[2] I would guess it would take several thousand artificial neurons, if not more. And that's just a rat! Even at the level of individual neurons, primates seem to be more sophisticated than other mammals, with apes more sophisticated than other primates. So your friend is onto something: biological computers are astonishingly sophisticated compared to 21st-century transistor technology, and this probably has implications for our ability to emulate biological intelligence even on supercomputers. Think of a neural network that was trained to draw physically-sensible spiderweb patterns across a wide range of geometries. Now think about running this neural network in a computer the size of a spider. And that's ignoring the difficult real-world planning an actual spider would have to do to create a physical spiderweb. [1] https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... [2] AI is not even close to this level of computational sophistication: https://www.universityofcalifornia.edu/news/biologists-transfer-memory-between-snails https://www.universityofcalifornia.edu/news/biologists-trans...

somewhereoutth 3y ago
No, a Turing complete system can perfectly simulate any other discreet (digital) computation system. What it cannot do is perfectly simulate any analogue system. By Cantor's diagonal argument, there are vastly more real numbers than natural numbers. Any system that is completely definable in the natural numbers (like digital computers) will not be able to 'reach' the vast majority of real numbers (in case you are concerned about the existence of real numbers, what is the square root of 2? What is the ratio of diameter to circumference?). Throw in some Chaos Theory and you can see how far away any Turing complete system is from a biological brain.

somewhereoutth 3y ago

Your friend is correct. I call it the Cardinality Barrier - Cantor showed that there are incomprehensibly more real numbers than natural (whole) numbers. Any digital system can therefore be only an approximation. It is likely that the secret of intelligence lies beyond any approximation we might make in silicon. Of course AGI advocates won't like this idea, as it dooms their entire project from the off. It may be that the only realistic way to make a human like brain, is to make a human (which is quite easy actually).

layer8 3y ago

You’d have to explain why the analog couldn’t be sufficiently approximated by the digital. Slightly disturbing the brain by electromagnetic waves doesn’t cause it to cease to function, so it’s unclear why the deviation caused by numeric approximation would cause that.