5 ms·

> Digital computers deal with integers, binary sequences, deterministic logic, algorithms, and time that is idealized into discrete increments. Analog compute

> Digital computers deal with integers, binary sequences, deterministic logic, algorithms, and time that is idealized into discrete increments. Analog computers deal with real numbers, non-deterministic logic, and continuous functions, including time as it exists as a continuum in the real world. As a computer scientist/computational neuroscientist, I don't really buy the analogue vs. digital distinction. Basic information theory tells us that any noisy continuous system is equivalent to a discrete system of a certain resolution/bit-depth. As the author continues to write > [...] analog computers embrace noise; a real-world neural network needing a certain level of noise to work. A few bits are sufficient to describe the output of a real-world neuron. Action potential timing has jitter in the sub-millisecond range, which means that relatively coarse discrete time steps are sufficient for a simulation. Yes, our brains are extremely noisy systems, yet this is exactly the reason why they (in theory at least) can be simulated on a discrete/digital computer. In practice there is little to be gained from analogue computation, except for a (potentially) reduced energy consumption compared to a digital implementation of the system in question. But on a theoretical level, nothing changes.

whatshisface 8y ago
>But on a theoretical level, nothing changes. Although your overall point is correct, I disagree with this statement. It is in fact only theory that distinguishes between continuous systems and very high resolution discrete ones, because the time-evolution of an ideal computer is nowhere differentiable while the time-evolution of a physical system is everywhere differentiable. Indeed, it is not only "discrete" and "continuous" that are indistinguishable below a certain threshold. For any data, there are an infinite number of continuous theories that are all indistinguishable from each other, and also are indistinguishable from an infinite number of discrete theories, which are yet also indistinguishable from each other - all that it takes for this to happen is for us to agree that the data doesn't specify anything below a certain scale. Then, every theory that agrees on the large scale will match the data, leaving room for anything you can imagine at the bottom. So discrete vs. continuous isn't the core point.
- aurelian15 8y ago
  Yes, thank you for the clarification, my statement is ambiguous. I meant to say that the theory required to describe a computation on a practical analogue computer (that is noisy) is no different than the theory required to describe the same computation on a digital computer, because they essentially are both discrete systems. However, as you point out (at least as I understand your first paragraph), when we analyse/build systems on an abstract level we assume (often as a simplification) that they are ideal continuous systems.
logicchains 8y ago
>Basic information theory tells us that any noisy continuous system is equivalent to a discrete system of a certain resolution/bit-depth. Not only that, but the https://en.wikipedia.org/wiki/Bekenstein_bound https://en.wikipedia.org/wiki/Bekenstein_bound also tells us that in a finite region of space with finite energy, there's a fixed bound on its information content. So if the brain exists in finite region of space with finite energy, then it can be described without loss of accuracy by a discrete system.
- TheOtherHobbes 8y ago
  Only if you disconnect the brain from the rest of the universe around it. Which doesn't usually end well.
k9s9 8y ago
Well analog computation being noisey could be the reason we experience things like emotion. It's easy to say emotion has no value, until you see it in action bringing some sense of control to say a family that has gone through trauma or a country through war. It doesn't look like digital computation (not digital encoding) can produce such outcomes. We are constantly seeing, be it the NSA/Zuck/Wall St/China etc etc having access to ridiculous amounts of digital computational power, but being totally surprised on a daily basis by the realization they aren't in control.
- aurelian15 8y ago
  > Well analog computation being noisey could be the reason we experience things like emotion. Hm, I don't really see why this should be the case. Emotions are a pretty well studied in both animals and humans and are, to put it very handwavingly, merely a global change of brain state/equilibria, for example modulated by brain regions such as the Amygdala and/or the release of neuromodulators [1]. From my understanding, there is nothing about emotions that cannot be computed by a digital computer, and there is little about emotions that is related to noise. I'll let philosophers think about the experience part of your statement. [1] https://en.wikipedia.org/wiki/Amygdala https://en.wikipedia.org/wiki/Amygdala
  - JohnJamesRambo 8y ago
    Ok show me a computer with emotions.
    - deleted 8y ago
      [deleted]
    - deleted 8y ago
      [deleted]
    - etCeteraaa 8y ago
      You really wouldn’t want to see one. But all that’s needed are are a handful of rules, to provide for a system that emotes. You’d probably dismiss it as an inauthentic toy, but emotions actually aren’t the core aspect of agency. Anyway, the rules just need to assemble a goal, a threshold for equilibium, and reactions for deviation from that equilibrium. Bonus points if you account for radiant measurements of equilibrium. What I mean by that is anticipation of adjacent conditions that signal a probable loss of equilibrium, such that the system doesn’t just react to an unbalanced circumstance, but also things that could lead to an undesired imbalance. Examples: A. If the cup is disturbed so that the milk spills, then a negative experience ensues. B. If a balloon, inflated with ordinary compressed air, sinks onto the grass and pops, a negative experience ensues. C. Ambulate through an environment obstructed by complex obstacles, and negotiate each obstacle without falling onto the ground. Falling onto the ground will result in a negative experience. Each of these three goals represents a targeted state of equilibrium: don’t spill the milk, keep the balloon safe, don’t fall down go boom. Now, layer an array of reactions on top of the branched set of possible outcomes. You can also buld up variations on top of each branch. Positive branches are indicated in moments of success at achieving the goal. Negative branches are indicated upon equilibrium being defeated. So the computer or robot can externalize its inner state with a happy face or a sad face, but we’re missing some of the emotional range. When would anger display? When the machine can assign blame and consider revenge, of course. So if an entity (preferably a rival robot, since we wouldn’t want the robot to exact revenge on a person) knocks over the milk, pops the balloon, tackles the robot, the obvious motive is to make sure that never happens again, the root cause is the rival entity. Stand back up, destroy the entity, and acquire more milk, another balloon, and try to achieve equilibrium, and thus happiness again. Prior to reacquiring its happy state, the machine can externalize an angry face if it can assign blame to a detected responsible entity, in all other cases, it would simply be sad, until it can stand back up, inflate another balloon and pour itself another glass of milk to protect. If it cannot set things back in order, as desired, then it is simply permanently sad (no balloon, no milk, unable to stand or walk), forever. See how that works? It’s actually not much more complicated than that.
rv-de 8y ago
> Basic information theory tells us that any noisy continuous system is equivalent to a discrete system of a certain resolution/bit-depth. Technically you are right - but practically you are wrong. Like 'technically' any logical or even emotional reasoning can be modeled with if-else-structures (maybe add some randomness). But why aren't we able yet to actually create human-like reasoning? Because that approach is 'practically' not useful. That's why the most powerful ML solutions aren't realized with Prolog, but with neural networks at the moment. > A few bits are sufficient to describe the output of a real-world neuron. Possibly. But no computer is so far able to even remotely simulate or emulate what is actually going on within a real-world neuron. And that is required to fundamentally understand the output. > except for a (potentially) reduced energy consumption And that is quite a big deal. Because the effect of reduced energy in a parallelized system is going to be exponentially relevant! > But on a theoretical level, nothing changes. But on a practical level everything will change.

whatshisface 8y ago

>But on a theoretical level, nothing changes. Although your overall point is correct, I disagree with this statement. It is in fact only theory that distinguishes between continuous systems and very high resolution discrete ones, because the time-evolution of an ideal computer is nowhere differentiable while the time-evolution of a physical system is everywhere differentiable. Indeed, it is not only "discrete" and "continuous" that are indistinguishable below a certain threshold. For any data, there are an infinite number of continuous theories that are all indistinguishable from each other, and also are indistinguishable from an infinite number of discrete theories, which are yet also indistinguishable from each other - all that it takes for this to happen is for us to agree that the data doesn't specify anything below a certain scale. Then, every theory that agrees on the large scale will match the data, leaving room for anything you can imagine at the bottom. So discrete vs. continuous isn't the core point.

aurelian15 8y ago
Yes, thank you for the clarification, my statement is ambiguous. I meant to say that the theory required to describe a computation on a practical analogue computer (that is noisy) is no different than the theory required to describe the same computation on a digital computer, because they essentially are both discrete systems. However, as you point out (at least as I understand your first paragraph), when we analyse/build systems on an abstract level we assume (often as a simplification) that they are ideal continuous systems.

logicchains 8y ago

>Basic information theory tells us that any noisy continuous system is equivalent to a discrete system of a certain resolution/bit-depth. Not only that, but the https://en.wikipedia.org/wiki/Bekenstein_bound https://en.wikipedia.org/wiki/Bekenstein_bound also tells us that in a finite region of space with finite energy, there's a fixed bound on its information content. So if the brain exists in finite region of space with finite energy, then it can be described without loss of accuracy by a discrete system.

TheOtherHobbes 8y ago
Only if you disconnect the brain from the rest of the universe around it. Which doesn't usually end well.

k9s9 8y ago

Well analog computation being noisey could be the reason we experience things like emotion. It's easy to say emotion has no value, until you see it in action bringing some sense of control to say a family that has gone through trauma or a country through war. It doesn't look like digital computation (not digital encoding) can produce such outcomes. We are constantly seeing, be it the NSA/Zuck/Wall St/China etc etc having access to ridiculous amounts of digital computational power, but being totally surprised on a daily basis by the realization they aren't in control.

aurelian15 8y ago
> Well analog computation being noisey could be the reason we experience things like emotion. Hm, I don't really see why this should be the case. Emotions are a pretty well studied in both animals and humans and are, to put it very handwavingly, merely a global change of brain state/equilibria, for example modulated by brain regions such as the Amygdala and/or the release of neuromodulators [1]. From my understanding, there is nothing about emotions that cannot be computed by a digital computer, and there is little about emotions that is related to noise. I'll let philosophers think about the experience part of your statement. [1] https://en.wikipedia.org/wiki/Amygdala https://en.wikipedia.org/wiki/Amygdala
- JohnJamesRambo 8y ago
  Ok show me a computer with emotions.
  - deleted 8y ago
    [deleted]
  - deleted 8y ago
    [deleted]
  - etCeteraaa 8y ago
    You really wouldn’t want to see one. But all that’s needed are are a handful of rules, to provide for a system that emotes. You’d probably dismiss it as an inauthentic toy, but emotions actually aren’t the core aspect of agency. Anyway, the rules just need to assemble a goal, a threshold for equilibium, and reactions for deviation from that equilibrium. Bonus points if you account for radiant measurements of equilibrium. What I mean by that is anticipation of adjacent conditions that signal a probable loss of equilibrium, such that the system doesn’t just react to an unbalanced circumstance, but also things that could lead to an undesired imbalance. Examples: A. If the cup is disturbed so that the milk spills, then a negative experience ensues. B. If a balloon, inflated with ordinary compressed air, sinks onto the grass and pops, a negative experience ensues. C. Ambulate through an environment obstructed by complex obstacles, and negotiate each obstacle without falling onto the ground. Falling onto the ground will result in a negative experience. Each of these three goals represents a targeted state of equilibrium: don’t spill the milk, keep the balloon safe, don’t fall down go boom. Now, layer an array of reactions on top of the branched set of possible outcomes. You can also buld up variations on top of each branch. Positive branches are indicated in moments of success at achieving the goal. Negative branches are indicated upon equilibrium being defeated. So the computer or robot can externalize its inner state with a happy face or a sad face, but we’re missing some of the emotional range. When would anger display? When the machine can assign blame and consider revenge, of course. So if an entity (preferably a rival robot, since we wouldn’t want the robot to exact revenge on a person) knocks over the milk, pops the balloon, tackles the robot, the obvious motive is to make sure that never happens again, the root cause is the rival entity. Stand back up, destroy the entity, and acquire more milk, another balloon, and try to achieve equilibrium, and thus happiness again. Prior to reacquiring its happy state, the machine can externalize an angry face if it can assign blame to a detected responsible entity, in all other cases, it would simply be sad, until it can stand back up, inflate another balloon and pour itself another glass of milk to protect. If it cannot set things back in order, as desired, then it is simply permanently sad (no balloon, no milk, unable to stand or walk), forever. See how that works? It’s actually not much more complicated than that.

rv-de 8y ago

> Basic information theory tells us that any noisy continuous system is equivalent to a discrete system of a certain resolution/bit-depth. Technically you are right - but practically you are wrong. Like 'technically' any logical or even emotional reasoning can be modeled with if-else-structures (maybe add some randomness). But why aren't we able yet to actually create human-like reasoning? Because that approach is 'practically' not useful. That's why the most powerful ML solutions aren't realized with Prolog, but with neural networks at the moment. > A few bits are sufficient to describe the output of a real-world neuron. Possibly. But no computer is so far able to even remotely simulate or emulate what is actually going on within a real-world neuron. And that is required to fundamentally understand the output. > except for a (potentially) reduced energy consumption And that is quite a big deal. Because the effect of reduced energy in a parallelized system is going to be exponentially relevant! > But on a theoretical level, nothing changes. But on a practical level everything will change.