4 ms·

Why do you think "biological neuron"s sidestep the halting problem? People, who happen to run on biological neurons, have a sense of boredom that tries other a

by wnoise 3y ago

Why do you think "biological neuron"s sidestep the halting problem? People, who happen to run on biological neurons, have a sense of boredom that tries other approaches, and is also willing to eventually "give up", which aren't well captured in the standard algorithmic approaches.

ImHereToVote 3y ago
Isn't that exactly how attention based neural networks work?
- nyrikki 3y ago
  Attention is probably most easily conceptualized as run time reweighting. A powerful tool but it doesn't really change the underlying model it is just modifying the weights at runtime.
nyrikki 3y ago
The halting problem simply doesn't apply to biological neurons, they don't side step them, unless your context is humans writing down an Algorithm. The rules of a human writing down an algorithm is the same thing as a Turing machine running an algorithm. The halting problem applies for any system of computation that is at least as powerful as a TM, including any type of arithmetic or non-arithmetic calculation that is well-defined, AKA deterministic. Cortical neuron firing is non-deterministic and more closely is modeled as probabilistic but still stochastic. https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1 https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1 Machine learning is constrained by the halting problem. HALT is the conical example for what is decidable, but other problems exist and sometimes PAC learnability hits practical limits far before the finite time limits of RE. As an example not invoking HALT: https://arxiv.org/abs/2208.10255 https://arxiv.org/abs/2208.10255 There are absolutely constraints on BNNs, but as BNNs aren't deterministic Turing machines, it doesn't apply. The real question is why do people resort to elementary oversimplified models of biological brains? If you are in the field of studying the brain you will look for deterministic models that fit your needs to make computation more likely to be tractable. But the false equivalency of ANNs to BNNs is problematic as a distraction from finding tractable solutions for computation.

by wnoise 3y ago

ImHereToVote 3y ago

Isn't that exactly how attention based neural networks work?

nyrikki 3y ago
Attention is probably most easily conceptualized as run time reweighting. A powerful tool but it doesn't really change the underlying model it is just modifying the weights at runtime.

nyrikki 3y ago

The halting problem simply doesn't apply to biological neurons, they don't side step them, unless your context is humans writing down an Algorithm. The rules of a human writing down an algorithm is the same thing as a Turing machine running an algorithm. The halting problem applies for any system of computation that is at least as powerful as a TM, including any type of arithmetic or non-arithmetic calculation that is well-defined, AKA deterministic. Cortical neuron firing is non-deterministic and more closely is modeled as probabilistic but still stochastic. https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1 https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1 Machine learning is constrained by the halting problem. HALT is the conical example for what is decidable, but other problems exist and sometimes PAC learnability hits practical limits far before the finite time limits of RE. As an example not invoking HALT: https://arxiv.org/abs/2208.10255 https://arxiv.org/abs/2208.10255 There are absolutely constraints on BNNs, but as BNNs aren't deterministic Turing machines, it doesn't apply. The real question is why do people resort to elementary oversimplified models of biological brains? If you are in the field of studying the brain you will look for deterministic models that fit your needs to make computation more likely to be tractable. But the false equivalency of ANNs to BNNs is problematic as a distraction from finding tractable solutions for computation.