4 ms·

Great so biological neurons can perform feats of intuition, that simulated neurons simply can't. Even theoretically. Gotcha. Thank you very much for the explana

Great so biological neurons can perform feats of intuition, that simulated neurons simply can't. Even theoretically. Gotcha. Thank you very much for the explanation.

famouswaffles 3y ago
Yeah that's not the case lol. ann's can model any function even non linear ones.
- nyrikki 3y ago
  Pairwise linear in respect to inputs, for feed forward networks like those that use attention is the restriction related to linear or linearizable. Those words are important. But we know that ML cannot model all turning machines in the limit. As to more reasonable models that gets more complex. Rice's theorem in respect to functions doesn't fit with you claim: For any non-trivial property of partial functions, no general and effective method can decide whether an algorithm computes a partial function with that property. The generalization problem being the important part. But if you want to claim that ANNs solve all of the undecidable problems too, write the paper and become famous.
  - deleted 3y ago
    [deleted]
  - famouswaffles 3y ago
    >But we know that ML cannot model all turning machines in the limit. No computer is truly turing without infinite memory, including humans. >Rice's theorem in respect to functions doesn't fit with you claim: For any non-trivial property of partial functions, no general and effective method can decide whether an algorithm computes a partial function with that property. The generalization problem being the important part. This is...irrelevant. The theorem makes the claim for complete generalization (which humans do not demonstrate) as you say. It doesn't actually matter if the algorithm can't make the claim of property for all processes. Just like it doesn't matter that computers or humans are not really turing complete. We don't need a computer that is exactly a human. We need a computer that works.
    - ImHereToVote 3y ago
      I dunno. The guy that's freebasing copium sounds more convincing. Sorry.
      - nyrikki 3y ago
        Well as all you have as a response is an ad hominem you aren't the intended audience anyway. I would love to learn why I am wrong.
        ImHereToVote 3y ago
        You are clearly a smart guy, you are also clearly on the far side of the spectrum. Since it took you a while to notice that I was yanking your chain. That's ok, I'm somewhat of a full spectrum spaz myself. The problem with smart people is that they have a very hard time to notice when their brain has sidestepped rational thought, and started to go into an emotional latent space. The reason for this is that your emotional life has no problem utilizing complex topics to shield itself from your conscious attention. Face it. There isn't anything special about what a bio-neuron does. We just choose to ignore a lot of the intricacies of the bio-neuron. Because it is clearly irrelevant in order to sidestep halting. If a task suffers from halts the NN will simply side step the problem. Just like us. We are at a stage of technological development. Where we need to effen stop, and put all our effort into alignment. You realize this is true inside that cholesterol bag you call a brain. Your brain is just doing gymnastics of a preeteen soviet girl level, to stop yourself from getting it. Just get it.
        nyrikki 3y ago
        The halting problem is part of asymptotic analysis on this case. Just like with elementary limits you never reach it but you approach it. The halting problem on what we typically call computers decidable but not in practical time lines. The fact that a TM is not physically realizable doesn't change that claim. Logical conjunctions are an example of something that is difficult in PAC learning and we know it is at least super polynomial but we don't if there are tractable forms like Schaefer's dicotomy allowing for linear time solutions for HORNSat as an example. The tooling that works for asymptotic analysis on deterministic Turing machines does not transfer to biological nurons, because they simply aren't deterministic Turing machines. Neurobiologists are fully aware of the limitations of modeling cortical neurons as deterministic systems. While in pop science that difference may not be popular it is the general consensus of experts. Your claims that cortical neurons are the same as a deterministic Turing machine is not the best accepted theory today. In fact recent research says that qbits are a closer model to cortical neurons. You can't blindly carry over the properties of deterministic Turing machines to qbits. As non-deterministic Turing machine s are typically assigned to the special case of the type of NTM that defines the complexity class NP, I won't complicate the conversation with trying to explain the implications.
nyrikki 3y ago
Not quite that simple, or well known. We know that the class of total Turing computable functions is not learnable in the limit. But as to what else is AI-complete is mostly open questions. But there is a difference. Obviously ML is far better than humans in some domains so it is not a simple dichotomy.
- ImHereToVote 3y ago
  So what is it about a biological neuron that sidesteps the halting problem? Is it because there is a weight and bias set to it. Someone should tell OpenAI to add weights and biases to their LLMs. That will make them make have an emotional like intuition, will it not? EDIT: Maybe if we train the ANNs to focus their training on attention based techniques. Then they will simply tire on halts and continue on other problems until the model has sufficiently grokked lesser problems to focus on the previous halts.
  - wnoise 3y ago
    Why do you think "biological neuron"s sidestep the halting problem? People, who happen to run on biological neurons, have a sense of boredom that tries other approaches, and is also willing to eventually "give up", which aren't well captured in the standard algorithmic approaches.
    - ImHereToVote 3y ago
      Isn't that exactly how attention based neural networks work?
      - nyrikki 3y ago
        Attention is probably most easily conceptualized as run time reweighting. A powerful tool but it doesn't really change the underlying model it is just modifying the weights at runtime.
    - nyrikki 3y ago
      The halting problem simply doesn't apply to biological neurons, they don't side step them, unless your context is humans writing down an Algorithm. The rules of a human writing down an algorithm is the same thing as a Turing machine running an algorithm. The halting problem applies for any system of computation that is at least as powerful as a TM, including any type of arithmetic or non-arithmetic calculation that is well-defined, AKA deterministic. Cortical neuron firing is non-deterministic and more closely is modeled as probabilistic but still stochastic. https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1 https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1 Machine learning is constrained by the halting problem. HALT is the conical example for what is decidable, but other problems exist and sometimes PAC learnability hits practical limits far before the finite time limits of RE. As an example not invoking HALT: https://arxiv.org/abs/2208.10255 https://arxiv.org/abs/2208.10255 There are absolutely constraints on BNNs, but as BNNs aren't deterministic Turing machines, it doesn't apply. The real question is why do people resort to elementary oversimplified models of biological brains? If you are in the field of studying the brain you will look for deterministic models that fit your needs to make computation more likely to be tractable. But the false equivalency of ANNs to BNNs is problematic as a distraction from finding tractable solutions for computation.

Great so biological neurons can perform feats of intuition, that simulated neurons simply can't. Even theoretically. Gotcha. Thank you very much for the explanation.

famouswaffles 3y ago

Yeah that's not the case lol. ann's can model any function even non linear ones.

nyrikki 3y ago
Pairwise linear in respect to inputs, for feed forward networks like those that use attention is the restriction related to linear or linearizable. Those words are important. But we know that ML cannot model all turning machines in the limit. As to more reasonable models that gets more complex. Rice's theorem in respect to functions doesn't fit with you claim: For any non-trivial property of partial functions, no general and effective method can decide whether an algorithm computes a partial function with that property. The generalization problem being the important part. But if you want to claim that ANNs solve all of the undecidable problems too, write the paper and become famous.
- deleted 3y ago
  [deleted]
- famouswaffles 3y ago
  >But we know that ML cannot model all turning machines in the limit. No computer is truly turing without infinite memory, including humans. >Rice's theorem in respect to functions doesn't fit with you claim: For any non-trivial property of partial functions, no general and effective method can decide whether an algorithm computes a partial function with that property. The generalization problem being the important part. This is...irrelevant. The theorem makes the claim for complete generalization (which humans do not demonstrate) as you say. It doesn't actually matter if the algorithm can't make the claim of property for all processes. Just like it doesn't matter that computers or humans are not really turing complete. We don't need a computer that is exactly a human. We need a computer that works.
  - ImHereToVote 3y ago
    I dunno. The guy that's freebasing copium sounds more convincing. Sorry.
    - nyrikki 3y ago
      Well as all you have as a response is an ad hominem you aren't the intended audience anyway. I would love to learn why I am wrong.
      - ImHereToVote 3y ago
        You are clearly a smart guy, you are also clearly on the far side of the spectrum. Since it took you a while to notice that I was yanking your chain. That's ok, I'm somewhat of a full spectrum spaz myself. The problem with smart people is that they have a very hard time to notice when their brain has sidestepped rational thought, and started to go into an emotional latent space. The reason for this is that your emotional life has no problem utilizing complex topics to shield itself from your conscious attention. Face it. There isn't anything special about what a bio-neuron does. We just choose to ignore a lot of the intricacies of the bio-neuron. Because it is clearly irrelevant in order to sidestep halting. If a task suffers from halts the NN will simply side step the problem. Just like us. We are at a stage of technological development. Where we need to effen stop, and put all our effort into alignment. You realize this is true inside that cholesterol bag you call a brain. Your brain is just doing gymnastics of a preeteen soviet girl level, to stop yourself from getting it. Just get it.
        nyrikki 3y ago
        The halting problem is part of asymptotic analysis on this case. Just like with elementary limits you never reach it but you approach it. The halting problem on what we typically call computers decidable but not in practical time lines. The fact that a TM is not physically realizable doesn't change that claim. Logical conjunctions are an example of something that is difficult in PAC learning and we know it is at least super polynomial but we don't if there are tractable forms like Schaefer's dicotomy allowing for linear time solutions for HORNSat as an example. The tooling that works for asymptotic analysis on deterministic Turing machines does not transfer to biological nurons, because they simply aren't deterministic Turing machines. Neurobiologists are fully aware of the limitations of modeling cortical neurons as deterministic systems. While in pop science that difference may not be popular it is the general consensus of experts. Your claims that cortical neurons are the same as a deterministic Turing machine is not the best accepted theory today. In fact recent research says that qbits are a closer model to cortical neurons. You can't blindly carry over the properties of deterministic Turing machines to qbits. As non-deterministic Turing machine s are typically assigned to the special case of the type of NTM that defines the complexity class NP, I won't complicate the conversation with trying to explain the implications.

nyrikki 3y ago

Not quite that simple, or well known. We know that the class of total Turing computable functions is not learnable in the limit. But as to what else is AI-complete is mostly open questions. But there is a difference. Obviously ML is far better than humans in some domains so it is not a simple dichotomy.

ImHereToVote 3y ago
So what is it about a biological neuron that sidesteps the halting problem? Is it because there is a weight and bias set to it. Someone should tell OpenAI to add weights and biases to their LLMs. That will make them make have an emotional like intuition, will it not? EDIT: Maybe if we train the ANNs to focus their training on attention based techniques. Then they will simply tire on halts and continue on other problems until the model has sufficiently grokked lesser problems to focus on the previous halts.
- wnoise 3y ago
  Why do you think "biological neuron"s sidestep the halting problem? People, who happen to run on biological neurons, have a sense of boredom that tries other approaches, and is also willing to eventually "give up", which aren't well captured in the standard algorithmic approaches.
  - ImHereToVote 3y ago
    Isn't that exactly how attention based neural networks work?
    - nyrikki 3y ago
      Attention is probably most easily conceptualized as run time reweighting. A powerful tool but it doesn't really change the underlying model it is just modifying the weights at runtime.
  - nyrikki 3y ago
    The halting problem simply doesn't apply to biological neurons, they don't side step them, unless your context is humans writing down an Algorithm. The rules of a human writing down an algorithm is the same thing as a Turing machine running an algorithm. The halting problem applies for any system of computation that is at least as powerful as a TM, including any type of arithmetic or non-arithmetic calculation that is well-defined, AKA deterministic. Cortical neuron firing is non-deterministic and more closely is modeled as probabilistic but still stochastic. https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1 https://www.biorxiv.org/content/10.1101/2022.12.03.518978v1 Machine learning is constrained by the halting problem. HALT is the conical example for what is decidable, but other problems exist and sometimes PAC learnability hits practical limits far before the finite time limits of RE. As an example not invoking HALT: https://arxiv.org/abs/2208.10255 https://arxiv.org/abs/2208.10255 There are absolutely constraints on BNNs, but as BNNs aren't deterministic Turing machines, it doesn't apply. The real question is why do people resort to elementary oversimplified models of biological brains? If you are in the field of studying the brain you will look for deterministic models that fit your needs to make computation more likely to be tractable. But the false equivalency of ANNs to BNNs is problematic as a distraction from finding tractable solutions for computation.