5 ms·

There must be something "turing complete-ish" about neural nets. As in although the primitive element of a human neuron is complex, you can achieve identical co

There must be something "turing complete-ish" about neural nets. As in although the primitive element of a human neuron is complex, you can achieve identical computational power with a much simpler model.

skohan 4y ago
Wait are you saying you can achieve identical computational power to a biological neural network with an ANN? How do you justify that assumption?
- deltaonefour 4y ago
  No. I'm not. I'm saying there must be simple model primitives that don't involve electromagnetic fields and other strange communication channels that arise out of evolution that can be used to model intelligence. We aren't fully clear about what this model of intelligence is, but I'm instinctively sure that it won't have to involve anything as complex as biological neurons that communicate through thousands of different pathways and electromagnetic side effects too.
  - levitatorius 4y ago
    It's fascinating that we have simple primitives or notions of analysis, deduction, causation, yet no artificial system where those features of intelligence emerge on their own.
    - deltaonefour 4y ago
      Not true. It's called unsupervised learning.
      - levitatorius 4y ago
        What do you think is missing then? Unsupervised learning == general intelligence?
        deltaonefour 4y ago
        Not talking about agi. I am doing proof by contradiction. Yes current models of ML are primitive but the reasoning attributes you brought up have been reproduced in ML.. albeit in a primitive way. It is still proof by contradiction, what you say is categorically not true.
  - skohan 4y ago
    But it's a massive leap of faith to assume that the massive and varied array of information processing modalities in the brain are mostly unnecessary, or even that the "important" part of how the brain processes information can be modeled using classical computing at all in a way that's more efficient than biological neurons. That's just a feeling you have.
    - deltaonefour 4y ago
      It's not a massive leap of faith at all. Two reasons: 1. We know how natural selection works, and it clearly often produces unnecessary parts for many things. 2. There is physical evidence of evolution creating unnecessary and inefficient parts and components. 1. Natural selection works by randomly creating a fixed set of features both bad and good and then selecting the best one. Thus the features are bounded by randomness. If the random mutation doesn't create the most efficient part then The only thing that can be selected is an inefficient part. This follows that it is VERY possible that many processing modalities of the brain CAN be unnecessary. It is NOT a massive leap of faith. Additionally the selection criteria is simply survival. Not efficiency. As long as the "processing modalities" in the brain aid in survival the selection process does not interfere. Thus all kinds of arbitrary processing modalities both efficient and inefficient can occur so long as the modalities do not detriment survival. Why does stupidity evolve? Well stupid people aren't smart enough to create nuclear bombs to kill themselves. Thus stupidity is a possible trait, similar to how inefficient and unnecessary "processing modalities" are possible, mayhap being stupider aided in survival or didn't contribute anything at all. Just looking at people responding to me on HN, alot of them are pretty stupid (physical evidence). Not pointing out which one, certainly not you. 2. There is (alot) of physical evidence of the above. For example, the wheel is a very efficient way of traveling yet no animal has evolved some form of a wheel. The reasoning is actually more complicated about why a wheel hasn't evolved but the general idea I described above still applies... both natural selection and random mutation were not able to generate the series of intermediate traits required to form wheels. Thus from the reasoning above it is very possible for the brain to evolve with unnecessary and inefficient "processing modalities". Yes this idea did begin as a gut feeling. Subconscious processing is often times correct and logical. It is unwise to constantly mistrust it. It is always wise to analyze these "feelings" in attempt to break down the logic behind why it occurred. What my self analysis tells me is this: Given the fact that MANY of our mathematical models for physical processes involve elegant and straightforward primitives it follows that intelligence will also VERY likely be such a model as well and very likely much more simplified then the human brain.
cs137 4y ago
Neural nets can, provably, approximate any function, modulo certain conditions about continuity (i.e., can't be discontinuous in infinitely many places, that sort of thing). Universality is trivial to prove with boolean functions (because a neural net can do NAND) but also it's also mostly true (again, subject to certain analytical demands) on R^n. The problem is that it can require an exponential amount of time as well as data to actually get a decent network--and, of course, there's a high risk of it being overfit to idiosyncracies in the data. The standard approach to neural network training is to convert it into a gigantic calculus problem and brute-force it, and this doesn't always result in timely convergence or even a generalizable model.
- skohan 4y ago
  But is there any evidence that the totality of what the human brain does can be modeled as a mathematical function?
  - cs137 4y ago
    Whether we are more than computers is almost a religious question (I believe yes, but I can't prove it). However, any computation that we can prove through finite language, in the sense of absolute mathematical proof, can be checked by a mechanical process or, equivalently, a Turing machine (computer). If there's something more, we don't have the tools to prove it.
    - deltaonefour 4y ago
      >Whether we are more than computers is almost a religious question (I believe yes, but I can't prove it). This is actually not true and can logically be shown. We can model atoms with computation. Thus if all matter including the brain is made up of atoms then it can be simulated via a math equation.
      - cs137 4y ago
        We can't perfectly model a physical system (a quantum system, at some level) with classical computing. We'd need a quantum computer, and we still don't know what are the limitations on our ability to build one (the quantum computers currently being built are not at nearly the level we'd need for this sort of analysis). The idea that we understand what physical matter really is, how it works, whether it can produce qualia, et al, is simply off the mark. Of course most physical systems can be reasoned about macroscopically and are relatively deterministic--we can understand how computers work because the functionality is almost entirely classical--and we know that quantum mechanics is both mathematically and experimentally valid, but we not at all gotten to the bottom of this. In fact, one can argue that physics as we understand it can't be used to explain qualia. Physics requires symmetries (translational, rotational, spacetime) and asserts that there is no "privileged" reference frame that defines, say, the "objective" origin or x-, y-, z-axes. If you have two pure kilogram spheres of iron--of course, nothing macroscopic is actually 100.00% pure in practice, but it is physically conceivable and possible--they are exactly the same. One is a perfect substitute for (or copy of) the other. Consciousness likely violates this. We believe (although, without proof) that a perfect physical copy of a person would harbor a separate consciousness and that they would in fact diverge. This suggests that consciousness is a product not of the physical state--structural equality--but is driven by something more like pointer equality. The question is what it's "pointing" from. And, of course, we have no idea; we can't even prove that this is the case.
imtringued 4y ago
I am honestly shocked that nobody mentioned spiking neural networks which are capable of continuous learning. They are much harder to train because conventional neural networks can be differentiated automatically and after that you just dump more hardware at the problem. The upside of spiking neural network accelerators is that they use impossibily low amounts of energy.
- deltaonefour 4y ago
  Likely because people are unfamiliar with what you're talking about. I never heard of this.

skohan 4y ago

Wait are you saying you can achieve identical computational power to a biological neural network with an ANN? How do you justify that assumption?

deltaonefour 4y ago
No. I'm not. I'm saying there must be simple model primitives that don't involve electromagnetic fields and other strange communication channels that arise out of evolution that can be used to model intelligence. We aren't fully clear about what this model of intelligence is, but I'm instinctively sure that it won't have to involve anything as complex as biological neurons that communicate through thousands of different pathways and electromagnetic side effects too.
- levitatorius 4y ago
  It's fascinating that we have simple primitives or notions of analysis, deduction, causation, yet no artificial system where those features of intelligence emerge on their own.
  - deltaonefour 4y ago
    Not true. It's called unsupervised learning.
    - levitatorius 4y ago
      What do you think is missing then? Unsupervised learning == general intelligence?
      - deltaonefour 4y ago
        Not talking about agi. I am doing proof by contradiction. Yes current models of ML are primitive but the reasoning attributes you brought up have been reproduced in ML.. albeit in a primitive way. It is still proof by contradiction, what you say is categorically not true.
- skohan 4y ago
  But it's a massive leap of faith to assume that the massive and varied array of information processing modalities in the brain are mostly unnecessary, or even that the "important" part of how the brain processes information can be modeled using classical computing at all in a way that's more efficient than biological neurons. That's just a feeling you have.
  - deltaonefour 4y ago
    It's not a massive leap of faith at all. Two reasons: 1. We know how natural selection works, and it clearly often produces unnecessary parts for many things. 2. There is physical evidence of evolution creating unnecessary and inefficient parts and components. 1. Natural selection works by randomly creating a fixed set of features both bad and good and then selecting the best one. Thus the features are bounded by randomness. If the random mutation doesn't create the most efficient part then The only thing that can be selected is an inefficient part. This follows that it is VERY possible that many processing modalities of the brain CAN be unnecessary. It is NOT a massive leap of faith. Additionally the selection criteria is simply survival. Not efficiency. As long as the "processing modalities" in the brain aid in survival the selection process does not interfere. Thus all kinds of arbitrary processing modalities both efficient and inefficient can occur so long as the modalities do not detriment survival. Why does stupidity evolve? Well stupid people aren't smart enough to create nuclear bombs to kill themselves. Thus stupidity is a possible trait, similar to how inefficient and unnecessary "processing modalities" are possible, mayhap being stupider aided in survival or didn't contribute anything at all. Just looking at people responding to me on HN, alot of them are pretty stupid (physical evidence). Not pointing out which one, certainly not you. 2. There is (alot) of physical evidence of the above. For example, the wheel is a very efficient way of traveling yet no animal has evolved some form of a wheel. The reasoning is actually more complicated about why a wheel hasn't evolved but the general idea I described above still applies... both natural selection and random mutation were not able to generate the series of intermediate traits required to form wheels. Thus from the reasoning above it is very possible for the brain to evolve with unnecessary and inefficient "processing modalities". Yes this idea did begin as a gut feeling. Subconscious processing is often times correct and logical. It is unwise to constantly mistrust it. It is always wise to analyze these "feelings" in attempt to break down the logic behind why it occurred. What my self analysis tells me is this: Given the fact that MANY of our mathematical models for physical processes involve elegant and straightforward primitives it follows that intelligence will also VERY likely be such a model as well and very likely much more simplified then the human brain.

cs137 4y ago

Neural nets can, provably, approximate any function, modulo certain conditions about continuity (i.e., can't be discontinuous in infinitely many places, that sort of thing). Universality is trivial to prove with boolean functions (because a neural net can do NAND) but also it's also mostly true (again, subject to certain analytical demands) on R^n. The problem is that it can require an exponential amount of time as well as data to actually get a decent network--and, of course, there's a high risk of it being overfit to idiosyncracies in the data. The standard approach to neural network training is to convert it into a gigantic calculus problem and brute-force it, and this doesn't always result in timely convergence or even a generalizable model.

skohan 4y ago
But is there any evidence that the totality of what the human brain does can be modeled as a mathematical function?
- cs137 4y ago
  Whether we are more than computers is almost a religious question (I believe yes, but I can't prove it). However, any computation that we can prove through finite language, in the sense of absolute mathematical proof, can be checked by a mechanical process or, equivalently, a Turing machine (computer). If there's something more, we don't have the tools to prove it.
  - deltaonefour 4y ago
    >Whether we are more than computers is almost a religious question (I believe yes, but I can't prove it). This is actually not true and can logically be shown. We can model atoms with computation. Thus if all matter including the brain is made up of atoms then it can be simulated via a math equation.
    - cs137 4y ago
      We can't perfectly model a physical system (a quantum system, at some level) with classical computing. We'd need a quantum computer, and we still don't know what are the limitations on our ability to build one (the quantum computers currently being built are not at nearly the level we'd need for this sort of analysis). The idea that we understand what physical matter really is, how it works, whether it can produce qualia, et al, is simply off the mark. Of course most physical systems can be reasoned about macroscopically and are relatively deterministic--we can understand how computers work because the functionality is almost entirely classical--and we know that quantum mechanics is both mathematically and experimentally valid, but we not at all gotten to the bottom of this. In fact, one can argue that physics as we understand it can't be used to explain qualia. Physics requires symmetries (translational, rotational, spacetime) and asserts that there is no "privileged" reference frame that defines, say, the "objective" origin or x-, y-, z-axes. If you have two pure kilogram spheres of iron--of course, nothing macroscopic is actually 100.00% pure in practice, but it is physically conceivable and possible--they are exactly the same. One is a perfect substitute for (or copy of) the other. Consciousness likely violates this. We believe (although, without proof) that a perfect physical copy of a person would harbor a separate consciousness and that they would in fact diverge. This suggests that consciousness is a product not of the physical state--structural equality--but is driven by something more like pointer equality. The question is what it's "pointing" from. And, of course, we have no idea; we can't even prove that this is the case.

imtringued 4y ago

I am honestly shocked that nobody mentioned spiking neural networks which are capable of continuous learning. They are much harder to train because conventional neural networks can be differentiated automatically and after that you just dump more hardware at the problem. The upside of spiking neural network accelerators is that they use impossibily low amounts of energy.

deltaonefour 4y ago
Likely because people are unfamiliar with what you're talking about. I never heard of this.