4 ms·

The 1943 NN compu sci model corresponds to a 1943 brain model, sure. NNs in an ML sense arent this model. What does any modern NN compu-sci model correspond to

The 1943 NN compu sci model corresponds to a 1943 brain model, sure. NNs in an ML sense arent this model. What does any modern NN compu-sci model correspond to? You're making a genetic fallacy here: that since the origin of X had property P, so must X. Eg., marriage was historically a legal contract of ownership, so today it is a legal contract of ownership. The brain has nothing akin to back-propagation, nor supervised training. "Neurones" are not "data about house prices". According to any even highly simplified neurological model, a compu sci model of this, would compute with activations in a radically different manner than a dot-product over domain-specific data. They are also not a weighted directed graph. In a sense almost any mathematical function is a weighted graph if one takes edges to be, eg., general function composition; and nodes to be input/output domains of those functions. However to say a "NN is a network" is presumably meant to be more significant than this trivial observation that essentially "all functions" are. In particular everything which distinguishes a NN from any mathematical functions are operations such as supervised training, backprop, regularization, pooling, etc. And these are not expressible as functions within this graph, but they are functions over the graph. So any actual "network diagram" presented isn't a network. Its a bad diagram of a mathematical function which also includes functions over this diagram. In any case, a neural network -- in the ML sense -- is neither neural nor a network. It is a function of a mathematical function that has no isomorphism to anything in the brain.

Retric 5y ago
Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You can argue there is some quantitative difference between modern neutral networks but they hadn’t reached the point of trying to train a neural network, it was much more abstract. Also, the brain does actually use a form of supervised learning. Pain for example is effectively a form of input labeling and lack of pain the wanted output.
- mjburgess 5y ago
  If the neural network is just the piece-wise linear regression function that the NN algorithm generates -- then its just a network like any other mathematical function. And it's a pretty crappy mathematical function, with zero homology to the brain. Nothing in the brain is piece-wise linear.
  - Retric 5y ago
    NN are a more than just feed forward NN. Assuming infinite precision finite NN can be turing complete.
    - mjburgess 5y ago
      So is backprop a property or not? > Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You are the one claiming it isnt
      - Retric 5y ago
        No, I don’t think you understand the terminology. Feed forward NN means a NN without loops. Back propagation is a method of generating a new NN that only works on a subset of NN. I can generate a random number from a rolling a size sided dice once, just not any possible number.
        mjburgess 5y ago
        I don't know what we're describing by "NN"; which is, in large part, exactly my point. I was taking you to be flip-flopping on backwards/forwards feeding; rather than to be talking about recurrent loops. What is meant to be meaningfully a graph? What definition of "neural network" are you using? The NN model itself, even recurrent, is just a regression function. In the sense that a Turing machine is just an implementation of a function from the Naturals to Naturals -- yes, a regression function, is likewise, a program. The thing which is "Turing complete" cannot possibly be any particular regression function, as that is simply one program. So it has to be "the function of the function" -- which isnt a graph. So again: what exactly is a graph here? What is the definition of NN you're using? The NN family of algorithms are just means of interpolating through compressed feature spaces; ie., they are just kernel machines which remember compressions of their training data and compute distances for predictions. That process, like essentially any function-generator, can without much work be made Turing complete. This, like being a "universal funciton approximiator" isnt an interesting property. But either way, it still doest explain how any of this has anything to do with networks, brains, or anything like it.

Retric 5y ago

Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You can argue there is some quantitative difference between modern neutral networks but they hadn’t reached the point of trying to train a neural network, it was much more abstract. Also, the brain does actually use a form of supervised learning. Pain for example is effectively a form of input labeling and lack of pain the wanted output.

mjburgess 5y ago
If the neural network is just the piece-wise linear regression function that the NN algorithm generates -- then its just a network like any other mathematical function. And it's a pretty crappy mathematical function, with zero homology to the brain. Nothing in the brain is piece-wise linear.
- Retric 5y ago
  NN are a more than just feed forward NN. Assuming infinite precision finite NN can be turing complete.
  - mjburgess 5y ago
    So is backprop a property or not? > Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You are the one claiming it isnt
    - Retric 5y ago
      No, I don’t think you understand the terminology. Feed forward NN means a NN without loops. Back propagation is a method of generating a new NN that only works on a subset of NN. I can generate a random number from a rolling a size sided dice once, just not any possible number.
      - mjburgess 5y ago
        I don't know what we're describing by "NN"; which is, in large part, exactly my point. I was taking you to be flip-flopping on backwards/forwards feeding; rather than to be talking about recurrent loops. What is meant to be meaningfully a graph? What definition of "neural network" are you using? The NN model itself, even recurrent, is just a regression function. In the sense that a Turing machine is just an implementation of a function from the Naturals to Naturals -- yes, a regression function, is likewise, a program. The thing which is "Turing complete" cannot possibly be any particular regression function, as that is simply one program. So it has to be "the function of the function" -- which isnt a graph. So again: what exactly is a graph here? What is the definition of NN you're using? The NN family of algorithms are just means of interpolating through compressed feature spaces; ie., they are just kernel machines which remember compressions of their training data and compute distances for predictions. That process, like essentially any function-generator, can without much work be made Turing complete. This, like being a "universal funciton approximiator" isnt an interesting property. But either way, it still doest explain how any of this has anything to do with networks, brains, or anything like it.