4 ms·
Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You can argue there is some quantitative difference between modern neutral networks
by Retric 5y ago
Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You can argue there is some quantitative difference between modern neutral networks but they hadn’t reached the point of trying to train a neural network, it was much more abstract.
Also, the brain does actually use a form of supervised learning. Pain for example is effectively a form of input labeling and lack of pain the wanted output.
- mjburgess 5y agoIf the neural network is just the piece-wise linear regression function that the NN algorithm generates -- then its just a network like any other mathematical function. And it's a pretty crappy mathematical function, with zero homology to the brain. Nothing in the brain is piece-wise linear.
- Retric 5y agoNN are a more than just feed forward NN. Assuming infinite precision finite NN can be turing complete.
- mjburgess 5y agoSo is backprop a property or not? > Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You are the one claiming it isnt
- Retric 5y agoNo, I don’t think you understand the terminology. Feed forward NN means a NN without loops. Back propagation is a method of generating a new NN that only works on a subset of NN. I can generate a random number from a rolling a size sided dice once, just not any possible number.
- mjburgess 5y agoI don't know what we're describing by "NN"; which is, in large part, exactly my point. I was taking you to be flip-flopping on backwards/forwards feeding; rather than to be talking about recurrent loops. What is meant to be meaningfully a graph? What definition of "neural network" are you using? The NN model itself, even recurrent, is just a regression function. In the sense that a Turing machine is just an implementation of a function from the Naturals to Naturals -- yes, a regression function, is likewise, a program. The thing which is "Turing complete" cannot possibly be any particular regression function, as that is simply one program. So it has to be "the function of the function" -- which isnt a graph. So again: what exactly is a graph here? What is the definition of NN you're using? The NN family of algorithms are just means of interpolating through compressed feature spaces; ie., they are just kernel machines which remember compressions of their training data and compute distances for predictions. That process, like essentially any function-generator, can without much work be made Turing complete. This, like being a "universal funciton approximiator" isnt an interesting property. But either way, it still doest explain how any of this has anything to do with networks, brains, or anything like it.