3 ms·

So is backprop a property or not? > Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You are the one claiming it isnt

Retric 5y ago
No, I don’t think you understand the terminology. Feed forward NN means a NN without loops. Back propagation is a method of generating a new NN that only works on a subset of NN. I can generate a random number from a rolling a size sided dice once, just not any possible number.
- mjburgess 5y ago
  I don't know what we're describing by "NN"; which is, in large part, exactly my point. I was taking you to be flip-flopping on backwards/forwards feeding; rather than to be talking about recurrent loops. What is meant to be meaningfully a graph? What definition of "neural network" are you using? The NN model itself, even recurrent, is just a regression function. In the sense that a Turing machine is just an implementation of a function from the Naturals to Naturals -- yes, a regression function, is likewise, a program. The thing which is "Turing complete" cannot possibly be any particular regression function, as that is simply one program. So it has to be "the function of the function" -- which isnt a graph. So again: what exactly is a graph here? What is the definition of NN you're using? The NN family of algorithms are just means of interpolating through compressed feature spaces; ie., they are just kernel machines which remember compressions of their training data and compute distances for predictions. That process, like essentially any function-generator, can without much work be made Turing complete. This, like being a "universal funciton approximiator" isnt an interesting property. But either way, it still doest explain how any of this has anything to do with networks, brains, or anything like it.

3 ms·

So is backprop a property or not? > Back propagation etc isn’t a property of a NN it’s a means to generate a NN. You are the one claiming it isnt

by mjburgess 5y ago

Retric 5y ago
No, I don’t think you understand the terminology. Feed forward NN means a NN without loops. Back propagation is a method of generating a new NN that only works on a subset of NN. I can generate a random number from a rolling a size sided dice once, just not any possible number.
- mjburgess 5y ago
  I don't know what we're describing by "NN"; which is, in large part, exactly my point. I was taking you to be flip-flopping on backwards/forwards feeding; rather than to be talking about recurrent loops. What is meant to be meaningfully a graph? What definition of "neural network" are you using? The NN model itself, even recurrent, is just a regression function. In the sense that a Turing machine is just an implementation of a function from the Naturals to Naturals -- yes, a regression function, is likewise, a program. The thing which is "Turing complete" cannot possibly be any particular regression function, as that is simply one program. So it has to be "the function of the function" -- which isnt a graph. So again: what exactly is a graph here? What is the definition of NN you're using? The NN family of algorithms are just means of interpolating through compressed feature spaces; ie., they are just kernel machines which remember compressions of their training data and compute distances for predictions. That process, like essentially any function-generator, can without much work be made Turing complete. This, like being a "universal funciton approximiator" isnt an interesting property. But either way, it still doest explain how any of this has anything to do with networks, brains, or anything like it.