4 ms·
This is probably a "publishing in the press" translation issue, but the notion that neural networks have no memory is dead wrong. DNNs encode incredibly specif
by evrydayhustling 7y ago
This is probably a "publishing in the press" translation issue, but the notion that neural networks have no memory is dead wrong. DNNs encode incredibly specific histories of training data in their weights, which is both a privacy/security risk [1] and a phenomena that is actively being utilized in many dnn architectures. Overall, this has a "brand the framework before proving exceptional capabilities" feel.
[1] https://arxiv.org/abs/1802.08232 https://arxiv.org/abs/1802.08232
- gnode 7y agoI don't think that's what they meant. I interpreted it to mean: neural networks are not trained in a continuous online manner. That said, there are neural network architectures which have state, such as recurrent neural networks, time-delay neural networks, and long short-term memory. However, the state is used for covering problem domains with a temporal nature, rather than for reflective learning. It's typically reset between different inputs.
- sgillen 7y agoThere are lots of examples of networks being trained online, I see it all the time in robotic control.