4 ms·
So, they didn't have backprop and automatic differentiation in the 50s. That's pretty fundamental and not just "fiddling with architectures"
by habitue 8y ago
So, they didn't have backprop and automatic differentiation in the 50s. That's pretty fundamental and not just "fiddling with architectures"
- gambler 8y agoBut being fundamental in this context is a bad thing. It's not like there is a single "neural" architecture that's getting better and better. There are dozens of different architectures with their own optimizations, shortcuts, functions and parameters.
- Cybiote 8y agoThis statement is fairly inaccurate. If you check Peter Madderom 1966 thesis, you'll see that it states the earliest work on automatic differentiation was done in the 1950s. It's just that back then, it was called Analyitc differentiation. You can see many of the key ideas already existed back then, including research into specializations for efficiently applying the chain rule. https://academic.oup.com/comjnl/article/7/4/290/354207 https://academic.oup.com/comjnl/article/7/4/290/354207
- habitue 8y agoAh, you're right on AD. But backprop was invented in the 80s