3 ms·
> But this technique was discovered decades ago. It just took this long for computers to speed up, for it to be usable. This isn't really true. Old techniques
by sanxiyn 6y ago
> But this technique was discovered decades ago. It just took this long for computers to speed up, for it to be usable.
This isn't really true. Old techniques don't work well even with modern compute. Now, try replacing sigmoid with ReLU in old networks and see it improve.
- Retric 6y agoMany 90’s era attempts that failed do work vastly better on modern hardware. There are many tweaks that showed up once people could fine tune using vast amounts of processing power, but I would put ~80% of the advancements from HW and 20% from algorithms. Having a million or more times the processing power really makes or breaks modern ML.