3 ms·
Another thing that could happen is a fundamental algorithmic advance to replace backprop. The brain doesn't use it, and backprop doesn't have answers for how to
by abstractcontrol 4y ago
Another thing that could happen is a fundamental algorithmic advance to replace backprop. The brain doesn't use it, and backprop doesn't have answers for how to do long term credit assignment and continuous learning. It works poorly with RL as well.
Better algorithms would require writing new ML libraries which would weaken the stranglehold GPUs have on ML. New niches opening would inevitably be bad for Nvidia, but good for upstarts.
- webmaven 4y agoGeoffrey Hinton recently came up with a new "forward forward" algorithm that seems to fit the bill: https://bdtechtalks.com/2022/12/19/forward-forward-algorithm-geoffrey-hinton/ https://bdtechtalks.com/2022/12/19/forward-forward-algorithm...