4 ms·
I have seen this too. My pet theory is that, at all times, the brain is busy doing gradient descent on all the problems you have ever encountered in your life.
by plutonorm 5y ago
I have seen this too. My pet theory is that, at all times, the brain is busy doing gradient descent on all the problems you have ever encountered in your life. Everything is getting improved in parallel. So when you step away from the problem for months at a time and come back you are reaping the benefits of all that background computation.
- Retric 5y agoAlternatively, you where doing gradient decent with daily practice but got stuck at a local minima. Taking a real break helps you get past that. However, I don’t think either represent what’s actually happening very well.
- snovv_crash 5y agoI think what happens is that during the active learning part you heavily overfit and end up in a local minima. Afterwards life acts like a regularisation, and when you return to it you are no longer in a minima but now have a gradient to descend on again.