4 ms·
I'm not surprised you can't use it to make it better, but one might imagine gradients would go to zero as you fed the model its own output.
by anon291 2y ago
I'm not surprised you can't use it to make it better, but one might imagine gradients would go to zero as you fed the model its own output.
- slashdave 2y agoNo, not even close. Gradients don't come to zero in the first place. Training is never perfect.
- anon291 2y agoLet's restate. I'd imagine you end up in local minima that are difficult to escape using model generated data. So sure, non-zero gradients, but if you plot the gradients, I would expect them to orbit at that point. But it seems like they diverge.
- slashdave 2y agoMini-batches and dropout mean that you are constantly jumping out of and into other minima during training of any type (highly-redundant solution space is an important feature of deep learning). This is deliberate and necessary to explore the gigantic parameter space of these huge LLM models.
- anon291 2y agoSure, but one might think that training on self-generated data would keep you in a constrained subset of minima, but that is not the case.