3 ms·
whoops, definitely a typo! It should be 5e-4 for as the base "pretraining" LR, you're absolutely correct. your intuition is sound, but my fingers are not.
by pact_inference 1y ago
whoops, definitely a typo! It should be 5e-4 for as the base "pretraining" LR, you're absolutely correct.
your intuition is sound, but my fingers are not.