4 ms·
Yes it's a great combo. I think fastai is the only library that actually has both of these built-in. However, because the LR warm-up is built-in to Ranger, it'
by jph00 6y ago
Yes it's a great combo. I think fastai is the only library that actually has both of these built-in.
However, because the LR warm-up is built-in to Ranger, it's actually a bit more fiddly to use - i.e. you really need to understand what it's doing. It doesn't work great with `Learner.fine_tune` and gradual unfreezing more generally, since you don't really want a full separate warmup for each phase.
So I don't see it becoming the default or main optimizer shown in the course. But it's great to learn and use.