5 ms·
I assume that multiplying by a given factor shouldn't matter since you still have the learning rate as a factor (which is itself a factor of the gradient). This
by halflings 10y ago
I assume that multiplying by a given factor shouldn't matter since you still have the learning rate as a factor (which is itself a factor of the gradient). This might just mean that the learning rate should be lower or higher with this method.
- gabrielgoh 10y agothis is a good way to think about this.
- im3w1l 10y agoThe question is then really about which method makes it easier to tune parameters or which helps intuition the most.