3 ms·
Going to go out on a limb and say they are probably referring to the gradient calculus required for updating the model. https://en.wikipedia.org/wiki/Different
by meowkit 3y ago
Going to go out on a limb and say they are probably referring to the gradient calculus required for updating the model.
https://en.wikipedia.org/wiki/Differentiable_programming https://en.wikipedia.org/wiki/Differentiable_programming
See automatic differentiation.
- throwawaymaths 3y agoCorrect, but note that if you subject a standard hash table algo to AD it won't magically become a transformer. (Hashes in the "normal construction" are discrete functions and thus aren't really continuous or differentiable, neither are lookup tables)