2 ms·
Do LLMs still use backprop?
by brcmthrowaway 1y ago
Do LLMs still use backprop?
- ForceBru 1y agoAre LLMs still trained by (variants of) stochastic GRADIENT descent? AFAIK what used to be called "backprop" is nowadays known as "automatic differentiation". It's widely used in PyTorch, JAX etc
- imtringued 1y agoGradient descent doesn't matter here. Second order and higher methods still use lower order derivatives. Back propagation is reverse mode auto differentiation. They are the same thing. And for those who don't understand what back propagation is, it is just an efficient method to calculate the gradient for all parameters.
- deleted 1y ago[deleted]
- samsartor 1y agoYes. Pretraining and fine-tuning use standard Adam optimizers (usually with weight-decay). Reinforcement learning has been the odd-man out historically, but these days almost all RL algorithms also use backprop and gradient descent.