3 ms·
LLMs can be deterministic machines, you just need to control the random seeds and run it on the same hardware to avoid numerics differences. Gradient descent i
by spongebobstoes 11mo ago
LLMs can be deterministic machines, you just need to control the random seeds and run it on the same hardware to avoid numerics differences.
Gradient descent is not a total black box, although it works so well as to be unintuitive. There is ongoing "interpretability" research too, with several key results already.
- nahuel0x 11mo agoDeterministic doesn't necessarily mean that can be understood by an human mind. You can think about a process entirely deterministic but so complex and with so many moving parts (and probably chaotic) that a humble human cannot comprehend.