4 ms·
It's the latter. For every LLM out there. They are trained to memorize, not reason. It will take radically different training techniques to make these networks
by ProjectArcturis 3y ago
It's the latter. For every LLM out there. They are trained to memorize, not reason. It will take radically different training techniques to make these networks reason in a human-like way.
- jdkee 3y agoThen how do you explain zero-shot performance?
- ben_w 3y agoMemorising is so trivial we've been doing it by default since forever, regardless of if that means magnetic core memory, the Jacquard Loom, the Gutenberg press, the ceramic movable type China had for a few centuries before Gutenberg, or using a stick to smudge words into soft clay tablets that were accidentally made permanent by a house fire. AI like this aren't just memorisation. They almost certainly don't think like us — even if they did at a low level, the training regime would take the equivalent of hundreds of human lifetimes, and the number of parameters in the larger models is a thousandth of the number in a human brain.