4 ms·
Francois is not a researcher, however. LLMs aren’t just plain “memories”. They are very explicitly _associative_ memories. And it just so happens that this is m
by ein0p 3y ago
Francois is not a researcher, however. LLMs aren’t just plain “memories”. They are very explicitly _associative_ memories. And it just so happens that this is mostly what our own brains do, too.
Actual cognition is slow and expensive for us, and we try to use it as little as possible, filling in what we can with easy, associative, low energy, near instant stuff.
Therein lies the reason why AI can be considered a boon for us humans. If machines took over the mundane work that just drains our energy and doesn't add much value, we could finally have the time to actually do what they can’t - think deeply about stuff, with their help where we find our faculties lacking. Rocket for the mind, if you will, rather than a bicycle.
- PheonixPharts 3y agoYou must have a pretty impressive resume if you don’t consider François a researcher! The wikipedia would disagree with you [0], as would anyone that has had any interaction with him on the subject. 0. https://en.wikipedia.org/wiki/Fran%C3%A7ois_Chollet https://en.wikipedia.org/wiki/Fran%C3%A7ois_Chollet
- ein0p 3y agoTry to find a single paper on Transformers or LLMs in general in Francois’ scientific output: https://dblp.org/pid/116/8242.html https://dblp.org/pid/116/8242.html Don’t get me wrong, Keras is impressive, and Francois is impressive as well. But for insight on LLMs you should probably listen to people who specialize in them.
- kmac_ 3y agoIf you measure knowledge by the number of publications, then LLMs know nothing.
- vidarh 3y agoThey made the argument he's not a researcher in this field, not that he doesn't know anything. And an LLM is indeed not a researcher in this field either.
- kmac_ 3y agoIt appears that my previous comment lacked clarity. I assumed the inherent absurdity and illogicality of the statement would be self-evident. Dismissing François's opinion, a seasoned professional with extensive expertise in model development and the creator of ARC, solely because he asserts that LLMs are far from achieving AGI and using publication count as the metric for evaluating him is not a well-reasoned argument. While LLMs and transformers are indeed remarkable achievements and will undoubtedly reveal more properties, they have yet to exhibit any true signs of "intelligence."
- vidarh 3y agoAcknowledging that they have not yet shown signs of "true intelligence" is a vastly more moderate claim than the person above ascribed to him.
- djmips 3y agoMore like AI will finally be the bicycle for the mind.
- ein0p 3y agoIDK, credit where credit is due: traditional computers got us much further than we’d be able to go on our own. So they’re a fine “bicycle” in my view.
- djmips 3y agoAn AGI might be something that can harness the LLM but also self learn.
- vidarh 3y agoI think even "actual cognition" is highly unlikely to need anything much more than associative memories plus some state and a loop. The expense being having to "execute" a large number of steps rather than "just" effectively pulling learned results from "cache". To be very reductive, an associative memory can hold a truth table. Put minimal state, IO and a loop around that and you have a universal Turing machine. Which is why the "it's just memory" or "it's just Markov chains" is so tedious - it says near nothing about the computational power of the system including the model. There's plenty of reason, of course, to question to what extent we know the abilities of the models, but when people assume dismissing it as "memory" or "just statistics" or "just Markov chains" I usually take it as a signal they don't understand how few limitations that imposes.
- csomar 3y agoI wonder if cognition or intelligence is actually "brain damage". Your bad memory is leading to erroneous paths but some of these will eureka. Genetic evolution and natural selection are essentially that. If your memory is too bad, then you are either insane or in an advanced state of Alzheimer. If you have enough stable paths to lead a quasi-normal life, then you become an inventor or an artist; or something atypical. Hallucination is the feature, not the bug.