5 ms·
François Chollet, author of Keras, lays it out pretty clearly that LLMs are just memorizing [0] (there’s more links if you follow that thread). I think LLMs ar
by PheonixPharts 3y ago
François Chollet, author of Keras, lays it out pretty clearly that LLMs are just memorizing [0] (there’s more links if you follow that thread).
I think LLMs are incredible, and spend most of my days working with them closely, but they are not nearly as close to “AGI” as people think primarily due to their inability to really generalize.
At the end of the day LLMs aren’t that different than old school n-gram Markov chains, except rather than working on n-grams, they’re working in a (very sophisticated) latent space. Their power is really these incredible latent languages spaces we’re still just starting to understand.
In all my years of tech the “AI” space is the most curious hype-bubble since the things people expect to happen are entirely out line with what is possible, while at the same time the potential of these models is still, imho, underexplored and largely ignored by the vast majority of people attempting to build things with them.
99% of the people I know working in this space are just calling APIs and trying to do some variant of code generations, where a small minority of people are really trying to figure out what’s going on in these models and what can be done with them successfully.
0. https://twitter.com/fchollet/status/1755250582334709970 https://twitter.com/fchollet/status/1755250582334709970
- dartos 3y agoI think there’s a lot of people who got into tech in 2020. They are new programmers and technologists and made a life change in 2020. I think they were a big part of the crypto bubble. Lots of talent, hungry for that sweet startup gold, but without the technical background to really know what’s going on. I believe these same groups are operating in the same way with AI. Recklessly bashing together APIs and cloud services to create MVPs. It’s all the worst parts of startup culture concentrated. Anyway, that’s why i think most of the AI space rn is just people calling APIs and acting like they discovered fire. </salty rant>
- deleted 3y ago[deleted]
- CamperBob2 3y agoShow me a Markov chain that can thrash a 9-dan Go master. "But that wasn't an LLM." OK, show me a Markov chain that can write a Python program that can play Go at all.
- xapata 3y agoSemantics, I suppose. Those ANNs were Markov chains, in a sense.
- vidarh 3y agoPop a loop around a Markov chain that provides a "tape interface" and you have something capable of simulating a Turing machine. So when people bring up the Markov chains argument, they're saying next to nothing about the potential computational abilities of the system, even though they usually intend to dismiss it. I tend to see people bringing that up in a dismissive way (not suggesting you are) as a clear indication they either haven't thought the argument through or do not understand how little it takes for a system to be Turing complete, and so for that argument to be meaningless.
- xanderlewis 3y agoI mean, I’m not sure what you’re trying to prove by asking for a Markov chain model like that. It’s trivially true that you can have a Markov chain output whatever you like (somewhat artificially, but we are talking about memorisation here) if you pick your training data carefully.
- CamperBob2 3y agoSo "pick the training data carefully," and show me what I'm asking for, given that it is "trivial."
- xanderlewis 3y agoThe data consists of a graph whose vertices correspond to the output sequence you desire and where there is an edge (with probability 1) from string x to string y if and only if x precedes y. This model will arise from the desired sequence as a single training example (learning the probability of each pair of consecutive tokens), provided none are repeated. Now run your Markov chain with initial input {first token in your sequence}.
- danielmarkbruce 3y agoA lot of the words he uses in that tweet aren't well defined. e.g. memorization, dataset (does he mean the literal words/tokens or any token that is close in space after embedding?), pattern, category, program. The tweet is practically meaningless. I'm not criticizing him because his blog post is nuanced and he clearly understands what he's talking about, but that tweet almost certainly means something quite specific to him and he's communicating quite poorly. As you mention, there is a sophisticated representation of the tokens. It's so sophisticated that one may reasonably stop calling them tokens (or, even data) and start calling them "concepts". Now, if someone (or something) has memorized how all the concepts go together... that's pretty darn intelligent.
- tymscar 3y agoIm curios, what sort of area you think needs more research when it comes to these models
- VirusNewbie 3y agoBut we can trivially show that the larger models can generalize for some questions even if we verify the answer isn’t in the training set.
- astrange 3y agoWe can prove that transformers can do computation beyond memorization, and that they can learn it from examples, unless you think learning an algorithm is memorization. https://arxiv.org/abs/2301.05217 https://arxiv.org/abs/2301.05217 https://arxiv.org/abs/2310.16028 https://arxiv.org/abs/2310.16028
- bamboozled 3y agoMaybe there doing something we don’t really have a name for yet ? This is why it causes so much controversy. Not really generalizing, not memorizing, maybe approximating ?
- dataflow 3y ago"Vibing"?
- mirekrusin 3y agoNo need to come up with any new name, we already have it: 1. autocompletion or 2. next token prediction and 3. (reverse) diffusion
- ein0p 3y agoThere is a name for this already: associative memory. That’s how you catch a ball: you condition your memory of catching a ball with proprioceptive and visual input, much like a multimodal transformer. There’s no thinking involved - you wouldn’t be able to catch it if you had to think
- vidarh 3y agoAn associative memory can model truth tables of logic circuits, and so the steps to go from associative memory to arbitrary computation pretty much requires adding state and a loop. And we do provide state and a loop. As such, while that does not prove that any given model is capable of reasoning, describing what goes on at any step as associative memory says nothing meaningful about the computational power of the wider system. It's very possible the way we train current models will turn out to place fundamental limits on the abilities the models will have, but that will not be because the models act as associative memories.
- ein0p 3y agoFrancois is not a researcher, however. LLMs aren’t just plain “memories”. They are very explicitly _associative_ memories. And it just so happens that this is mostly what our own brains do, too. Actual cognition is slow and expensive for us, and we try to use it as little as possible, filling in what we can with easy, associative, low energy, near instant stuff. Therein lies the reason why AI can be considered a boon for us humans. If machines took over the mundane work that just drains our energy and doesn't add much value, we could finally have the time to actually do what they can’t - think deeply about stuff, with their help where we find our faculties lacking. Rocket for the mind, if you will, rather than a bicycle.
- PheonixPharts 3y agoYou must have a pretty impressive resume if you don’t consider François a researcher! The wikipedia would disagree with you [0], as would anyone that has had any interaction with him on the subject. 0. https://en.wikipedia.org/wiki/Fran%C3%A7ois_Chollet https://en.wikipedia.org/wiki/Fran%C3%A7ois_Chollet
- ein0p 3y agoTry to find a single paper on Transformers or LLMs in general in Francois’ scientific output: https://dblp.org/pid/116/8242.html https://dblp.org/pid/116/8242.html Don’t get me wrong, Keras is impressive, and Francois is impressive as well. But for insight on LLMs you should probably listen to people who specialize in them.
- mirekrusin 3y agoWhat is intelligence or generalization if not "sophisticated, complex latent space" navigation?
- eigenket 3y agoAs far as I know we don't have much real understanding of how human or even animal intelligence works. It may be that it is entirely based on "sophisticated, complex latent space" navigation or it may be that it has nothing whatsoever to do with that (or anything in-between).
- K0balt 3y agoI think this is exactly what most people are misunderstanding. LLMs are wonderful tools for navigating the sociolinguistic matrix, which embeds the majority of human knowledge both explicitly and covertly with a huge amount of contextual nuance. Transformers are capable of decoding and operating on this covert knowledge as easily as the encyclopedic knowledge that we tend to assume is the “main part” when we write something down. What llms demonstrate is that language covertly encodes logic and algorithmic knowledge that is at least as rich as the “factual” encoding that written content seems to be at face value. LLMs just make this knowledge accessible for computation. What is amazing is that this function alone is capable of producing a simulacrum of agency and intelligence all by itself. This suggests that the human cultural component is a pretty huge part of what we consider to be “human” and without it we’d just be clever apes. It’s the clever ape part that LLMs don’t have, but it’s possible that the transformer model might be able to “ape” most of that as well if applied to the task of existing as an embodied entity in the world (see transformer use in robotic task completion) I would not be terribly surprised if comprehensive multimodality, encompassing the entire spectrum of sensory experience as well as physical environment interactivity, gets us really close to something we could consider AGI. Just as LLMs extract the embedded relationships in written knowledge, the physical and sensory spaces encode an enormous quantity of information that can similarly be generalised to extrapolate a cornucopia of additional concepts (gravity, object permanence, physics in general, relativity, etc). Meaningfully “tokenizing” these spaces will likely be the key to making this work effectively.
- 3y ago
- andy_ppp 3y agoOur brains can reuse patterns they learned in one area into another area, LLMs need specific examples and can combine these in weird unpredictable and often inhuman ways losing context and meaning, the bits humans care about. They often aren’t even a good starting point compared to just thinking deeply for 5 minutes.
- alecco 3y agoAbout [0], Phi-2 is sort of a proof of concept of having very narrow and high quality dataset normal transformer models can perform at 8x to 10x better results. Of course, if you add messy prompts they will fail! What a dumb view. And the Yi-* models are suspect of being trained on the test set or at least be contaminated. All the other models barely move and if they do, it's probably an artifact of being multiple choice. There were papers showing most models improve if they can reason the answer by letting it have more tokens in the answer. The chat elo-like rankings are much more interesting: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... For completeness, here's the paper linked in the tweet https://arxiv.org/pdf/2402.01781.pdf https://arxiv.org/pdf/2402.01781.pdf