3 ms·
The main point here is that Large Language Models are crushing through barriers that people like Gary Marcus have previously deemed impossible to surpass. Check
by zwaps 4y ago
The main point here is that Large Language Models are crushing through barriers that people like Gary Marcus have previously deemed impossible to surpass.
Check past episodes by these guys on Machine Learning Street Talk podcast (the whole "symbolic AI" crowd) for some gold quotes that just seem silly now!
I mean, we are not that far from 2017, when LLM's couldn't really write coherent text. And, for example, we had people saying then that they would never be really coherent because text is too sparse, see Chomsky etc. etc. Instead, all they would do is to copy+paste training examples. Naturally, these people didn't really understand how transformers form interdependencencies and how they are composing sequences in a much more complex fashion - as by now is obvious. However, at they time, and with their crude understanding of these models, these guys sure were confident to be right.
Next was Questions Answering, Reasoning, Math problems, Deduction, Coding... each things that LLMs could never do! They were so sure of it. And sure enough, a new model comes around that does it pretty well.
Researchers are bitter for two reasons.
First, We do not say that LLMs are perfect or general AI. Nobody says this. It's an obvious point not worth arguing over.
But Gary Marcus et al. have become extremely popular. It's probably because some readers are slightly concerned about AI, and want someone to tell them that AI isn't really AI yet (duh) and won't replace their precious job (yet?). Their point is a strawman, and what goes beyond it (AI can't ever... X) is mostly wrong.
Second, however, these people have never contributed to the actual models. They are not part of the progress that - notably - is crushing through these barriers. They are bystanders.
And it's freaking annoying, because their understanding of LLM's is imperfect (as is the case for everyone!), and yet they come up with these statements of absolute certainty.
There is indeed research into these questions. But it is far, very far from resolved. In that sense, these people are - sorry to say - charlatans.
Well at least, everyone I know rolls their eye when the next Gary Marcus article or tweet comes around.
- cfiggers 4y ago> their understanding of LLM's is imperfect (as is the case for everyone!), and yet they come up with these statements of absolute certainty. Oh interesting, so when they accuse ChatGPT of being dangerous because it can be "confidently inaccurate" it's the pot calling the kettle black
- zwaps 4y agoChatGPT is trained via reinforcement learning to give answers that - ultimately - human non-experts would judge to be plausible. Answers were not trained to be correct or accurate - hence we should not expect ChatGPT to do well there. Sociologists have found that humans seek plausible answers more than correct answers, especially when trying to make sense of a situation. For that reason, the training objective makes sense for a chatbot. Now, anyone who tries to seel ChatGPT as giving truthful answers is of course doing a disfavor to the model and its training objective. However, anyone also trying to sell the above either as a) Deep insight, or b) inherent limitation of LLMs is indeed a pot calling the kettle black. Going from plausibility to truthfulness is probably not a giant leap - at least not compared to what was achieved in the past three years. There's active research on it, and I am sure a good solution will arise. However, ChatGPT is not that, and it isn't meant to be. It's not the training objective. Making this a huge point is again such a BS strawman as the other things I mentioned before.
- mannykannot 4y ago> ChatGPT is trained via reinforcement learning to give answers that - ultimately - human non-experts would judge to be plausible. Ostensibly, it is being trained give replies that are more likely than most to be given by humans. On the reasonable (at least IMHO) assumption that people's speech is biased towards saying things that their target audience might see as plausible, this can be seen as doing what you say in the above quoted. This is not, however, the same as being able to judge the plausibility of the semantic content of those statements, so I remain skeptical that the current methods for training LLMs are capable of creating that ability, which in turn leads me to be skeptical that going from plausible statements to reasonably reliably true statements will be a relatively small step in comparison*. I'm open to being surprised, however, as I already have been by what has been achieved so far. The LLM-does-math paper you mention in your sibling comment looks extremely interesting, and maybe I will change my mind... * At least by a continuation or scaling-up of current methods. Humans may be more likely to say truthful things than falsehoods (at least in some areas and some contexts) but a gap has formed between plausible and truthful LLM productions, and I don't see any particular reason to think more of the same will close it.
- deleted 4y ago[deleted]
- elicksaur 4y agoWe have different definitions of pretty well. It can’t do basic math consistently.
- zwaps 4y agoIf you prompt it correctly it seems to be able to solve most basic math problems you throw at it. Here's a PNAS claiming human-level performance: https://www.pnas.org/doi/full/10.1073/pnas.2123433119 https://www.pnas.org/doi/full/10.1073/pnas.2123433119 There's a deluge of papers and research right now, so there's quite a bit of complexity to saying "pretty well". However, let me say it this way: Compared to other LLMs, recent OpenAI models score highly on logic and math exercises. Yes, there ARE better LLM's trained to do math computations (especially fine-tuned for certain problems), but I'd say ChatGPT is certainly impressive as a general text and code model. The other side of the medallion of saying "pretty well" is this: There is no other type of computational approach that is able to solve free-form logic or math queries in any capacity. There is no symbolic approach that can "extract" a math problem from text and then solve it - in code or otherwise, whereas LLMs are getting close to human performance on such (and related tasks). So yeah. Pretty well is what I'd claim.
- elicksaur 4y agoAs I understand that paper, I would take issue that the LLM isn’t “doing” math at all. Look at Figure 4 for the process to be most clear. It’s generating a text program that when run via Python can solve the problem. All the LLM is doing is matching the equation in the question to whatever operators it needs in Python syntax. I wouldn’t call that doing or understanding math.
- zwaps 4y agoSo yes, you’ll probably want to take a model trained on math problems to forgoe the step with code. But then, note that writing correct programs is indeed a high level solution to a full text math problem, is it not? Going from there to solving it directly should be a matter of some tuning. Finally, who said understanding math? The whole debate is about getting shockingly useful results precisely without symbolic reasoning. If the main contention is that got does not do symbolic reasoning, then we are back at Gary Marcus… yes we know this. It’s not why researchers are so amazed by these models. Its that they output steps (or in this case code since its Codex based) solving university level math with a simple transformer architecture