4 ms·
> These aren't "general" models. They're statistical models. They're autocorrect or autocomplete on steroids -- and autocorrect/autocomplete don't require symbo
by stavros 2y ago
> These aren't "general" models. They're statistical models. They're autocorrect or autocomplete on steroids -- and autocorrect/autocomplete don't require symbolic reasoning.
This is very "humans are just hunks of matter! They can't think!".
- globnomulous 2y agoI'm not sure what you're getting at. Could you explain?
- stavros 2y agoThat "they predict the next token" doesn't necessarily imply "they can't reason".
- globnomulous 2y agoI'm not saying "they predict tokens; therefore, they can't reason." I'm saying "something that can't reason can predict tokens, so prediction isn't evidence of reasoning." More specifically, my comment aims to meet the challenge posed by the person I answered: > I highly recommend letting that fact be a little bit impressive, someday. There’s no way you live through any event that’s more historically significant, other than perhaps an apocalypse or two. [...] If you have any actual scientific argument as to why a model that can score 90-100 on a typical IQ test has only 1/10th the symbolic reasoning skills of a human, I’d love to eat my words. I have no idea what would constitute a "scientific argument" in this instance, given that the challenge itself is unscientific, but, regardless, the results that so impress this person are, without question, achievable without reasoning, symbolic or otherwise. To say that the model "muses" or "has [...] symbolic reasoning" is to make a wild, arbitrary leap of faith that the data, and workings of these models, do not support. The models are token-prediction machines. That's it. They differ not in kind but in scale from the software that generates predictions in our cell-phone keyboards. The person I answered can be as impressed as he wants to be by the high quality he thinks he sees in the predictions. That's fine. I'm not. In that respect, we just disagree. But if he's impressed because he thinks the model's predictions must or do betoken reasoning, he's off in la la land -- and so his wide-eyed, bushy-tailed enthusiasm is based on nonsense. It's no different from believing that your phone keyboard is capable of reasoning, simply because you are delighted that it guesses the 'right' word often enough to please you.
- stavros 2y agoIf your argument is "they can't reason" (plus some other stuff about how they work), what reasoning test has an LLM failed for you to conclude that they can't reason? Whenever I've given an LLM a reasoning test, it seems to do fine, so it really does sound to me like the argument is "they can't really reason, because <irrelevant fact about how they work internally>".
- globnomulous 2y agoThat isn't the argument. I've stated the argument twice. My longer response to you starts by stating the core of the argument as succinctly and clearly as I can. That's the first paragraph of the post. Not only are you still not getting it. You're also twisting what I wrote into claims I have not made. I'm not going to explain myself a third time. I'll instead say this: if you think these models must be reasoning when they produce outputs that pass reasoning tests, then you should also believe, every time you see a photo of a dog on a computer screen, that a real, actual dog is somewhere inside the device.
- stavros 2y agoYou're right, but my argument is this: You said: > I'm not saying "they predict tokens; therefore, they can't reason." I'm saying "something that can't reason can predict tokens, so prediction isn't evidence of reasoning." This is true. Reasoning is evidence of reasoning, and LLMs do pass reasoning tests. Yes, the way they work doesn't imply that they can reason, the fact that they can reason implies that. You also said: > These models deal with tokens similarly. They don't know what a token is or represents -- or we have no reason to think they do. I have no reason to think that other people know what concepts are or what they represent, just that they can convincingly output a stream of words when asked to explain a context. The argument bugs me because you can replace "they predict the next token" with "they are collections of neurons producing output voltages in response to input voltages" and you'll have the exact same argument about humans.
- globnomulous 2y ago
- tripzilch 2y agoOf course, because the statement "they can reason" is an extraordinary one, requiring rather extraordinary evidence. Therefore I think it's reasonable to say they cannot, until someone actually comes up with that evidence.
- fennecfoxy 2y agoTo be fair parent pointing out they're purely statistical machines predicting next token is incorrect anyways. They are essentially next token predictors after first training, but then instruct models are fine tuned on reasoning and Q/A scenarios, afaik early research has determined that this isn't just pure parroting, that it does actually result in some logic in there as well. People also have to remember the training for these is super shallow at the moment, when compared with what humans go through in our lifespans as well as our millions of years of evolution (as humans).
- globnomulous 2y agoWhat you're saying doesn't contradict what I wrote. I said models are trained. You're saying they're trained and fine tuned -- i.e. continue to be trained. I also didn't say they do any kind of parroting or that logic doesn't take place. I'm saying, rather, that the models do what they're taught to do, and what they're taught to do are computations that give us a result that looks like reasoning, just the way I could use 3ds max as a teenager to generate on my computer screen an output that looked like a cube. There was never an actual cube in my computer when I did that. To say that the model is reasoning because what it does resembles reasoning is no different from saying there was an actual cube somewhere in my computer every time I rendered one.