3 ms·

So I understand the author has high standing in the community. But I think they are making actually disingenuous arguments by mixing assertions that are true b

by ftxbro 4y ago

So I understand the author has high standing in the community. But I think they are making actually disingenuous arguments by mixing assertions that are true but irrelevant together with assertions that are probably wrong. For example we can break down the following firehose of assertions by the author: Performance is amazing ... but ... they make stupid mistakes Factual errors, logical errors, inconsistency, limited reasoning, toxicity... LLMs have no knowledge of the underlying reality They have no common sense & they can’t plan their answer Unpopular Opinion about AR-LLMs Auto-Regressive LLMs are doomed. They cannot be made factual, non-toxic, etc. They are not controllable > they make stupid mistakes OK maybe some make stupid mistakes, but it's clear that increasingly advanced GPT-N are making fewer of them. > Factual errors Raw LLMs are pure bullshitters but it turns out that facts usually make better bullshit (in its technical sense) than lies. So advanced GPT-N usually are more factual. Furthermore, raw GPT-4 (before reinforcement training) has excellent calibration of its certainty of its beliefs as shown in Figure 8 of the technical report, at least for multiple choice questions. > logical errors Same thing. More advanced ones make fewer logical errors, for whatever reason. It's an emergent property. > inconsistency Nothing about LLMs requires consistency just like nothing about human meaning and understanding requires consistency, but more advanced LLMs emergently give more coherent continuations. This is especially funny because the opposite argument used to be given for why robots will never be on the level of humans - robots are C3P0-like mega-dorks whose wiring will catch fire and circuit boards will explode if we ask them to follow two conflicting rules. > limited reasoning Of course their reasoning is limited. Our reasoning is limited too. Larger language models appear to have less-limited reasoning. > toxicity There is nothing saying that raw LLMs won't be toxic. Probably they will be, according to most definitions. That's why corporations lobomize them with human feedback reinforcement learning as a final 'polishing' step. Some humans are huge assholes too, but probably they have meaning and understanding anyway. > LLMs have no knowledge of the underlying reality OK fine you can say that any p-zombie has no knowledge of the underlying reality if you want, if that's your objection. Or maybe they are saying LLMs don't have pixel buffer visual or time series audio inputs. Does that mean when those are added (they have already been added) then LLMs can possibly get meaning and understanding? > They have no common sense If you say that inhuman automata are by definition incapable of common sense then sure they have no common sense. But if you are talking about testing for common sense, then GPT-N is unlocking a mindblowing amount of common sense as N is increasing. > they can’t plan their answer Probably they are saying this because of next-token-prediction which is tautologically true, in the same way that it's true that humans speak one word after another. But the implication is wrong. They can plan their answer in any sense that matters. > Auto-Regressive LLMs are doomed. OK. Do you mean in terms of technical capabilities, or in terms of societal acceptance? They are different things. Or do you mean they are doomed to never attain meaning and understanding? > They cannot be made factual, non-toxic, etc. They are not controllable. Those same criticisms can all be made against even the most human of humans. Does it mean humans have no meaning or understanding? No. Of course these are also conditional on whatever prompt you are putting to get them to answer questions. If you prompt an advanced raw GPT-N to make stupid mistakes and factual and logical errors and to act especially toxic then it will do it. And, perhaps, only then will it have truly attained meaning and understanding.

foobiekr 4y ago
I don’t think there is any evidence at all that they meaningfully make fewer mistakes. In addition while humans make mistakes they will not casually make contradictory claims the way GPT* do. Basic self-contradictions of the sort even a three year old would catch. Define how they are planning their answer in any sense that matters, please.
- ftxbro 4y ago
  > I don’t think there is any evidence at all that they meaningfully make fewer mistakes. I'm not sure I was clear, when I wrote "increasingly advanced GPT-N are making fewer [mistakes]" I didn't necessarily mean they were making fewer mistakes than humans, but rather that GPT N+1 makes fewer mistakes than GPT N. I assume this is pretty uncontroversial because for example the evidence in the test suites where bigger N generally get better scores meaning fewer mistakes. > In addition while humans make mistakes they will not casually make contradictory claims the way GPT* do. Basic self-contradictions of the sort even a three year old would catch. OK I believe you saw some of them. Do you think the contradictory claims imply that large language models need sensory grounding for meaning and understanding? If they are mistakes then presumably the more advanced models will have fewer mistakes, but this is only an extrapolation from the evidence of model progressions on test suites. The mistakes that 3 year olds would catch I would like to see, especially if GPT 4 fails them. I will be curious to see if GPT 5 will fail them also. If they are more like, taking opposing opinion stances in different query instances, then maybe it's not a problem that prevents meaning and understanding. Maybe it's just how the AI rolls. > Define how they are planning their answer in any sense that matters, please. Honestly idk what you expect me to put here. Probably you are saying that AIs that generate things word by word are by definition not planning, even if they might have a big context window. But like I was saying, people also speak word by word. Does it mean people aren't planning either? I mean, GPT-4 scores well on suites of puzzles. Presumably those suites involve some puzzles that require thinking about (planning) the solution to the puzzle and then writing the solution word by word. I thought of two simple ones from the top of my head. Probably you won't say they require any planning but I'm not sure. The first example I prompted gpt3.5 "Find two numbers you add together and get ten." and it put "Two numbers that add up to ten are 5 and 5." Does that answer require planning? Maybe or maybe not I guess it depends how you define it. The other one I made from the top of my head was to put the last line of a story first and have the gpt3.5 make a story that ends with that line. I prompted it like "Make a story that ends with the exact sentence 'And that's why we always plant the corn so early in the year.'" and it wrote a story. I don't know if you will say it required any planning or not. Probably not. I don't know what kind of evidence you would want. Anyway here is the story it made: Once upon a time, there was a farmer named John who lived in a small village. John was known for his exceptional farming skills and his ability to grow the best corn in the entire region. One year, John decided to experiment with planting his corn later in the season, hoping to yield a better harvest. As the days went by, John noticed that his corn was not growing as well as it usually did. The plants were smaller, and the ears of corn were not as big as they should be. John was worried that he had made a mistake by planting his corn later in the year. One day, John decided to visit his neighbor, who was also a farmer, to ask for advice. His neighbor told him that planting corn later in the season was not a good idea because the soil was not warm enough, and the corn would not have enough time to grow before the cold weather set in. John realized that he had made a mistake and decided to plant his corn early the following year. He prepared the soil, planted the seeds, and took care of the plants every day. As a result, his corn grew tall and strong, and he had a bountiful harvest. From that day on, John always planted his corn early in the year, and he never had a bad harvest again. He shared his story with other farmers in the village, and they all learned from his mistake. And that's why we always plant the corn so early in the year.

by ftxbro 4y ago

foobiekr 4y ago

I don’t think there is any evidence at all that they meaningfully make fewer mistakes. In addition while humans make mistakes they will not casually make contradictory claims the way GPT* do. Basic self-contradictions of the sort even a three year old would catch. Define how they are planning their answer in any sense that matters, please.

ftxbro 4y ago
> I don’t think there is any evidence at all that they meaningfully make fewer mistakes. I'm not sure I was clear, when I wrote "increasingly advanced GPT-N are making fewer [mistakes]" I didn't necessarily mean they were making fewer mistakes than humans, but rather that GPT N+1 makes fewer mistakes than GPT N. I assume this is pretty uncontroversial because for example the evidence in the test suites where bigger N generally get better scores meaning fewer mistakes. > In addition while humans make mistakes they will not casually make contradictory claims the way GPT* do. Basic self-contradictions of the sort even a three year old would catch. OK I believe you saw some of them. Do you think the contradictory claims imply that large language models need sensory grounding for meaning and understanding? If they are mistakes then presumably the more advanced models will have fewer mistakes, but this is only an extrapolation from the evidence of model progressions on test suites. The mistakes that 3 year olds would catch I would like to see, especially if GPT 4 fails them. I will be curious to see if GPT 5 will fail them also. If they are more like, taking opposing opinion stances in different query instances, then maybe it's not a problem that prevents meaning and understanding. Maybe it's just how the AI rolls. > Define how they are planning their answer in any sense that matters, please. Honestly idk what you expect me to put here. Probably you are saying that AIs that generate things word by word are by definition not planning, even if they might have a big context window. But like I was saying, people also speak word by word. Does it mean people aren't planning either? I mean, GPT-4 scores well on suites of puzzles. Presumably those suites involve some puzzles that require thinking about (planning) the solution to the puzzle and then writing the solution word by word. I thought of two simple ones from the top of my head. Probably you won't say they require any planning but I'm not sure. The first example I prompted gpt3.5 "Find two numbers you add together and get ten." and it put "Two numbers that add up to ten are 5 and 5." Does that answer require planning? Maybe or maybe not I guess it depends how you define it. The other one I made from the top of my head was to put the last line of a story first and have the gpt3.5 make a story that ends with that line. I prompted it like "Make a story that ends with the exact sentence 'And that's why we always plant the corn so early in the year.'" and it wrote a story. I don't know if you will say it required any planning or not. Probably not. I don't know what kind of evidence you would want. Anyway here is the story it made: Once upon a time, there was a farmer named John who lived in a small village. John was known for his exceptional farming skills and his ability to grow the best corn in the entire region. One year, John decided to experiment with planting his corn later in the season, hoping to yield a better harvest. As the days went by, John noticed that his corn was not growing as well as it usually did. The plants were smaller, and the ears of corn were not as big as they should be. John was worried that he had made a mistake by planting his corn later in the year. One day, John decided to visit his neighbor, who was also a farmer, to ask for advice. His neighbor told him that planting corn later in the season was not a good idea because the soil was not warm enough, and the corn would not have enough time to grow before the cold weather set in. John realized that he had made a mistake and decided to plant his corn early the following year. He prepared the soil, planted the seeds, and took care of the plants every day. As a result, his corn grew tall and strong, and he had a bountiful harvest. From that day on, John always planted his corn early in the year, and he never had a bad harvest again. He shared his story with other farmers in the village, and they all learned from his mistake. And that's why we always plant the corn so early in the year.