10 ms·

I won't take a strong stance on whether or not LLMs actually do reasoning, but I will say that this decrease in performance is similar to what I see in college

by parsimo2010 2y ago

I won't take a strong stance on whether or not LLMs actually do reasoning, but I will say that this decrease in performance is similar to what I see in college freshmen (I'm currently teaching a calculus course in which almost half of the students took AP calc in high school). They perform well on simple questions. Requiring students to chain multiple steps together, even simple steps, results in decreased accuracy and higher variance (I have no data on whether this decrease is linear or not, as the paper assumes that the decrease should be linear with the number of steps). We see similar results with adding unrelated statements into a problem- many students are trained to make sure to use all given information in solving a problem- if you leave out something that the instructor gives you, then you probably forgot to do something important. So while I don't take a stance on what an LLM does should be considered reasoning, I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence. In other words, average Americans exhibit similar limitations on their reasoning as good LLMs. Which on the one hand is a little disappointing to me in terms of the human performance but is kind of good news for LLMs- they aren't doing graduate-level research but they are already capable of helping a large portion of the population.

skydhash 2y ago
Not to disparage American school system (my country’s is worse) but it’s very much easy mode. I know that not everyone is suited to academic excellence, but it’s definitely easier to learn when young. I do believe too much hand holding actively harm learning.
- BriggyDwiggs42 2y ago
  I don’t think the issue with American schools is that there’s too much hand holding. If anything, it’s the opposite; teachers at drastically underfunded schools don’t have any time to help the students of their 50 person class through the confused curriculum.
  - skydhash 2y ago
    Here, we have to go through 4 state exams just to get to university. The first when you’re 11, the second at 14, then two consecutive ones at 17 and 18. There’s a national curriculum that the exams will be about, although the schools are free to add to it. So however you feel about the school or the teacher, you have to master the subjects enough to go through. And that means paying attention in class, cram before it, or hoping you can cheat. We have our own problem too, but the consensus among all the people I know that have moved to the US is that classes are easy there. Not a bad thing per se (better explanation, better understanding instead of rote memorizing).
    - BriggyDwiggs42 2y ago
      Yeah you’ll never convince me that one-time exams are a good system to determine the rest of a student’s life, but I don’t disagree our schools are much easier. I’m just arguing the issue is less on a behavior of teachers level and more on a funding and incentives level. If I recall correctly, one issue is that schools are incentivized to lower educational standards to prevent students from repeating grades so that that receive more funding.
  - exoverito 2y ago
    Baltimore would be a counterexample. They spend $22k per student, with a student-teacher ratio of 15 to 1. This still results in remarkably poor performance, with only 8% of students proficient in math and 22% in reading. Culture and genetics would be next obvious explanations.
    - mdp2021 2y ago
      > obvious explanations I'd want to assess a few lessons first.
    - BriggyDwiggs42 2y ago
      Genetics lmao come on man.
    - imtringued 2y ago
- hintymad 2y ago
  > Not to disparage American school system (my country’s is worse) but it’s very much easy mode I used to be very upset about how low the bar of the US school has when it comes to STEM subjects. There was a meme that contrasted the difference between maths in 1970s and 2010s. In the meme kids used to learn how to find the area of an irregular shape, while now the kids are asked to color a regular shape. But then I made peace, as I realized that the US people simply didn't think that it was that important to push everyone to be good at STEM -- just some level of general understanding is good enough. To most people, the level of STEM as in IIT's JEE or in various national entrance exams in Eastern European countries is for elite students. The US school systems would rather have kids spend more time on sports, on ECs, on APs of kids' own choices, and etc. That's really just different trade offs. For parents like me, that means I don't have to worry about ECs, but I'll have to find tutors, serious tutoring schools like AOPS, and private teachers for STEM subjects. Or if my kids are truly talented, I'll guide them to find the right study groups, summer camps, and college courses. I used to feel pain as I believed that the students in the middle, which were the majority, would be left behind. But I realized, especially after I've got kids, that the majority of the students were not into STEM anyway. If they had a choice, they'd rather spend time watching YouTube channels and hang out with their friends.
debit-freak 2y ago
> In other words, average Americans exhibit similar limitations on their reasoning as good LLMs. It's not even clear this is a good example of "reasoning". You can progress all the way through multi-variable calculus with just decent pattern-matching, variable-substitution, and rote memorization of sufficient lists of rules. I imagine for "reasoning" ability to apply you need to be able to detect incoherency and reject an approach—and incoherency detection seems to be a big missing ingredient right now (...which many humans lack, too!). On the other side—any such ability would cripple a chatbot's ability to answer questions about the real world as our world is characterized (via description with informal language) by incoherent and contradictory concepts that can only be resolved through good-faith interpretation of the questioner. A large mark of intelligence (in the colloquial sense, not the IQ sense) is the ability to navigate both worlds.
richerram 2y ago
This, it is like when I hear interviews of PHDs talking about AI and they mention something like "AI will be smarter than humans", I am like "really?, where have you been all this time?, do you smart people ever leave your labs and go see the real world?, LLMs are already smarter that the huge majority of Humans in this planet, what are you talking about?"
- zeroonetwothree 2y ago
  This must be some bizarre definition of “smarter”.
  - kkzz99 2y ago
    I don't think you know how "smart" the average human is.
  - richerram 2y ago
    I know it won't be in the holistic or deep philosophical way but even by just predicting the next token without any real sense of the real world LLMs are already capable of simulating basic reasoning which a lot of people lacks, I mean even the 7B llama2 model can tell you the Earth ain't flat... go figure.
- lupire 2y ago
  Can an AI walk and chew gum at the same time?
  - lukeschlather 2y ago
    I think the answer to this question might actually be yes, but I think there are plenty of things humans can do while walking that AI can't do at all. At least, not yet.
- goatlover 2y ago
  Smarter than people in generating text, or smarter in oerforming all the other things people do as they go about their lives?
ActorNightly 2y ago
> I won't take a strong stance on whether or not LLMs actually do reasoning, I don't understand why people are still confused about this. When these models fundamentally have a randomness parameter to make them appear like they are actually thinking instead of deterministically outputting information, it should be clear that there is no reasoning going on.
- kkzz99 2y ago
  "deterministally outputting information" neither do humans.
- growthwtf 2y ago
  I don't see how the latter follows from the former. Here's how I think about it: the fact that it can interpret the same words differently in different contexts alone shows that even on a temperature of 0 (i.e., lowest randomness possible) there could be something that possibly resembles reasoning happening. It might be a mimicry of reasoning, but I don't think that having adjustable parameters on how random they are makes it any less of one. I also don't see how that idea would fit in with the o1 models, which explicitly have "reasoning" tokens. Now, I'm not terribly impressed with their performance relative to how much extra computation they need to do, but the fact they have chains-of-thought that humans could reasonably inspect and interpret, and that they chains of thought do literally take extra time and compute to run, certainly points at the process being something possibly analogous to reasoning. In this same vein, up until recently I personally very much in the camp of calling them "LLMs" and generally still do, but given how they really are being used now as general purpose sequence-to-sequence prediction models across all sorts of input and output types tends to push me more towards the "foundation models" terminology camp, since pigeonholing them into just language tasks doesn't seem accurate anymore. o1 was the turning point for me on this personally, since it is explicitly predicting and being optimized for correctness in the "reasoning tokens" (in scare quotes again since that's what openai calls it). All that said, I personally think that calling what they do reasoning, and meaning it in the exact same way as how humans reason, is anthropomorphizing the models in a way that's not really useful. They clearly operate in ways that are quite different from humans in many ways. Sometimes that might imitate human reasoning, other times it doesn't. But, the fact they have that randomness parameter seems to be to be totally unrelated to any of the above thoughts or merits about the models having reasoning abilities.
mdp2021 2y ago
> Which on the one hand is a little disappointing to me in terms of the human performance but is kind of good news for LLMs Here's the recurrent reminder that we build tools (calculators, cranes etc.) to outperform the strong, not the weak.
- deleted 2y ago
  [deleted]
gosub100 2y ago
> They perform well on simple questions. Requiring students to chain multiple steps together, even simple steps, results in decreased accuracy and higher variance you mean when you give lessons and homework problems of the form (A) -> (B), but then on test-day you give them completely different problems? "Given D, which (A,B, C) is required to produce it?". Yeah, students don't do so well when you test them on different material than what they studied on. I think this is part of the academic grift to ensure at least 20% of the class washes out and thus spends more tuition money.
hintymad 2y ago
> I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence. Is this because the questions used in high school exams in the US are too simple, or do they have too similar patterns in the training data? I tried really simple but novel questions that required true understanding of the underlying math concepts, and the results were consistently bad. I also tried questions at the level of entrance exams of high school in China, and the results were equally bad. It was quite clear that LLM didn't understand math. It could match some patterns, but such pattern match could be useful to only skilled students.
- MVissers 2y ago
  Which model? The field moves so fast it’s hard to validate statements like this without that info. O1-preview?
  - hintymad 2y ago
    GPT-4o. I tried only a few samples on o1-preview, and the results were bad. That did not have any statistical significance, though
    - asey 2y ago
      Could you give an example?
wkirby 2y ago
> I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence. This might be true in a strict sense, but I think it's really, really important to consider the uses of LLMs vs a high-school graduate. LLMs are confidently wrong (and confidently correct) with the exact same measure, and in many ways they are presented to users as unimpeachable. If I ask an average person to do a medium-complex logic problem, my human brain discounts their answer because I've been socialized to believe that humans are bad at logic. I will take any answer I'm given with usually appropriate skepticism. LLMs, on the other hand, are on the computer: an interface I've been socialized to believe is always correct on matters of math and logic. That's what it is, a logic machine. Second guessing the computer on matters of logic and arithmetic almost always result in me realizing my puny human mind has done something wrong. To me, this directly contradicts your conclusion: LLMs are mostly only capable of misleading large portions of the population.
- pishpash 2y ago
  Would be good to put equivalent grades on LLM's then. Instead of GPT-4o, it's GPT-11th grade.
  - 1oooqooq 2y ago
    that's in the wrong directing of what the comment said
  - steveBK123 2y ago
    The problem is that some % of the time they answer at an 11th grade level, and some % of the time they answer at a 4th grade level, with no way of knowing beforehand. The only way of knowing is after the fact, when the prompter has the knowledge to discern whether the answer is correct or not. In which case why are we asking questions we already know the answer of, other than these are toys and fun? I really wonder what happen when the novelty wears off in a few years, where these things are actually useful.
- Eisenstein 2y ago
  This is not inherent in the LLM though. Society will adjust to it after learning some very predictable (and predicted) lessons, just like it always does.
ojosilva 2y ago
LLM gets things right, when it does, due to the sheer massive information ingested during training, it can use probabilities to extract a right answer from deep in the model. Humans on the other hand have developed a more elaborate scheme to process, or reason, data without having to read through 1 billion math problems and stack overflow answers. We listen to some explanations, a YT video, a few exercises and we're ready to go. The fact that we may get similar grades (at ie high school math) is just a spot coincidence of where both "species" (AI x Human) are right now at succeeding. But if we look closer at failure, we'll see that we fail very differently. AI failure right now looks, to us humans, very nonsensical.
- pishpash 2y ago
  Nah, human failures look equally nonsensical. You're just more attuned to use their body language or peer judgement to augment your reception. Really psychotic humans can bypass this check.
- ben_w 2y ago
  While I'd agree human failures are different from AI failures, human failures are necessarily also nonsensical. Familiar, human, but nonsensical — consider how often a human disagreeing with another will use the phrase "that's just common sense!" I think the larger models are consuming in the order of 100k as much as we do, and while they have a much broader range of knowledge, it's not 100k as much breadth.
  - steveBK123 2y ago
    Well it's a breadth & depth problem isn't it? Humans are nonsensical, but in somewhat predictable error rates by domain, per individual. So you hire people with the skillsets, domain expertise, and error rates you need. With an LLM, it's nonsensical in a completely random way prompt to prompt. It's sort of like talking into a telephone and sometimes Einstein is on the other end, and sometimes it's a drunken child. You have no idea when you pick up the phone which way its going to go. We feed these things nearly the entirety of human knowledge, and the output still feels rather random. LLMs have all that information and then still have a ~10% chance of messing up simple mathematical comparison that an average 12 year old would not. Other times we delegate much more complex tasks to LLMs and they work great! But given the nondeterminism it becomes hard to delegate tasks you can't check the work of, if it is important.
elicksaur 2y ago
>So while I don't take a stance on what an LLM does should be considered reasoning >I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence This is taking a stance.
fhe 2y ago
if your experience is coming from teaching college freshmen, then that's a sample that's significantly above average among high school graduates. I think only about 1/2 of all high school graduates go on to further their education, and that includes community colleges. and I agree with your assessment -- while it's true that in a long conversation, chatgpt veers off and doesn't keep a coherent line of thought, it is not noticeably worse than the average conversation I have with people.
FabHK 2y ago
Are college students more likely to get it wrong when you change the numbers from the example problem (as reported here for LLMs)?
- sdenton4 2y ago
  You can absolutely psych students out by adding weird numbers to a problem, yes.
vasilipupkin 2y ago
I think it's an absurd question in some sense LLMs perform maximization of conditional probability of the next word being correct. Suppose they get to the point where they do that with 100% accuracy. How can you tell the difference between that and "Reasoning"? You can't. So then the question of whether they are "Reasoning" or not is religious, not quantitative.

skydhash 2y ago

Not to disparage American school system (my country’s is worse) but it’s very much easy mode. I know that not everyone is suited to academic excellence, but it’s definitely easier to learn when young. I do believe too much hand holding actively harm learning.

BriggyDwiggs42 2y ago
I don’t think the issue with American schools is that there’s too much hand holding. If anything, it’s the opposite; teachers at drastically underfunded schools don’t have any time to help the students of their 50 person class through the confused curriculum.
- skydhash 2y ago
  Here, we have to go through 4 state exams just to get to university. The first when you’re 11, the second at 14, then two consecutive ones at 17 and 18. There’s a national curriculum that the exams will be about, although the schools are free to add to it. So however you feel about the school or the teacher, you have to master the subjects enough to go through. And that means paying attention in class, cram before it, or hoping you can cheat. We have our own problem too, but the consensus among all the people I know that have moved to the US is that classes are easy there. Not a bad thing per se (better explanation, better understanding instead of rote memorizing).
  - BriggyDwiggs42 2y ago
    Yeah you’ll never convince me that one-time exams are a good system to determine the rest of a student’s life, but I don’t disagree our schools are much easier. I’m just arguing the issue is less on a behavior of teachers level and more on a funding and incentives level. If I recall correctly, one issue is that schools are incentivized to lower educational standards to prevent students from repeating grades so that that receive more funding.
- exoverito 2y ago
  Baltimore would be a counterexample. They spend $22k per student, with a student-teacher ratio of 15 to 1. This still results in remarkably poor performance, with only 8% of students proficient in math and 22% in reading. Culture and genetics would be next obvious explanations.
  - mdp2021 2y ago
    > obvious explanations I'd want to assess a few lessons first.
  - BriggyDwiggs42 2y ago
    Genetics lmao come on man.
  - imtringued 2y ago
hintymad 2y ago
> Not to disparage American school system (my country’s is worse) but it’s very much easy mode I used to be very upset about how low the bar of the US school has when it comes to STEM subjects. There was a meme that contrasted the difference between maths in 1970s and 2010s. In the meme kids used to learn how to find the area of an irregular shape, while now the kids are asked to color a regular shape. But then I made peace, as I realized that the US people simply didn't think that it was that important to push everyone to be good at STEM -- just some level of general understanding is good enough. To most people, the level of STEM as in IIT's JEE or in various national entrance exams in Eastern European countries is for elite students. The US school systems would rather have kids spend more time on sports, on ECs, on APs of kids' own choices, and etc. That's really just different trade offs. For parents like me, that means I don't have to worry about ECs, but I'll have to find tutors, serious tutoring schools like AOPS, and private teachers for STEM subjects. Or if my kids are truly talented, I'll guide them to find the right study groups, summer camps, and college courses. I used to feel pain as I believed that the students in the middle, which were the majority, would be left behind. But I realized, especially after I've got kids, that the majority of the students were not into STEM anyway. If they had a choice, they'd rather spend time watching YouTube channels and hang out with their friends.

debit-freak 2y ago

> In other words, average Americans exhibit similar limitations on their reasoning as good LLMs. It's not even clear this is a good example of "reasoning". You can progress all the way through multi-variable calculus with just decent pattern-matching, variable-substitution, and rote memorization of sufficient lists of rules. I imagine for "reasoning" ability to apply you need to be able to detect incoherency and reject an approach—and incoherency detection seems to be a big missing ingredient right now (...which many humans lack, too!). On the other side—any such ability would cripple a chatbot's ability to answer questions about the real world as our world is characterized (via description with informal language) by incoherent and contradictory concepts that can only be resolved through good-faith interpretation of the questioner. A large mark of intelligence (in the colloquial sense, not the IQ sense) is the ability to navigate both worlds.

richerram 2y ago

This, it is like when I hear interviews of PHDs talking about AI and they mention something like "AI will be smarter than humans", I am like "really?, where have you been all this time?, do you smart people ever leave your labs and go see the real world?, LLMs are already smarter that the huge majority of Humans in this planet, what are you talking about?"

zeroonetwothree 2y ago
This must be some bizarre definition of “smarter”.
- kkzz99 2y ago
  I don't think you know how "smart" the average human is.
- richerram 2y ago
  I know it won't be in the holistic or deep philosophical way but even by just predicting the next token without any real sense of the real world LLMs are already capable of simulating basic reasoning which a lot of people lacks, I mean even the 7B llama2 model can tell you the Earth ain't flat... go figure.
lupire 2y ago
Can an AI walk and chew gum at the same time?
- lukeschlather 2y ago
  I think the answer to this question might actually be yes, but I think there are plenty of things humans can do while walking that AI can't do at all. At least, not yet.
goatlover 2y ago
Smarter than people in generating text, or smarter in oerforming all the other things people do as they go about their lives?

ActorNightly 2y ago

> I won't take a strong stance on whether or not LLMs actually do reasoning, I don't understand why people are still confused about this. When these models fundamentally have a randomness parameter to make them appear like they are actually thinking instead of deterministically outputting information, it should be clear that there is no reasoning going on.

kkzz99 2y ago
"deterministally outputting information" neither do humans.
growthwtf 2y ago
I don't see how the latter follows from the former. Here's how I think about it: the fact that it can interpret the same words differently in different contexts alone shows that even on a temperature of 0 (i.e., lowest randomness possible) there could be something that possibly resembles reasoning happening. It might be a mimicry of reasoning, but I don't think that having adjustable parameters on how random they are makes it any less of one. I also don't see how that idea would fit in with the o1 models, which explicitly have "reasoning" tokens. Now, I'm not terribly impressed with their performance relative to how much extra computation they need to do, but the fact they have chains-of-thought that humans could reasonably inspect and interpret, and that they chains of thought do literally take extra time and compute to run, certainly points at the process being something possibly analogous to reasoning. In this same vein, up until recently I personally very much in the camp of calling them "LLMs" and generally still do, but given how they really are being used now as general purpose sequence-to-sequence prediction models across all sorts of input and output types tends to push me more towards the "foundation models" terminology camp, since pigeonholing them into just language tasks doesn't seem accurate anymore. o1 was the turning point for me on this personally, since it is explicitly predicting and being optimized for correctness in the "reasoning tokens" (in scare quotes again since that's what openai calls it). All that said, I personally think that calling what they do reasoning, and meaning it in the exact same way as how humans reason, is anthropomorphizing the models in a way that's not really useful. They clearly operate in ways that are quite different from humans in many ways. Sometimes that might imitate human reasoning, other times it doesn't. But, the fact they have that randomness parameter seems to be to be totally unrelated to any of the above thoughts or merits about the models having reasoning abilities.

mdp2021 2y ago

> Which on the one hand is a little disappointing to me in terms of the human performance but is kind of good news for LLMs Here's the recurrent reminder that we build tools (calculators, cranes etc.) to outperform the strong, not the weak.

deleted 2y ago
[deleted]

gosub100 2y ago

> They perform well on simple questions. Requiring students to chain multiple steps together, even simple steps, results in decreased accuracy and higher variance you mean when you give lessons and homework problems of the form (A) -> (B), but then on test-day you give them completely different problems? "Given D, which (A,B, C) is required to produce it?". Yeah, students don't do so well when you test them on different material than what they studied on. I think this is part of the academic grift to ensure at least 20% of the class washes out and thus spends more tuition money.

hintymad 2y ago

> I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence. Is this because the questions used in high school exams in the US are too simple, or do they have too similar patterns in the training data? I tried really simple but novel questions that required true understanding of the underlying math concepts, and the results were consistently bad. I also tried questions at the level of entrance exams of high school in China, and the results were equally bad. It was quite clear that LLM didn't understand math. It could match some patterns, but such pattern match could be useful to only skilled students.

MVissers 2y ago
Which model? The field moves so fast it’s hard to validate statements like this without that info. O1-preview?
- hintymad 2y ago
  GPT-4o. I tried only a few samples on o1-preview, and the results were bad. That did not have any statistical significance, though
  - asey 2y ago
    Could you give an example?

wkirby 2y ago

> I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence. This might be true in a strict sense, but I think it's really, really important to consider the uses of LLMs vs a high-school graduate. LLMs are confidently wrong (and confidently correct) with the exact same measure, and in many ways they are presented to users as unimpeachable. If I ask an average person to do a medium-complex logic problem, my human brain discounts their answer because I've been socialized to believe that humans are bad at logic. I will take any answer I'm given with usually appropriate skepticism. LLMs, on the other hand, are on the computer: an interface I've been socialized to believe is always correct on matters of math and logic. That's what it is, a logic machine. Second guessing the computer on matters of logic and arithmetic almost always result in me realizing my puny human mind has done something wrong. To me, this directly contradicts your conclusion: LLMs are mostly only capable of misleading large portions of the population.

pishpash 2y ago
Would be good to put equivalent grades on LLM's then. Instead of GPT-4o, it's GPT-11th grade.
- 1oooqooq 2y ago
  that's in the wrong directing of what the comment said
- steveBK123 2y ago
  The problem is that some % of the time they answer at an 11th grade level, and some % of the time they answer at a 4th grade level, with no way of knowing beforehand. The only way of knowing is after the fact, when the prompter has the knowledge to discern whether the answer is correct or not. In which case why are we asking questions we already know the answer of, other than these are toys and fun? I really wonder what happen when the novelty wears off in a few years, where these things are actually useful.
Eisenstein 2y ago
This is not inherent in the LLM though. Society will adjust to it after learning some very predictable (and predicted) lessons, just like it always does.

ojosilva 2y ago

LLM gets things right, when it does, due to the sheer massive information ingested during training, it can use probabilities to extract a right answer from deep in the model. Humans on the other hand have developed a more elaborate scheme to process, or reason, data without having to read through 1 billion math problems and stack overflow answers. We listen to some explanations, a YT video, a few exercises and we're ready to go. The fact that we may get similar grades (at ie high school math) is just a spot coincidence of where both "species" (AI x Human) are right now at succeeding. But if we look closer at failure, we'll see that we fail very differently. AI failure right now looks, to us humans, very nonsensical.

pishpash 2y ago
Nah, human failures look equally nonsensical. You're just more attuned to use their body language or peer judgement to augment your reception. Really psychotic humans can bypass this check.
ben_w 2y ago
While I'd agree human failures are different from AI failures, human failures are necessarily also nonsensical. Familiar, human, but nonsensical — consider how often a human disagreeing with another will use the phrase "that's just common sense!" I think the larger models are consuming in the order of 100k as much as we do, and while they have a much broader range of knowledge, it's not 100k as much breadth.
- steveBK123 2y ago
  Well it's a breadth & depth problem isn't it? Humans are nonsensical, but in somewhat predictable error rates by domain, per individual. So you hire people with the skillsets, domain expertise, and error rates you need. With an LLM, it's nonsensical in a completely random way prompt to prompt. It's sort of like talking into a telephone and sometimes Einstein is on the other end, and sometimes it's a drunken child. You have no idea when you pick up the phone which way its going to go. We feed these things nearly the entirety of human knowledge, and the output still feels rather random. LLMs have all that information and then still have a ~10% chance of messing up simple mathematical comparison that an average 12 year old would not. Other times we delegate much more complex tasks to LLMs and they work great! But given the nondeterminism it becomes hard to delegate tasks you can't check the work of, if it is important.

elicksaur 2y ago

>So while I don't take a stance on what an LLM does should be considered reasoning >I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence This is taking a stance.

fhe 2y ago

if your experience is coming from teaching college freshmen, then that's a sample that's significantly above average among high school graduates. I think only about 1/2 of all high school graduates go on to further their education, and that includes community colleges. and I agree with your assessment -- while it's true that in a long conversation, chatgpt veers off and doesn't keep a coherent line of thought, it is not noticeably worse than the average conversation I have with people.

FabHK 2y ago

Are college students more likely to get it wrong when you change the numbers from the example problem (as reported here for LLMs)?

sdenton4 2y ago
You can absolutely psych students out by adding weird numbers to a problem, yes.

vasilipupkin 2y ago

I think it's an absurd question in some sense LLMs perform maximization of conditional probability of the next word being correct. Suppose they get to the point where they do that with 100% accuracy. How can you tell the difference between that and "Reasoning"? You can't. So then the question of whether they are "Reasoning" or not is religious, not quantitative.