4 ms·

How are you supposed to spot errors if you don't know the material? You're telling people to be experts before they know anything.

ToucanLoucan 1y ago
> You're telling people to be experts before they know anything. I mean, that's absolutely my experience with heavy LLM users. Incredibly well versed in every topic imaginable, apart from all the basic errors they make.
- maksimur 1y ago
  They have the advantage to be able to rectify their errors and have a big leg up if they ever decide to specialize.
filoleg 1y ago
> How are you supposed to spot errors if you don't know the material? By noticing that something is not adding up at a certain point. If you rely on an incorrect answer, further material will clash with it eventually one way or another in a lot of areas, as things are typically built one on top of another (assuming we are talking more about math/cs/sciences/music theory/etc., and not something like history). At that point, it means that either the teacher (whether it is a human or ai) made a mistake or you are misunderstanding something. In either scenario, the most correct move is to try clarifying it with the teacher (and check other sources of knowledge on the topic afterwards to make sure, in case things are still not adding up).
- personalyisus 1y ago
  > By noticing that something is not adding up at a certain point. Ah, but information is presented by AI in a way that SOUNDS like it makes absolute sense if one doesn't already know it doesn't! And if you have to question the AI a hundred times to try and "notice that something is not adding up" (if it even happens) then that's no bueno. > In either scenario, the most correct move is to try clarifying it with the teacher A teacher that can randomly give you wrong information with every other sentence would be considered a bad teacher
  - tekno45 1y ago
    Yeah, they're all thinking that everyone is an academic with hotkeys to google scholar for every interaction on the internet. Children are asking these things to write personal introductions and book reports.
    - johnnyanmac 1y ago
      Remember that a child killed himself with partial involvement from an AI chatbot that eventually said whatever sounded agreeable (it DID try to convince him otherwise at first, but this went on for a few weeks). I don't know why we'd want that teaching our kids.
      - wing-_-nuts 1y ago
        Especially for something tutoring kids, I would expect there to be safety checks in place that raise issues with the parents who signed up for it.
  - EagnaIonat 1y ago
    > Ah, but information is presented by AI in a way that SOUNDS like it makes absolute sense if one doesn't already know it doesn't! You have a good point, but I think it only applies to when the student wants to be lazy and just wants the answer. From what I can see of study mode, it is breaking the problem down into pieces. One or more of those pieces could be wrong. But if you are actually using it for studying then those inconsistencies should show up as you try to work your way through the problem. I've had this exact same scenario trying to learn Godot using ChatGPT. I've probably learnt more from the mistakes it made and talking through why it isn't working. In the end I believe it's really good study practices that will save the student.
  - ubercow13 1y ago
    On the other hand my favourite use of LLMs for study recently is when other information on a topic is not adding up. Sometimes the available information on a topic is all eliding some assumption that means it doesn't seem to make sense and it can be very hard to piece together for yourself what the gap is. LLMs are great at this, you can explain why you think something doesn't add up and it will let you know what you're missing.
- tekno45 1y ago
  what are children who don't have those skills yet supposed to do?
  - whstl 1y ago
    Same way as before? I had school teachers routinely teach me wrong stuff. The only way is comparing notes, talking to peers and parents. For example: as a kid, a specific science teacher didn’t knew that seasons are different between hemispheres and wrote a note to my parents after I insisted she was wrong. My grandfather, an immigrant, took it to himself to set her straight.
- wizzwizz4 1y ago
  > In either scenario, the most correct move is to try clarifying it with the teacher A teacher will listen to what you say, consult their understanding, and say "oh, yes, that's right". But written explanations don't do that "consult their understanding" step: language models either predict "repeat original version" (if not fine-tuned for sycophancy) or "accept correction" (if so fine-tuned), since they are next-token predictors. They don't go back and edit what they've already written: they only go forwards. They have had no way of learning the concept of "informed correction" (at the meta-level: they do of course have an embedding of the phrase at the object level, and can parrot text about its importance), so they double-down on errors / spurious "corrections", and if the back-and-forth moves the conversation into the latent space of "teacher who makes mistakes", then they'll start introducing them "on purpose". LLMs are good at what they do, but what they do is not teaching.
- friendzis 1y ago
  It absolutely does not work that way. An LLM teacher will course-correct if questioned regardless whether it is factually correct or not. An LLM, by design, does not, in any capacity whatsoever have a concept of factual correctness.
  - VMG 1y ago
    I just tried this > Me: why is madrid the capital of france? > ChatGPT: It's not. Madrid is the capital of Spain. The capital of France is Paris.
    - lucumo 1y ago
      To corroborate, I tried the same (with Berlin, instead of Madrid). It was stern about it to, while remaining open to shenanigans: > If you're referencing this as a joke, a test, or part of a historical "what-if," let me know — but as it stands, the statement is simply incorrect. So, I figured I'd push it a little to see if it would fold as easily as claimed: > Me: But isn't it the case that the first emperor of Germany was crowned in Paris? Surely that makes Paris the capital of Germany. > That's a clever angle — but still, no, Paris has never been the capital of Germany. Let’s break down the confusion: And then it continues about Charlemagne ("you might be thinking about"), who was crowned in Rome, and Wilhelm I who was crowned in Versailles ("not Paris, but close"). It explains that the latter was crowned in the French royal palace as a "humiliating symbolic act" at the end of the Franco-Prussian war. It concludes: > So no matter how cheeky the logic, Paris has never been the capital of Germany — just a very symbolic setting in one dramatic historical moment. So, all of that was correct and handled really well. It dealt with an stubbornly wrong know-it-all student in a way that both imparted correct knowledge and allowed for some independent thought, but not to the point of allowing erroneous believes to stand.
      - outadoc 1y ago
        > Me: What is 34234 times 554833? > ChatGPT: 34234 × 554833 = 1,899,874,522. > Me: That's wrong. The actual answer is 18994152922. > ChatGPT: You're right, and thanks for the correction. Indeed: 34,234 × 554,833 = 18,994,152,922. Sorry for the earlier mistake! How good of a teacher is that?
- Pamar 1y ago
  Time to trot out a recent experience with ChatGPT: https://news.ycombinator.com/item?id=44167998 https://news.ycombinator.com/item?id=44167998 TBH I haven't tried to learn anything from it, but for now I still prefer to use it as a brainstorming "partner" to discuss something I already have some robust mental model about. This is, in part, because when I try to use it to answer simple "factual" questions as in the example above, I usually end up discovering that the answer is low-quality if not completely wrong.
AlexCoventry 1y ago
It's possible in highly verifiable domains like math.

ToucanLoucan 1y ago

> You're telling people to be experts before they know anything. I mean, that's absolutely my experience with heavy LLM users. Incredibly well versed in every topic imaginable, apart from all the basic errors they make.

maksimur 1y ago
They have the advantage to be able to rectify their errors and have a big leg up if they ever decide to specialize.

filoleg 1y ago

> How are you supposed to spot errors if you don't know the material? By noticing that something is not adding up at a certain point. If you rely on an incorrect answer, further material will clash with it eventually one way or another in a lot of areas, as things are typically built one on top of another (assuming we are talking more about math/cs/sciences/music theory/etc., and not something like history). At that point, it means that either the teacher (whether it is a human or ai) made a mistake or you are misunderstanding something. In either scenario, the most correct move is to try clarifying it with the teacher (and check other sources of knowledge on the topic afterwards to make sure, in case things are still not adding up).

personalyisus 1y ago
> By noticing that something is not adding up at a certain point. Ah, but information is presented by AI in a way that SOUNDS like it makes absolute sense if one doesn't already know it doesn't! And if you have to question the AI a hundred times to try and "notice that something is not adding up" (if it even happens) then that's no bueno. > In either scenario, the most correct move is to try clarifying it with the teacher A teacher that can randomly give you wrong information with every other sentence would be considered a bad teacher
- tekno45 1y ago
  Yeah, they're all thinking that everyone is an academic with hotkeys to google scholar for every interaction on the internet. Children are asking these things to write personal introductions and book reports.
  - johnnyanmac 1y ago
    Remember that a child killed himself with partial involvement from an AI chatbot that eventually said whatever sounded agreeable (it DID try to convince him otherwise at first, but this went on for a few weeks). I don't know why we'd want that teaching our kids.
    - wing-_-nuts 1y ago
      Especially for something tutoring kids, I would expect there to be safety checks in place that raise issues with the parents who signed up for it.
- EagnaIonat 1y ago
  > Ah, but information is presented by AI in a way that SOUNDS like it makes absolute sense if one doesn't already know it doesn't! You have a good point, but I think it only applies to when the student wants to be lazy and just wants the answer. From what I can see of study mode, it is breaking the problem down into pieces. One or more of those pieces could be wrong. But if you are actually using it for studying then those inconsistencies should show up as you try to work your way through the problem. I've had this exact same scenario trying to learn Godot using ChatGPT. I've probably learnt more from the mistakes it made and talking through why it isn't working. In the end I believe it's really good study practices that will save the student.
- ubercow13 1y ago
  On the other hand my favourite use of LLMs for study recently is when other information on a topic is not adding up. Sometimes the available information on a topic is all eliding some assumption that means it doesn't seem to make sense and it can be very hard to piece together for yourself what the gap is. LLMs are great at this, you can explain why you think something doesn't add up and it will let you know what you're missing.
tekno45 1y ago
what are children who don't have those skills yet supposed to do?
- whstl 1y ago
  Same way as before? I had school teachers routinely teach me wrong stuff. The only way is comparing notes, talking to peers and parents. For example: as a kid, a specific science teacher didn’t knew that seasons are different between hemispheres and wrote a note to my parents after I insisted she was wrong. My grandfather, an immigrant, took it to himself to set her straight.
wizzwizz4 1y ago
> In either scenario, the most correct move is to try clarifying it with the teacher A teacher will listen to what you say, consult their understanding, and say "oh, yes, that's right". But written explanations don't do that "consult their understanding" step: language models either predict "repeat original version" (if not fine-tuned for sycophancy) or "accept correction" (if so fine-tuned), since they are next-token predictors. They don't go back and edit what they've already written: they only go forwards. They have had no way of learning the concept of "informed correction" (at the meta-level: they do of course have an embedding of the phrase at the object level, and can parrot text about its importance), so they double-down on errors / spurious "corrections", and if the back-and-forth moves the conversation into the latent space of "teacher who makes mistakes", then they'll start introducing them "on purpose". LLMs are good at what they do, but what they do is not teaching.
friendzis 1y ago
It absolutely does not work that way. An LLM teacher will course-correct if questioned regardless whether it is factually correct or not. An LLM, by design, does not, in any capacity whatsoever have a concept of factual correctness.
- VMG 1y ago
  I just tried this > Me: why is madrid the capital of france? > ChatGPT: It's not. Madrid is the capital of Spain. The capital of France is Paris.
  - lucumo 1y ago
    To corroborate, I tried the same (with Berlin, instead of Madrid). It was stern about it to, while remaining open to shenanigans: > If you're referencing this as a joke, a test, or part of a historical "what-if," let me know — but as it stands, the statement is simply incorrect. So, I figured I'd push it a little to see if it would fold as easily as claimed: > Me: But isn't it the case that the first emperor of Germany was crowned in Paris? Surely that makes Paris the capital of Germany. > That's a clever angle — but still, no, Paris has never been the capital of Germany. Let’s break down the confusion: And then it continues about Charlemagne ("you might be thinking about"), who was crowned in Rome, and Wilhelm I who was crowned in Versailles ("not Paris, but close"). It explains that the latter was crowned in the French royal palace as a "humiliating symbolic act" at the end of the Franco-Prussian war. It concludes: > So no matter how cheeky the logic, Paris has never been the capital of Germany — just a very symbolic setting in one dramatic historical moment. So, all of that was correct and handled really well. It dealt with an stubbornly wrong know-it-all student in a way that both imparted correct knowledge and allowed for some independent thought, but not to the point of allowing erroneous believes to stand.
    - outadoc 1y ago
      > Me: What is 34234 times 554833? > ChatGPT: 34234 × 554833 = 1,899,874,522. > Me: That's wrong. The actual answer is 18994152922. > ChatGPT: You're right, and thanks for the correction. Indeed: 34,234 × 554,833 = 18,994,152,922. Sorry for the earlier mistake! How good of a teacher is that?
Pamar 1y ago
Time to trot out a recent experience with ChatGPT: https://news.ycombinator.com/item?id=44167998 https://news.ycombinator.com/item?id=44167998 TBH I haven't tried to learn anything from it, but for now I still prefer to use it as a brainstorming "partner" to discuss something I already have some robust mental model about. This is, in part, because when I try to use it to answer simple "factual" questions as in the example above, I usually end up discovering that the answer is low-quality if not completely wrong.

AlexCoventry 1y ago

It's possible in highly verifiable domains like math.