3 ms·

Human teachers make mistakes too - perhaps even at higher rates than GPT in some cases. Instead of isolating a single factor of hallucinations I think you have

Human teachers make mistakes too - perhaps even at higher rates than GPT in some cases. Instead of isolating a single factor of hallucinations I think you have to also consider the cost and experience quality and make a more holistic judgement on whether these models are good or bad.

davesque 4y ago
Yes, but human teachers can eventually come to understand why they made a mistake. And they probably are more able to communicate how confident they are in their answers. I think that's because human teachers have a mental model that they're working from. On the other hand, we have no idea if transformer models are working from anything like a "mental model." Just because humans make mistakes (or even as many or more mistakes than machines) doesn't mean the nature or consequences of the mistakes are the same.
- sebzim4500 4y ago
  >Just because humans make mistakes (or even as many or more mistakes than machines) doesn't mean the nature or consequences of the mistakes are the same. True, but this shouldn't be hard to measure. It's not like we will go from these AIs will go from teaching no one to replacing all teachers overnight. Give a random sample of classes access to a GPT-n powered tutor, and see if they do better or worse over the next few years.
  - davesque 4y ago
    > this shouldn't be hard to measure > see if they do better or worse over the next few years This doesn't strike me as entirely ethical. Also, it may be hard to gauge the long term effects of subtle misinformation generated by these models.
    - sebzim4500 4y ago
      Surely there are more stupid experiments about teaching kids than this. Give them all ipads Give them all chromebooks Take away their ipads Make them all stay home and shut down the economy for 6 months etc. Especially since this one has a decent chance of working
    - kbelder 4y ago
      I remember CS Lewis in "That Hideous Strength" talking about schools; he observed that no parent would let an organization experiment on their children, but that they can easily be convinced to send their kids to an experimental school, which in effect is the same. That said, you have to compare results from different strategies, or else you'll never improve or react to changing circumstances. I think it's like medical trials; you experiment, but be willing to stop the moment you get good evidence that one option is worse than another.
- sebasvisser 4y ago
  Human teachers realising their mistakes and learning from it… how long ago was it that you were in an average school?
  - aitchnyu 4y ago
    One teacher claimed he watched Animal Planet showing ostriches burying their head in the sand and humans casually slaughtering them.

davesque 4y ago

Yes, but human teachers can eventually come to understand why they made a mistake. And they probably are more able to communicate how confident they are in their answers. I think that's because human teachers have a mental model that they're working from. On the other hand, we have no idea if transformer models are working from anything like a "mental model." Just because humans make mistakes (or even as many or more mistakes than machines) doesn't mean the nature or consequences of the mistakes are the same.

sebzim4500 4y ago
>Just because humans make mistakes (or even as many or more mistakes than machines) doesn't mean the nature or consequences of the mistakes are the same. True, but this shouldn't be hard to measure. It's not like we will go from these AIs will go from teaching no one to replacing all teachers overnight. Give a random sample of classes access to a GPT-n powered tutor, and see if they do better or worse over the next few years.
- davesque 4y ago
  > this shouldn't be hard to measure > see if they do better or worse over the next few years This doesn't strike me as entirely ethical. Also, it may be hard to gauge the long term effects of subtle misinformation generated by these models.
  - sebzim4500 4y ago
    Surely there are more stupid experiments about teaching kids than this. Give them all ipads Give them all chromebooks Take away their ipads Make them all stay home and shut down the economy for 6 months etc. Especially since this one has a decent chance of working
  - kbelder 4y ago
    I remember CS Lewis in "That Hideous Strength" talking about schools; he observed that no parent would let an organization experiment on their children, but that they can easily be convinced to send their kids to an experimental school, which in effect is the same. That said, you have to compare results from different strategies, or else you'll never improve or react to changing circumstances. I think it's like medical trials; you experiment, but be willing to stop the moment you get good evidence that one option is worse than another.
sebasvisser 4y ago
Human teachers realising their mistakes and learning from it… how long ago was it that you were in an average school?
- aitchnyu 4y ago
  One teacher claimed he watched Animal Planet showing ostriches burying their head in the sand and humans casually slaughtering them.