4 ms·

The internet is sampling the interesting samples, not necessarily a realistic picture. I'd love to see a good research study on this that shows the actual erro

by spion 3y ago

The internet is sampling the interesting samples, not necessarily a realistic picture. I'd love to see a good research study on this that shows the actual error rate as well as a comparison with other non-human alternatives (e.g. googling, using textbook only, etc) as well as possibly human (personal tutor, group instructor, ...)

Alupis 3y ago
> The internet is sampling the interesting samples, not necessarily a realistic picture. A tutor is expected to know the subject and guide the student. If, say, 10% of the time it guides the student into a false understanding, the damages are significant. It's very hard to unlearn something, particularly when you have confidence you know it well. My personal adventures with ChatGPT are probably close to a 50% success rate. It gets some stuff entirely wrong, a lot of stuff non-obviously wrong, and even more stuff subtly wrong - and it's up to you to be knowledgeable enough to wade through the BS. Students, learning a subject in school are by definition not knowledgeable enough to discern confident BS from correctness. Will ChatGPT be useful in the future? Yes, almost certainly. But let's not rush this and get it very wrong. The consequences can be staggering in the education space - children or adults.
- spion 3y ago
  I'm getting north of 90% success with GPT4, and while a dedicated tutor or a group instructor would definitely be better, none of the other non-human alternatives come close. Searching the internet and youtube tutorials can also lead to wrong information and false understanding - all self-directed methods have this pitfall. ChatGPT, however, is the only one where you can probe deeper once you find problems. If I had to place it somewhere, it would be between a study buddy and a tutor, closer to a study buddy. Still - a well designed study will give us a much better picture of where we actually are. I think that would be extremely valuable.

by spion 3y ago

Alupis 3y ago

> The internet is sampling the interesting samples, not necessarily a realistic picture. A tutor is expected to know the subject and guide the student. If, say, 10% of the time it guides the student into a false understanding, the damages are significant. It's very hard to unlearn something, particularly when you have confidence you know it well. My personal adventures with ChatGPT are probably close to a 50% success rate. It gets some stuff entirely wrong, a lot of stuff non-obviously wrong, and even more stuff subtly wrong - and it's up to you to be knowledgeable enough to wade through the BS. Students, learning a subject in school are by definition not knowledgeable enough to discern confident BS from correctness. Will ChatGPT be useful in the future? Yes, almost certainly. But let's not rush this and get it very wrong. The consequences can be staggering in the education space - children or adults.

spion 3y ago
I'm getting north of 90% success with GPT4, and while a dedicated tutor or a group instructor would definitely be better, none of the other non-human alternatives come close. Searching the internet and youtube tutorials can also lead to wrong information and false understanding - all self-directed methods have this pitfall. ChatGPT, however, is the only one where you can probe deeper once you find problems. If I had to place it somewhere, it would be between a study buddy and a tutor, closer to a study buddy. Still - a well designed study will give us a much better picture of where we actually are. I think that would be extremely valuable.