5 ms·

The thing about LLMs as a teacher is that the feedback loop is so short that progress can be made quite fast. Being able to ask a question, regardless of how st

The thing about LLMs as a teacher is that the feedback loop is so short that progress can be made quite fast. Being able to ask a question, regardless of how stupid or trivial it may seem, without fear of judgement is a huge learning hurdle that is overcome. Not only that but reverse engineering other code “if I have this code and wanted it to do this instead - what and how would we change? Etc”, and a short feedback loop for errors where you might have to copy and paste something a lot before it works, makes LLMs the ultimate teachers.

smcleod 3y ago
Absolutely. Fail fast, fail often. It makes it very easy to quickly try out ideas before you go down a rabbit hole. However I think for people that are completely new to programming it could quickly become a crutch.
- jskherman 3y ago
  Seconded. And it's not just programming. Any subject that is easy to moderately verifiable like most Math and Science in school can also benefit from this. We're one step closer to personal tutors and autodidactism for everyone. The other half of the learning journey would be arranging the curriculum for yourself. If only more people would focus on cultivating learning instead of locking down tech like this because they're afraid (and perhaps lazy to create new teaching methods).
sfn42 3y ago
I don't know about this take. A lot of what you say makes sense but your conclusion seems like a bit of a stretch. LLMs could be very useful for learning and probably are, but they also allow you to ask the wrong questions and circumvent the learning you were supposed to be doing. I wouldn't be surprised to see people using it to simply solve exercises for them, and those people will have a hard time trying to do things on their own.
- aconsult1 3y ago
  It's actually worse than that because although LLMs are capable of a lot, they still hallucinate quite frequently and sometimes are just plain stupid. Recently I was trying to manipulate data in Google Sheets and was using ChatGPT to help. In the beginning is was fantastic, I was very productive because I didn't have to stop and think about formulas, read crappy documentation, or analyze data transformation. ChatGPT just gave me the right answer in a split second, as long as I kept asking the right questions. Then I stumbled upon a particular issue that wasn't really too complicated but ChatGPT could not give me a correct answer. Unknowingly I spent 3 days trying to fix problems with the solution and every time I got an answer that was slightly wrong, not in subtle ways. I have 20 years experience as a software engineer and still, I continued to waste time in this loop. After 3 days I decided to apply my engineering skills and solved the matter in 30 min. Now I know the solution and it was way simpler than I thought. What surprised me was how dumb the whole process was. My questions certainly weren't the problem - as bad as they possibly were, the solution wasn't too far off. Not only ChatGPT had become a crutch but it put me in a situation that no human-tutor would ever put me. So removing the pain of quick interactions with a tutor has benefits but the technology is not quite ready to be considered as true guidance in forming (or even helping) someone's understanding of a subject. I've been using ChatGPT a lot for general language but when it requires logical thinking it falls pretty flat.
  - PoignardAzur 3y ago
    I'm skeptical of this takeaway. "I spent days trying to solve X by doing Y, then it turned out I could have solved it by doing Z instead" is an experience I've had countless times before LLMs were a thing. Sometimes you really do need these three days of stumbling before you can build up the confidence to do the easy solution. (Then again, I don't know the specifics of your case.)
    - aconsult1 3y ago
      I can't really disagree with you. Although I want to believe this problem is more exacerbated with LLMs than with human-tutors. I have no evidence other than hundreds of hours using ChatGPT.
ramblerman 3y ago
I agree about short feedback loops, but this was there long before LLMs, and got me into programming initially. I love that I could compile my program and try it without needing to check it with a teacher or superior. It cost literally nothing to try, and if it failed the computer would tell you it was wrong. In a sense you are constantly bouncing off of reality (or the limits of the compiler) and that allows you to learn really quickly by experiment. LLMs with their tendency to hallucinate or just provide you with the solution seem like they could be both a blessing and a curse (for learning).
- danielbln 3y ago
  Providing the solution is one thing, but being able to tell you more about the solution, why it is a certain way, what other examples are, what to look out for, etc. having an infinite dialog about the problem space, that's where it really shines. Banging my head against compiler output and combing through Google to figure out _why_ it failed isn't nearly as productive, imo. Otherwise yes, hallucinations are an issue, though I find them not nearly as detrimental as I thought I would, then again I'm not learning to program, I'm merely learning new frameworks or techniques, I already have the underlying foundation and can sniff out hallucinations probably better than a total novice.

smcleod 3y ago

Absolutely. Fail fast, fail often. It makes it very easy to quickly try out ideas before you go down a rabbit hole. However I think for people that are completely new to programming it could quickly become a crutch.

jskherman 3y ago
Seconded. And it's not just programming. Any subject that is easy to moderately verifiable like most Math and Science in school can also benefit from this. We're one step closer to personal tutors and autodidactism for everyone. The other half of the learning journey would be arranging the curriculum for yourself. If only more people would focus on cultivating learning instead of locking down tech like this because they're afraid (and perhaps lazy to create new teaching methods).

sfn42 3y ago

I don't know about this take. A lot of what you say makes sense but your conclusion seems like a bit of a stretch. LLMs could be very useful for learning and probably are, but they also allow you to ask the wrong questions and circumvent the learning you were supposed to be doing. I wouldn't be surprised to see people using it to simply solve exercises for them, and those people will have a hard time trying to do things on their own.

aconsult1 3y ago
It's actually worse than that because although LLMs are capable of a lot, they still hallucinate quite frequently and sometimes are just plain stupid. Recently I was trying to manipulate data in Google Sheets and was using ChatGPT to help. In the beginning is was fantastic, I was very productive because I didn't have to stop and think about formulas, read crappy documentation, or analyze data transformation. ChatGPT just gave me the right answer in a split second, as long as I kept asking the right questions. Then I stumbled upon a particular issue that wasn't really too complicated but ChatGPT could not give me a correct answer. Unknowingly I spent 3 days trying to fix problems with the solution and every time I got an answer that was slightly wrong, not in subtle ways. I have 20 years experience as a software engineer and still, I continued to waste time in this loop. After 3 days I decided to apply my engineering skills and solved the matter in 30 min. Now I know the solution and it was way simpler than I thought. What surprised me was how dumb the whole process was. My questions certainly weren't the problem - as bad as they possibly were, the solution wasn't too far off. Not only ChatGPT had become a crutch but it put me in a situation that no human-tutor would ever put me. So removing the pain of quick interactions with a tutor has benefits but the technology is not quite ready to be considered as true guidance in forming (or even helping) someone's understanding of a subject. I've been using ChatGPT a lot for general language but when it requires logical thinking it falls pretty flat.
- PoignardAzur 3y ago
  I'm skeptical of this takeaway. "I spent days trying to solve X by doing Y, then it turned out I could have solved it by doing Z instead" is an experience I've had countless times before LLMs were a thing. Sometimes you really do need these three days of stumbling before you can build up the confidence to do the easy solution. (Then again, I don't know the specifics of your case.)
  - aconsult1 3y ago
    I can't really disagree with you. Although I want to believe this problem is more exacerbated with LLMs than with human-tutors. I have no evidence other than hundreds of hours using ChatGPT.

ramblerman 3y ago

I agree about short feedback loops, but this was there long before LLMs, and got me into programming initially. I love that I could compile my program and try it without needing to check it with a teacher or superior. It cost literally nothing to try, and if it failed the computer would tell you it was wrong. In a sense you are constantly bouncing off of reality (or the limits of the compiler) and that allows you to learn really quickly by experiment. LLMs with their tendency to hallucinate or just provide you with the solution seem like they could be both a blessing and a curse (for learning).

danielbln 3y ago
Providing the solution is one thing, but being able to tell you more about the solution, why it is a certain way, what other examples are, what to look out for, etc. having an infinite dialog about the problem space, that's where it really shines. Banging my head against compiler output and combing through Google to figure out _why_ it failed isn't nearly as productive, imo. Otherwise yes, hallucinations are an issue, though I find them not nearly as detrimental as I thought I would, then again I'm not learning to program, I'm merely learning new frameworks or techniques, I already have the underlying foundation and can sniff out hallucinations probably better than a total novice.