The authors hypothesize that ChatGPT is useful for > Tasks where it’s easy for the user to check if the bot’s answer is correct, such as debugging help. I wou

The authors hypothesize that ChatGPT is useful for > Tasks where it’s easy for the user to check if the bot’s answer is correct, such as debugging help. I would qualify this fairly heavily. If your bug is "my program is throwing an error" or "FizzBuzz isn't producing the expected output", sure, a bot suggestion can be tested easily. But that's only the easiest kind of debugging, and without any deep appreciation of logic, I suspect it would tend to give you overly specific remedies that mask the problem or make new problems. "You want it to say FizzBuzz when n=15? How about if n=15 print FizzBuzz" In other words, what can be easily tested for correctness depends very heavily on how strong a grasp the user has on correctness in the domain. A novice without a good bullshit detector could be left worse off than if they had asked a person. It's not clear that the set of problems whose solution can truly be checked easily is all that big or useful. (I think our CS professor friends here are overgeneralizing from intuitions about P vs NP, a totally different context.) My two cents: it will be mainly useful as a different kind of search engine that can help you remember syntax, what does what, what parameters this thing needs, etc.