3 ms·

I asked it to give me an outline for a course on C programming. Here's an excerpt: Week 4: Lecture 7: Introduction to Command Line Interfaces Lecture 8:

I asked it to give me an outline for a course on C programming. Here's an excerpt: Week 4: Lecture 7: Introduction to Command Line Interfaces Lecture 8: Makefiles and Building C Programs ... Week 11: Lecture 21: Network Security and Cryptography Lecture 22: Network Programming with C and C++ Week 12: Lecture 23: Introduction to Makefiles and GCC Lecture 24: Advanced Debugging Techniques with GDB and Valgrind Week 13: Lecture 25: Advanced Computer Architecture Concepts Lecture 26: Advanced Topics in Concurrency and Parallelism What's interesting here is that it does present a relatively sensible schedule. But look closer and it's completely forgotten all context. It's forgotten that it already introduced these topics: Lecture 8: Makefiles and Building C Programs and Lecture 23: Introduction to Makefiles and GCC And it puts them in a completely illogical spot between networking and concurrency. So it doesn't understand anything about the order in which these topics are introduced. It doesn't understand that it shouldn't introduce them twice. All it knows is that the word "Introduction" often appears on class schedules. And so in writing a class schedule, it included this phrase a number of times. But it didn't put it in the right spot. So even though literally no one would ever make this mistake in devising a class schedule, ChatGPT will because this isn't a logical class schedule, it's a stochastic class schedule. Sometimes in a stochastic class, you cover the intro material at the end!

Veen 4y ago
You can probably get it to improve on that by asking it if the order of the topics makes sense. It can often self-correct if you ask it to reflect on what it's already said and give it a push in the right direction, although not always.
- ModernMech 4y ago
  I did this, and it gave a performance of self correction, but it didn't actually demonstrate an understanding of what it did wrong: I said: How are the students supposed to compile their C programs before week 12? It said: You make a good point. Students would need to be able to compile their C programs before week 12. In fact, compiling C programs should be introduced earlier in the course, ideally in the first few weeks along with the introduction to C. Which, again, makes sense at face value, but the schedule does cover building C programs in the first few weeks. The problem is that it shouldn't be covering it again in week 12. The response I was hoping for was You make a good point. We introduced compiling C programs in week 4, so doing this topic again in week 12 is redundant. Instead we can cover a different topic that fits nicely, which would be.... You know what ChatGPT reminds me of? Talking to an undergraduate who always nods and agrees that they are following your logic. Then when you ask them to tell you their understanding, it's either an exact parrot of what you said, or wildly off due to some preconceived notion or general lack of understanding of the topic. Either way, they can never let on that they are clueless.

Veen 4y ago

You can probably get it to improve on that by asking it if the order of the topics makes sense. It can often self-correct if you ask it to reflect on what it's already said and give it a push in the right direction, although not always.

ModernMech 4y ago
I did this, and it gave a performance of self correction, but it didn't actually demonstrate an understanding of what it did wrong: I said: How are the students supposed to compile their C programs before week 12? It said: You make a good point. Students would need to be able to compile their C programs before week 12. In fact, compiling C programs should be introduced earlier in the course, ideally in the first few weeks along with the introduction to C. Which, again, makes sense at face value, but the schedule does cover building C programs in the first few weeks. The problem is that it shouldn't be covering it again in week 12. The response I was hoping for was You make a good point. We introduced compiling C programs in week 4, so doing this topic again in week 12 is redundant. Instead we can cover a different topic that fits nicely, which would be.... You know what ChatGPT reminds me of? Talking to an undergraduate who always nods and agrees that they are following your logic. Then when you ask them to tell you their understanding, it's either an exact parrot of what you said, or wildly off due to some preconceived notion or general lack of understanding of the topic. Either way, they can never let on that they are clueless.