3 ms·

> we tasked participants (student programmers) with implementing a singly-linked 'shopping list' structure in C Uhuh... does anyone think that LLM would actua

> we tasked participants (student programmers) with implementing a singly-linked 'shopping list' structure in C Uhuh... does anyone think that LLM would actually impact this case? The LLM has likely been trained on thousands of "top-quality" code examples covering exactly this data structure. No one could plausibly regard LLMs as impairing a case of this kind. This seems quite silly.

moyix 4y ago
This seems like a silly objection to me, although I am one of the authors :) Yes, Codex has been trained on lots of linked lists. It has also been trained on lots of bad and insecure versions of linked lists! And when we did our earlier study ("Asleep at the Keyboard"), which looked at the security of code written by Copilot without human intervention, we found that it produced vulnerable code at high rates (40%) even for common scenarios like SQL injection and C string manipulation. So your prior that LLMs should be good at linked lists is not at all obvious; that's why we ran the study. Other complaints in this thread about studying students - sure, perhaps experienced developers will do better. These were upper level undergraduate and masters students though, so they're the kind of developer you will have writing code with Codex in industry within a year or so, and so they are still worth studying. It's not the whole picture, but we never claim it is.
IshKebab 4y ago
You are right, this is a silly study. "...in students" is pretty much the same as "...in mice". I'm waiting for someone to conclude that code comments are pointless because they make no difference to the ability of students to implement fizzbuzz.
PaulHoule 4y ago
Overall I think the safety problems get worse when the program gets more complex. There's a big difference between getting a 100 line program right and getting a 20,000 line program right. That said this is an underpowered study to really answer the question. It reminds me a little of that Scott Alexander blog post where he came to big conclusions based on 5 questions he asked ChatGPT. However there is a definite industry of people who write papers where they ask ChatGPT some questions and evaluate the replies.

moyix 4y ago

This seems like a silly objection to me, although I am one of the authors :) Yes, Codex has been trained on lots of linked lists. It has also been trained on lots of bad and insecure versions of linked lists! And when we did our earlier study ("Asleep at the Keyboard"), which looked at the security of code written by Copilot without human intervention, we found that it produced vulnerable code at high rates (40%) even for common scenarios like SQL injection and C string manipulation. So your prior that LLMs should be good at linked lists is not at all obvious; that's why we ran the study. Other complaints in this thread about studying students - sure, perhaps experienced developers will do better. These were upper level undergraduate and masters students though, so they're the kind of developer you will have writing code with Codex in industry within a year or so, and so they are still worth studying. It's not the whole picture, but we never claim it is.

IshKebab 4y ago

You are right, this is a silly study. "...in students" is pretty much the same as "...in mice". I'm waiting for someone to conclude that code comments are pointless because they make no difference to the ability of students to implement fizzbuzz.

PaulHoule 4y ago

Overall I think the safety problems get worse when the program gets more complex. There's a big difference between getting a 100 line program right and getting a 20,000 line program right. That said this is an underpowered study to really answer the question. It reminds me a little of that Scott Alexander blog post where he came to big conclusions based on 5 questions he asked ChatGPT. However there is a definite industry of people who write papers where they ask ChatGPT some questions and evaluate the replies.