3 ms·
From the paper: "Specifically, we first conduct keyword searches, such as “generated by ChatGPT” and “generated by Copilot”, to locate GitHub code files that i
by staplung 2y ago
From the paper:
"Specifically, we first conduct keyword searches, such as “generated by ChatGPT” and “generated by Copilot”, to locate GitHub code files that include such keywords, retaining only those files that contain GPT-generated code."
This seems like a pretty serious weakness to me; presumably there is a lot of code generated by LLMs that isn't annotated as such.
- KeplerBoy 2y agoI never saw code annotated as generated by Copilot. That's just not how that tool is used.
- Atotalnoob 2y agoShould it be labeled? I don’t label my code when I use GitHub copilot. I carefully read the code and use TDD where GitHub copilot generates code to match my test spec. I also provide samples of already written code to ensure it follows a similar pattern…
- cirrus3 2y agoYea, this paper is useless if that is the starting point.