7 ms·
If a weakness is common, then of course Copilot is going to suggest it. Copilot gives you popular responses not correct ones. Yet if a weakness is common, it
by faeriechangling 3y ago
If a weakness is common, then of course Copilot is going to suggest it. Copilot gives you popular responses not correct ones. Yet if a weakness is common, it also means that human coders frequently make the same mistake as well.
The studies results are rather unsurprising and its conclusions are oft-repeated advice. As many have said, treat copilot’s code in the same light you would treat a junior programmer’s code.
- pylua 3y agoI wonder if llm are biased towards older, more insecure implementations because there is a higher volume of old code vs new code. Same thing with the data it is trained on — not all code requires all levels of refinement. Most of the data is probably around average.
- squigz 3y agoI'm not sure that code being newer inherently means it will be more secure
- pylua 3y agoI don’t think it is a tautology , but I can imagine a cve scanner picking up older code with log4j where newer code may avoid that library altogether, just as an example. Since there is more older code than newer code would the llm be suspectible to that ?
- xmcqdpt2 3y agoEveryone still uses log4j after the fix. There aren't many full-featured alternatives... and the one that exist probably contain unfixed bugs.
- darkerside 3y ago> Most of the data is probably around average. I know this is not how distributions work, but I had to chuckle at the literal interpretation of this.
- d-z-m 3y agoI'd say the data is pretty normal.
- foota 3y agoThis makes me wonder about training an LLM on one language and then fine tuning it for another. If you train over only, say, JavaScript, and then finetune for C, I imagine it will be quite bad at writing safe code, even if it makes the code look like C, because it didn't have to learn about freeing and such. Similarly, would it pick up patterns from one language and keep then in the other? Maybe an LLM trained on Kotlin would be more likely to write functional code finetuned.
- fragmede 3y agoGiven that dataset anomalies can result in LLM output corruption, I'm not convinced that cross-training like that would even work.
- diogenes4 3y ago> Yet if a weakness is common, it also means that human coders frequently make the same mistake as well. It only means programmers commonly talk about it. This isn't the same thing as measuring incidence in production or distribution. Anyway, i'd argue the real question is "can the chatbot fix the code if requested to".
- lolinder 3y ago> It only means programmers commonly talk about it. This isn't the same thing as measuring incidence in production or distribution. Copilot was primarily trained on GitHub projects, not on communication between programmers. Patterns that frequently show up in Copilot output are most likely prevalent on GitHub, which is a pretty good indicator that they're common in production code.
- prosim 3y agoThat’s no longer true. Copilot uses the same ChatGPT-3.5 model as, well, ChatGPT. If it were trained on just GitHub projects, the chat features wouldn’t work at all.
- lolinder 3y agoYou're assuming that Copilot Chat and the regular completion are the same model. Do you have a source that says so? I'd assumed that they were two different models, since they're quite different tasks.
- prosim 3y agoFootnote 1 on page 2 explicitly mentions the 3.5 model and the research in this paper is only about auto completion: https://arxiv.org/pdf/2306.15033.pdf https://arxiv.org/pdf/2306.15033.pdf And this blog post states “beyond Codex”, again for auto completion: https://github.blog/2023-07-28-smarter-more-efficient-coding-github-copilot-goes-beyond-codex-with-improved-ai-model/ https://github.blog/2023-07-28-smarter-more-efficient-coding... Lastly, OpenAI states on the original Codex page: “OpenAI Codex is a descendant of GPT-3; its training data contains both natural language and billions of lines of source code from publicly available sources, including code in public GitHub repositories.” - It included GitHub repos, but it never was only GitHub repos. https://openai.com/blog/openai-codex https://openai.com/blog/openai-codex Update: GitHub Community Manager confirms it here: https://github.com/orgs/community/discussions/56975#discussioncomment-6086471 https://github.com/orgs/community/discussions/56975#discussi...
- crooked-v 3y ago> Copilot gives you popular responses not correct ones. That also sums up most of the issues with LLMs in general in one sentence.
- TheRoque 3y agoSums up the issues with democracy too, and a ton of other stuff
- deleted 3y ago[deleted]
- sambazi 3y agoeducating the "low-hanging fruit" is much more effective in moving the average than piling on excellence.
- ape4 3y agoDemocracy is a bit different. Hopefully the goal is a democratic government isn't to be intelligent but is to just make people's lives good (better?). In theory if people find their lives are getting worse then they can replace the government. But, sure, there are many examples of it not working. Such has the government making gas prices low because the people want that when its polluting the planet that the people live on.
- baby 3y agoWith humans too
- steve1977 3y agoWhich is why the term Artificial Intelligence is really a misnomer for LLMs. Artificial Mediocracy might be more fitting.
- throwaway20304 3y agoDo you think people with IQ below 80 are not intelligent?
- renewiltord 3y agoA junior programmer's code? This makes no sense. It's happening right in front of you. A junior programmer isn't going to write on my screen. I can just correct it right here I am currently holding the context in my head. These "security weakness" examples are print("first user registered, role set to admin", user, password) and pprint({"json":"somejunk", "classes": somefunc(user)}) Nah, this stuff I can easily spot while I'm writing code. For a junior programmer, I'm going to be looking at design, and then at common specific mistakes. For Copilot it's writing in front of me. I can easily exclude anything that isn't obviously correct because I'm in the state right there. It's a fantastic tool. If you go and use it and end up with `print(user_credentials)` I don't know what to tell you.
- baq 3y agoThe added complication is now you'll have to watch out for the junior+copilot combo, though it's a trade I personally am very willing to take.
- devjab 3y agoYou’ve had to watch out for “bad” programmers since the beginning of programming. Having been an external CS examiner for almost a decade now, I’m not too worried about LLMs in teaching because I’m not convinced they can do worse than what we’ve been doing this far. I do think it’s a little frightening that a lot of freshly educated Computer Scientists will have to “unlearn” a good amount of what they were taught to actually write good code, but on the flip side I work in a place where a big part of our BI related code is now written by people from social sciences because, well, they are easier to hire. That’s how you end up with pipelines that can’t handle 1000 PDF documents, because they are simply designed not to scale beyond one or two documents. Because that’s what you get when you google program, or alternatively, when you use ChatGPT, and it’s fine… at least until it isn’t, but it’s not like you can’t already make a lucrative career “fixing” things once they stop being “good enough”. So I’m not sure things will really change. If anything I think LLMs will be decent in the hands of both juniors and senior developers, it’s the mediocre developers who are in danger. At least with google programming they could easily tell if an SO answer or an article was from 20 years ago, that info isn’t readily available with LLMs. I fully expect to be paid well to clean up a lot of ChatGPT messes until the end of my career.
- hiAndrewQuinn 3y agoIt would be very interesting to fine-tune Copilot on the code of people widely regarded in their communities as experts, to see how the suggestions would change.
- reportgunner 3y agoBrawndo is great for plants because it has elecrolytes.
- smcg 3y agoThat's not how it's presented or how managers expect it to be used.