3 ms·
While this example is explicitly asking for a port (thus a copy), I also find in general that LLM's default behavior is to spit out new code from their vast pre
by mNovak 10mo ago
While this example is explicitly asking for a port (thus a copy), I also find in general that LLM's default behavior is to spit out new code from their vast pre-trained encyclopedia, vs adding an import to some library that already serves that purpose.
I'm curious if this will implicitly drive a shift in the usage of packages / libraries broadly, and if others think this is a good or bad thing. Maybe it cuts down the surface of upstream supply-chain attacks?
- MangoToupe 10mo agoAs a corollary, it might also increase the surface of upstream supply-chain attacks (patched or not) The package import thing seems like a red herring
- Retr0id 10mo agoIt's going to be fun if someone finds a security vulnerability in a commonly-emitted-by-LLMs code pattern. That'll be a lot harder to remediate than "Update dependency xyz"
- MangoToupe 10mo ago> if someone finds a security vulnerability in a commonly-emitted-by-LLMs code pattern how do you distinguish this from injecting a vulnerable dependency to a dependency list?
- Retr0id 10mo agoYou can more easily check for known-vulnerable dependencies
- MangoToupe 10mo agoRight, but if you can embed bad packages in LLMs, you can surely embed any kind of vulnerability imaginable.
- Retr0id 10mo agoI'm not thinking about deliberately embedded vulnerabilities, just accidental/emergent ones. The modern equivalent of devs copy-pasting stackoverflow answers that happen to contain SQL injection vulns.
- MangoToupe 10mo agoDoes the distinction make any difference?
- Retr0id 10mo agoYes, you'd take different actions to avoid each.