5 ms·
Being in the energy sector dependencies is something we intentionally avoid because we'd actually have to go through and review changes. What has helped this im
by devjab 1y ago
Being in the energy sector dependencies is something we intentionally avoid because we'd actually have to go through and review changes. What has helped this immensely is AI code assistance. One of the primary uses is to use it to write CLI tools for code and config generation in the tool chain. All of it is around making your life easier, without pulling external dependencies.
An example is that we create openapi docs with LLM's. Then we serve them with Go's net/http + FileServer, meaning that we never leave the standard library. Of course the LLM itself is a third party dependency, but when you use it to write CLI tools that then do the code generation, it never actually sees the code. That also means that the quality of those CLI tools are less important, because it is their output that matters.
It only goes so long of course. I'm not sure any of our front-end engineers would want to live in a world where they weren't allowed to use React, but then, their products are treated as though they are external anyway.
Anyway, it's a lot easier to make engineers stop using "quality of life" dependencies when you give them something that still makes their lives easy.
- landgenoot 1y agoDoesn't the LLM spit out the code of the dependency it has been trained on? How is that any different from just forking the dependency and treating it like your own?
- smokel 1y agoOne advantage might be that you would only implement the (small) part of the library that you actually need, potentially avoiding bugs such as Log4Shell. The risk of code duplication is a serious one though, and it would be nice if AI could automatically detect and clean this. I guess we are in an annoying "in-between" state of things, with a pretty unsure future.
- nurettin 1y agoYou would (probably) be avoiding commonly known exploits while introducing subtle AI-induced bugs like printing logs out of order or not locking/ordering resources properly.
- smokel 1y agoOh yes, I fully agree. It will be quite horrible if we don't handle things properly.
- keithplayer 1y agoThe benefit of using a library directly is your 3rd party library checks will warn you when a CVE is found in the version you are using. If an LLM creates the same functionality from copying a version of a library, you might be getting a version that already has known vulnerabilities, and you probably won't be pulling in any fixes/improvements in future until you find them.
- dotancohen 1y agoWhy not both? Fork the dependency and use that, to have a stable non-changing base which you use. And additionally, make the original project a dependency but don't actually use it. This way you'll get CVE information from your tooling.
- Ygg2 1y agoIf you fork a dependency and change features, the CVE information on original depenency is now no longer valid for your code. Your additions or removals can induce new CVEs, or render CVE for original lib a moot point.
- christophilus 1y agoIt’s easier to incrementally review the generated, specific code of an AI agent than it is to review a generic, overly featured long-lived library— and to keep on top of (and review) all changes to that library over time.
- devjab 1y agoNot necessarily, you can task the AI Agent with writing your tool without the use of external dependencies. Often the tool itself will be absolutely horrible, but this doesn't matter if the output of the tool is fine. As in the case of OpenAPI documentation, you basically get the LLM to write the equivalent of C#'s Swashbuckle (or whatever it's called these days) for your language. What it produces is horrible in comparison to the third party dependencies you would have pulled 5 years ago, but the output is the same. You can also let it use outside dependencies, but then you're right, then it would make little difference in regards to security. We figured this out because Go didn't have a "Swashbuckle", and nobody wanted to write the documentation or use codegens to go in the opposite direction. When it turned out to be so easy to use LLM's for these things, it grew into it's own thing in regards to replacing a lot of dependencies. Obviously the LLM is still the dependency, but you can replace all other external dependencies for a lot of things like this. Which means you won't acidentially pull something malicious. I imagine we're going to see more and more of these in-house dependency replacement tools. Coming from Go, I obviously use SQLC (which is run inside it's own isolated container that is only available for requests from developers and is only updated by hand). If we couldn't run SQLC in total isolation then I imagine we would have had to build our own "SQLC". I haven't tried but I'm pretty confident that you could use LLM's to build a similar (but much worse in quality) tool. In an ideal world, we would have been allocated the resources we needed, but in reality, it would have just made us slower as "IT" is still viewed as a cost center on par with HR except that we aren't even "IT".
- pluto_modadic 1y ago[flagged]