4 ms·
I think it depends a lot on what you're trying to get them to do and what you're working on. I don't really buy into the whole "prompt engineering" part. I use
by devjab 1y ago
I think it depends a lot on what you're trying to get them to do and what you're working on. I don't really buy into the whole "prompt engineering" part. I use agents quite a lot, but they have limitations. They excel at writing SQL, kubernetes configuration, bicep and similar... at least so long as you know what you're doing. I also have quite a lot of success with them keeping documentation and tests updated. I wouldn't recommend using them for your tests unless tests are a nice to have feature for you because you actively deal with corrupted states through explicit error handling, but we do.
In my experience every model absolutely sucks at doing any form of software engineering. The thing is though, you don't need to do any software engineering to write software... until you do. Since LLM's got decent at programming we've let them loose among our BI and solar plant engineers and they're building some really useful tools in Python. They have no idea what they are doing, and they are still getting stuff to work. It's obviously absolute horse shit, but for a lot of it, the shit code will be "good enough" for nobody having to touch it for it's lifecycle. For the rest, actual software engineers take over.
If you're working in a field where "good enough" is fine, then I imagine that LLM's can do a lot for you. It might sound like I'm being disrespectful, but I really don't mean to be. In my decades of experience with software in enterprise settings, I've seen so many programs build with horrible code just trundle along. Creating great value for the business while requiring no to minimal maintenance. Bad from a traditionalist software engineering perspective, but great from a business perspective (which is what matters). Not everything needs to scale after all. Anyway, I do question whether the author has actually worked with LLM's a lot. I would think that the strengths failings of LLM's would get obvious rather quickly to anyone who uses them.
- nyarlathotep_ 1y ago> Bad from a traditionalist software engineering perspective, but great from a business perspective (which is what matters). Not everything needs to scale after all. Anyway, I do question whether the author has actually worked with LLM's a lot. I would think that the strengths failings of LLM's would get obvious rather quickly to anyone who uses them. Few things I've seen here. > If you're working in a field where "good enough" is fine, then I imagine that LLM's can do a lot for you. There's some specific timesheet software used for Gov Contractors that's hands-down, the worst web "application" I've ever used. Opening devTools on one of these pages is a riot--one of the "APIs" used return HTML documents with js file extensions so the browser fails to parse, every JS file is 1K lines long with blocks of global variables with various naming conventions, using 'back' in the browser just sort of fails and returns you to the sign-in page. Any modern web programmer would have a stroke looking at this thing. Apparently it's the standard software used for this purpose, relied on all over the industry. At a previous employer, I had to deal with some internal site for financial transactions and billing. Same story, ancient looking and performing web 'app' that had specific incantations to get it to actually work "don't refresh the page until that loads", piles of JS vomiting bizarre errors in the console, just a remarkable piece of junk. Anyway, short story long, there's LOADS of software that's terrible by any metric that have crucial purposes floating around that are far worse in terms of 'code quality' than any LLM would spit out. Also, I have more experience than I like to remember dealing with 'body shop' codebases. There is no way that an LLM is even capable of generating such low quality code. I legitimately think several of the projects I saw were sabotage; there's no way a human would deliberately write such inscrutable junk. Ever seen random python runtime errors cause the type of something would change depending on which branches were hit?
- devjab 1y agoI would argue that "good enough" means that it doesn't add inconvenience or significant cost, but I know what you're talking about. That is the flip-side of it though, there is already so much non-AI garbage that it sometimes doesn't even registrer with the business when something was written by a MBA and some LLM. Not because that is necessarily a good thing, but it is what it is. Nothing we do that is user facing is created by AI (this doesn't include any of our business intelligence as that's another department). We currently have a frontend lead who passionately hates AI for basically all of the reasons you bring up, and as long as they keep delivering, I doubt that will change anytime soon. As far as the BI goes, well, that will probably always be a horrible mess because it's next to impossible to find someone who's interested in working with the development/design side of PowerBI. I'm not sure it's even because we won't pay them enough, there doesn't seem to be anyone around here who wants a job like that. We tell our collective selves that it'll be better when we switch to Microsoft Fabric, but we'll see.
- acuozzo 1y ago> There's some specific timesheet software used for Gov Contractors that's hands-down, the worst web "application" I've ever used. Deltek. Yuck.
- nyarlathotep_ 1y agoPop open the DevTools some time. It's a treat!