4 ms·
I think we need a mandatory disclosure on software "was >1% vibecoded" same as we have on allergens for food. This'll prevent its use in any safety-critical pla
by dmitrygr 1y ago
I think we need a mandatory disclosure on software "was >1% vibecoded" same as we have on allergens for food. This'll prevent its use in any safety-critical place.
- darth_avocado 1y agoWhat if the software was not vibe coded but upstream packages were?
- _fzslm 1y agoIf we do, we need to draw clear distinctions between different kinds of AI-driven development. What % of human intervention was there? A module written for me by AI, that was tightly specced with function signatures and behaviour cases, is going to be far more reliable (and arguably basically is human developed) than something an AI just wrote and filled in all the blanks with.
- delfinom 1y agoTechnically for safety critical, safe code was to be audited and certified to the standard anyway. We don't just pull random open source packages. Granted vibe coded junk will quickly get avoided if it is poorly written to the point that it makes auditing insufferable.
- bee_rider 1y agoThis shouldn’t really matter, software can also be written by very bad coders. If you care about safety, you care about the whole process—coding, sure, but also: code review, testing, what the design specs were, and what the failure-path is for when a bug (inevitably) makes it through. Big companies produce lots of safety critical code, and it is inevitable that some incompetent people will sneak into the gaps, there. So, it is necessary to design a process that accounts for commits written by incompetent people.
- bobsomers 1y agoEverything you said is 100% correct. However, part of designing and upholding a safety-critical software development process is looking for places to reduce or eliminate the introduction of bugs in the first place. Strong type systems, for example, eliminate entire classes of errors, so mandating that code is written in X language is a pro-active process decision to reduce the introduction of certain types of bugs. Restricting the use of AI tools could very much be viewed the same way.
- paulddraper 1y agoSo you would suggest ">1% in dynamically typed language" disclaimers as well?
- uncircle 1y ago> This shouldn’t really matter, software can also be written by very bad coders. The issue is that there is a non-zero likelihood that a vibe coder pushes code he doesn’t even understand how it actually works. At least a bad coder had to have written the thing themselves in the first place.
- bee_rider 1y agoFor something safety critical, individual programmers shouldn’t be able to push code directly anyway. However, a vibe-coder spamming the process with bad code could cause it to jam up, and prevent forward progress (assuming the project has a well designed safe process). I guess I did assume, though, that by “in any safety-critical place” they meant a place with a well-defined and rigorous process (surely there’s some safety-critical code out there written by seat-of-the-pants cowboys, but that is just a catastrophe waiting to happen).
- wiseowise 1y agoGiven performance of an average SE, that would be an improvement. So I don’t know what you’re saying.
- dmitrygr 1y agoFailure modes of human coders are well-understood. Failure modes of LLMs are not yet as well understood.