3 ms·

The article throws out a lot of potential issues with AI generated code, but doesn't stop for a moment to consider if solutions currently exist or might exist t

by johnfn 1y ago

The article throws out a lot of potential issues with AI generated code, but doesn't stop for a moment to consider if solutions currently exist or might exist to these problems. - Before LLMs, provided that your team did some of their due diligence, you could always expect to have some help when tackling new code-bases. Has the author never worked on legacy code before? - (oh, and it has forgotten everything about the initial writing process by then). Does the author not think this can ever be solved? - Because of the situation of a Bus Factor of zero that it creates, vibe coding is fundamentally flawed. That is, only until there is an AI that can generate 100% accurate code 100% of the time, and it is fed 100% accurate prompts. Why does it require 100% accuracy 100% of the time? Humans are not 100% accurate 100% of the time and we seem to trust them with our code.

ndneighbor 1y ago
Yea, I found the article to be overly reductive. I work on a shared ownership codebase, which at times, I am not going to be able to pull the original author at all times when I work on my branch. At least having a partner explain some aspects is a huge unlock for me. Maybe the OP shadowboxing a world with no humans, but no humans is already a status quo issue that my team and I face sometimes.
- lovich 1y ago
  He’ll, have they never had to maintain applications where the source code was lost? I haven’t had to do it too often, and I wouldn’t claim I’m very good at it, but on more than a handful of occasions I have had to decompile binaries to figure out what the fuck it was doing before I could write a wrapper around it to try and mitigate its unfixeable issues
  - sfink 1y ago
    The author was just saying that this will become the norm, not the exception. Ok, not this bad -- we can at least expect to have access to the AI-generated code for at least a little while longer. (I can imagine a future where AI programming gets just robust enough that some early adopter types will take their whiteboard sketches and prompts and prose test case descriptions, generate the code, compile it to a binary, then throw out the code. It's more agile! You can tweak the spec slightly and just turn the crank again!)
raincole 1y ago
> Why does it require 100% accuracy 100% of the time? I've said this before, but I'd say it again: anti-AI people, instead of AI users, are usually those who expect AI to be magical panacea. The vibe reminds me of some people who are against static typing because "it can't catch logical error anyway."
- bgwalter 1y ago
  No, we treat it as a bureaucratic, distracting and infantile way of software engineering that does not work.
- seadan83 1y ago
  No, it's not a wish nor an expectation for a magical panacea, but logic. The only code you do not have to maintain is code that is fully perfect (100% accurate, 100% of the time). Bus factor zero is a problem for code that needs to be maintained. If AI is short of the 100%, then it is generating code that needs to be maintained. Ergo, a bus factor zero will be a problem for AI generated code (until such time as that code is perfect).
AntwaneB 1y ago
Hello, author here. Thanks for your comment. I agree with your first point, maybe AI will close some of those gaps with future advances, but I think a large part of the damage will have been done by then. Regarding the memory of reasoning from LLMs, I think the issue is that even if you can solve it in the future, you already have code for which you've lost the artifacts associated with the original generation. Overall I find there's a lot of talks (especially in the mainstream media) about AI "always learning" when they don't actually learn new anything until a new model is released. > Why does it require 100% accuracy 100% of the time? Humans are not 100% accurate 100% of the time and we seem to trust them with our code. Correct, but humans writing code don't lead to a Bus Factor of 0, so it's easier to go back, understand what is wrong and address it. If the other gaps mentioned above are addressed, then I agree that this also partially goes away.
- mmhmmmmmm 1y ago
  [dead]
- scarface_74 1y ago
  How is that any different than any other legacy code where the reason for the decisions being made have long been forgotten?
- johnfn 1y ago
  > Regarding the memory of reasoning from LLMs, I think the issue is that even if you can solve it in the future, you already have code for which you've lost the artifacts associated with the original generation. Overall I find there's a lot of talks (especially in the mainstream media) about AI "always learning" when they don't actually learn new anything until a new model is released. But this already exists! At work, our code is full of code where the original reasoning for the code is lost. Sometimes someone has forgotten, sometimes the person who wrote it is no longer at the company any more, and so on. > Correct, but humans writing code don't lead to a Bus Factor of 0, so it's easier to go back, understand what is wrong and address it. But there are plenty of instances where I work with code that has a bus factor of 0. The conclusion of your article is that vibe coding is "fundamentally flawed". But every aspect you've described about vibe coding has an analog in normal software engineering, and I don't think you would claim that is "fundamentally flawed".
theshrike79 1y ago
I _wish_ I had an AI tool when digging into some legacy codebases in my past. "Just ask the author of that perl file". Sure, it was last edited in -97 and the author is the regional manager now, should I just book a meeting or...?

by johnfn 1y ago

ndneighbor 1y ago

Yea, I found the article to be overly reductive. I work on a shared ownership codebase, which at times, I am not going to be able to pull the original author at all times when I work on my branch. At least having a partner explain some aspects is a huge unlock for me. Maybe the OP shadowboxing a world with no humans, but no humans is already a status quo issue that my team and I face sometimes.

lovich 1y ago
He’ll, have they never had to maintain applications where the source code was lost? I haven’t had to do it too often, and I wouldn’t claim I’m very good at it, but on more than a handful of occasions I have had to decompile binaries to figure out what the fuck it was doing before I could write a wrapper around it to try and mitigate its unfixeable issues
- sfink 1y ago
  The author was just saying that this will become the norm, not the exception. Ok, not this bad -- we can at least expect to have access to the AI-generated code for at least a little while longer. (I can imagine a future where AI programming gets just robust enough that some early adopter types will take their whiteboard sketches and prompts and prose test case descriptions, generate the code, compile it to a binary, then throw out the code. It's more agile! You can tweak the spec slightly and just turn the crank again!)

raincole 1y ago

> Why does it require 100% accuracy 100% of the time? I've said this before, but I'd say it again: anti-AI people, instead of AI users, are usually those who expect AI to be magical panacea. The vibe reminds me of some people who are against static typing because "it can't catch logical error anyway."

bgwalter 1y ago
No, we treat it as a bureaucratic, distracting and infantile way of software engineering that does not work.
seadan83 1y ago
No, it's not a wish nor an expectation for a magical panacea, but logic. The only code you do not have to maintain is code that is fully perfect (100% accurate, 100% of the time). Bus factor zero is a problem for code that needs to be maintained. If AI is short of the 100%, then it is generating code that needs to be maintained. Ergo, a bus factor zero will be a problem for AI generated code (until such time as that code is perfect).

AntwaneB 1y ago

Hello, author here. Thanks for your comment. I agree with your first point, maybe AI will close some of those gaps with future advances, but I think a large part of the damage will have been done by then. Regarding the memory of reasoning from LLMs, I think the issue is that even if you can solve it in the future, you already have code for which you've lost the artifacts associated with the original generation. Overall I find there's a lot of talks (especially in the mainstream media) about AI "always learning" when they don't actually learn new anything until a new model is released. > Why does it require 100% accuracy 100% of the time? Humans are not 100% accurate 100% of the time and we seem to trust them with our code. Correct, but humans writing code don't lead to a Bus Factor of 0, so it's easier to go back, understand what is wrong and address it. If the other gaps mentioned above are addressed, then I agree that this also partially goes away.

mmhmmmmmm 1y ago
[dead]
scarface_74 1y ago
How is that any different than any other legacy code where the reason for the decisions being made have long been forgotten?
johnfn 1y ago
> Regarding the memory of reasoning from LLMs, I think the issue is that even if you can solve it in the future, you already have code for which you've lost the artifacts associated with the original generation. Overall I find there's a lot of talks (especially in the mainstream media) about AI "always learning" when they don't actually learn new anything until a new model is released. But this already exists! At work, our code is full of code where the original reasoning for the code is lost. Sometimes someone has forgotten, sometimes the person who wrote it is no longer at the company any more, and so on. > Correct, but humans writing code don't lead to a Bus Factor of 0, so it's easier to go back, understand what is wrong and address it. But there are plenty of instances where I work with code that has a bus factor of 0. The conclusion of your article is that vibe coding is "fundamentally flawed". But every aspect you've described about vibe coding has an analog in normal software engineering, and I don't think you would claim that is "fundamentally flawed".

theshrike79 1y ago

I _wish_ I had an AI tool when digging into some legacy codebases in my past. "Just ask the author of that perl file". Sure, it was last edited in -97 and the author is the regional manager now, should I just book a meeting or...?