6 ms·

>The people really leading AI coding right now (and I’d put myself near the front, though not all the way there) don’t read code. They manage the things that pr

>The people really leading AI coding right now (and I’d put myself near the front, though not all the way there) don’t read code. They manage the things that produce code. I can’t imagine any other example where people voluntarily move for a black box approach. Imagine taking a picture on autoshot mode and refusing to look at it. If the client doesn’t like it because it’s too bright, tweak the settings and shoot again, but never look at the output. What is the logic here? Because if you can read code, I can’t imagine poking the result with black box testing being faster. Are these people just handing off the review process to others? Are they unable to read code and hiding it? Why would you handicap yourself this way?

manmal 9mo ago
> I can’t imagine any other example where people voluntarily move for a black box approach. Anyone overseeing work from multiple people has to? At some point you have to let go and trust people‘s judgement, or, well, let them go. Reading and understanding the whole output of 9 concurrently running agents is impossible. People who do that (I‘m not one of them btw) must rely on higher level reports. Maybe drilling into this or that piece of code occasionally.
- re-thc 9mo ago
  > Anyone overseeing work from multiple people has to? That's not a black box though. Someone is still reading the code. > At some point you have to let go and trust people‘s judgement Where's the people in this case? > People who do that (I‘m not one of them btw) must rely on higher level reports. Does such a thing exist here? Just "done".
  - manmal 9mo ago
    > Someone is still reading the code. But you are not. That’s the point? > Where's the people in this case? Juniors build worse code than codex. Their superiors also can‘t check everything they do. They need to have some level of trust for doing dumb shit, or they can’t hire juniors. > Does such a thing exist here? Just "done". Not sure what you mean. You can definitely ask the agent what it built, why it built it, and what could be improved. You will get only part of the info vs when you read the output, but it won’t be zero info.
    - re-thc 9mo ago
      > You can definitely ask the agent what it built, why it built it, and what could be improved. If that was true we’d have what they call AGI. So no, it doesn’t actually give you those since it can’t reason and logic in such a way.
      - manmal 9mo ago
        What does that have to do with AGI?
        re-thc 9mo ago
        What you asked for is AGI. How else does it think, reason and logic to answer your "why"? It doesn't do that currently even if you think it does.
    - just6979 9mo ago
      You: "Why did you build this?" LLM: "Because the embeddings in your prompt are close to some embeddings in my training data. Here's some seemingly explanatory text with that is just similar embeddings to other 'why?' questions." You: "What could be improved?" LLM: "Here's some different stuff based on other training data with embeddings close to the original embeddings, but different. --- It's near zero useful information. Example imformation might be "it builds" (baseline necessity, so useless info), "it passes some tests" (fairly baseline, more useful, but actually useless if you don't know what the tests are doing), or "it's different" (duh).
      - deleted 9mo ago
        [deleted]
      - manmal 9mo ago
        If I asked you, about a piece of code that you didn’t build, „What would you improve?“, how would that be fundamentally different?
        8mo ago
- torben-friis 9mo ago
  >At some point you have to let go and trust people‘s judgement. Indeed. People. With salaries, general intelligence, a stake in the matter and a negative outcome if they don’t take responsibility. >Reading and understanding the whole output of 9 concurrently running agents is impossible. I agree. It is also impossible for a person to drive two cars at once… so we don’t. Why is the starting point of the conversation that one should be able to use 9 concurring agents? I get it, writing code no longer has a physical bottleneck. So the bottleneck becomes the next thing, which is our ability to review outputs. It’s already a giant advancement, why are we ignoring that second bottleneck and dropping quality assurance as well? Eventually someone has to put their signature on the thing being shippable.
  - deleted 9mo ago
    [deleted]
  - wtetzner 9mo ago
    Is reviewing outputs really more efficient than writing the code? Especially if it's a code base you haven't written code in?
    - kuschku 9mo ago
      It is not. To review code you need to have an understanding of the problem that can only be built by writing code. Not necessarily the final product, but at least prototypes and experiments that then inform the final product.
- ink_13 9mo ago
  An AI agent cannot be held accountable
  - manmal 9mo ago
    Neither can employees, in many countries.
CharlesW 9mo ago
AI-assisted coding is not a black box in the way that managing an engineering team of humans is. You see the model "thinking", you see diffs being created, and occasionally you intervene to keep things on track. If you're leveraging AI professionally, any coding has been preceded by planning (the breadth and depth of which scale with the task) and test suites.
csallen 9mo ago
> Imagine taking a picture on autoshot mode and refusing to look at it. If the client doesn’t like it because it’s too bright, tweak the settings and shoot again, but never look at the output. The output of code isn't just the code itself, it's the product. The code is a means to an end. So the proper analogy isn't the photographer not looking at the photos, it's the photographer not looking at what's going on under the hood to produce the photos. Which, of course, is perfectly common and normal.
- alanbernstein 9mo ago
  Right, it seems the appropriate analogy is the shift from analog-photograph-developers to digital camera photographers.
- torben-friis 9mo ago
  >The output of code isn't just the code itself, it's the product. The code is a means to an end. I’ll bite. Is this person manually testing everything that one would regularly unit test? Or writing black box tests that he does know are correct because of being manually written? If not, you’re not reviewing the product either. If yes, it’s less time consuming to actually read and test the damn code
  - CuriouslyC 9mo ago
    I mostly ignore code, I lean on specs + tests + static analysis. I spot check tests depending on how likely I think it is for the agent to have messed up or misinterpreted my instructions. I push very high test coverage on all my projects (85%+), and part of the way I build is "testing ladders" where I have the agent create progressively bigger integration tests, until I hit e2e/manual validation.
    - torben-friis 9mo ago
      >I spot check tests depending on how likely I think it is for the agent to have messed up or misinterpreted my instructions So a percentage of your code, based on your gut feeling, is left unseen by any human by the moment you submit it. Do you agree that this rises the chance of bugs slipping by? I don’t see how you wouldn’t. And considering the fact that your code output is larger, the percentage of it that is buggy is larger, and (presumably) you write faster, have you considered the conclusion in terms of the compounding likelihood of incidents?
notepad0x90 9mo ago
people care about results. Better processes need to produce better results. this is programming not a belief system where you have to adhere to some view or else.
Aeolun 9mo ago
> What is the logic here? It is right often enough that your time is better spent testing the functionality than the code. Sometimes it’s not right, and you need to re-instruct (often) or dive in (not very often).
- torben-friis 9mo ago
  I can’t imagine retesting all the functionality of a well established product for possible regressions not being stupidly time consuming. This is the very reason why we have unit tests in the first place, and why they are far more numerous in tests than end-to-end ones.
Xirdus 9mo ago
> I can’t imagine any other example where people voluntarily move for a black box approach. I can think of a few. The last 78 pages of any 80-page business analysis report. The music tracks of those "12 hours of chill jazz music" YouTube videos. Political speeches written ahead of time. Basically - anywhere that a proper review is more work than the task itself, and the quality of output doesn't matter much.
- ink_13 9mo ago
  So... things where the producer doesn't respect the audience? Because any such analysis would be worth as much as a 4.5 hour atonal bass solo.
  - sroerick 9mo ago
    You can get an AI to listen to that bass solo for you
    - just6979 9mo ago
      But can you get an AI to zone out on a fluffy couch at the center point of a dank hi-fi setup with the volume cranked to 11, while chillin' on 50mg of THC? And will you enjoy paying someone else to let the AI to do that?
ForHackernews 9mo ago
>Imagine taking a picture on autoshot mode Almost everyone does this. Hardly anyone taking pictures understands what f-stop or focal length are. Even those who do seldom adjust them. There dozens of other examples where people voluntarily move to a black box approach. How many Americans drive a car with a manual transmission?
- weikju 9mo ago
  You missed out on the rest of the analogy though, which is the part where the photo is not reviewed before handing it over to the client.
- sigseg1v 9mo ago
  Hey it's me! I shoot with manual focus lenses in RAW and drive a standard. There are dozens of us!
weikju 9mo ago
Don’t read the code, test for desired behavior, miss out on all the hidden undesired behavior injected by malicious prompts or AI providers. Brave new world!
- thefz 9mo ago
  You made me imagine AI companies maliciously injecting backdoors in generated code no one reads, and now I'm scared.
  - gibsonsmog 9mo ago
    My understanding is that it's quite easy to poison the models with inaccurate data, I wouldn't be surprised if this exact thing has happened already. Maybe not an AI company itself, but it's definitely in the purview of a hostile actor to create bad code for this purpose. I suppose it's kind of already happened via supply chain attacks using AI generated package names that didn't exist prior to the LLM generating them.
  - djeastm 9mo ago
    One mitigation might be to use one company's model to check the work of another company's code and depend on market competition to keep the checks and balances.
    - thefz 9mo ago
      What about writing the actual code yourself
      - just6979 9mo ago
        Nah, more fun to burn money.
    - just6979 9mo ago
      Then how many models deep do you go before it's more cost effective to just hire a junior dev, supply them with a list of common backdoors, and have them scan the code?
raincole 9mo ago
> Because if you can read code, I can’t imagine poking the result with black box testing being faster. I don't know... it depends on the use case. I can't imagine even the best front-end engineer ever can read HTML faster than looking at the rendered webpage to check if the layout is correct.
- nubg 9mo ago
  Good analogy.
seanmcdirmid 9mo ago
> What is the logic here? Because if you can read code, I can’t imagine poking the result with black box testing being faster. The AI also writes the black box tests, what am I missing here?
- torben-friis 9mo ago
  >The AI also writes the black box tests, what am I missing here? If the AI misinterpreted your intentions and/or missed something in productive code, tests are likely to reproduce rather than catch that behavior. In other words, if “the ai is checking as well” no one is.
  - seanmcdirmid 9mo ago
    That's true. Never let the AI know about the code it wrote when writing the test for sure. Write multiple tests, have an arbitrator (also AI) figure out if implementation or tests are wrong when tests fail. Have the AI heavily comment code and heavily comment tests in the language of your spec so you can manually verify if the scenarios/parts of the implementations make sense when it matters. etc...etc... > In other words, if “the ai is checking as well” no one is. "I tried nothing, and nothing at all worked!"
hjoutfbkfd 9mo ago
your metaphor is wrong. code is not the output. functionality is the output, and you do look at that.
- torben-friis 9mo ago
  Explain then how testing the functionality (not the new one; regressions included, this is not a school exercise) is faster than checking the code. Are you writing black box testing by hand, or manually checking, everything that would normally be a unit test? We have unit tests precisely because of how unworkable the “every test is black box” approach is.
straydusk 9mo ago
No pun intended but - it's been more "vibes" than science that I've done this. It's more effective. When I focus my attention on the harness layer (tests, hooks, checks, etc), and the inputs, my overall velocity improves relative to reading & debugging the code directly. To be fair - it is not accurate to say I absolutely never read the code. It's just rare, and it's much more the exception than the rule. My workflow just focuses much more on the final product, and the initial input layer, not the code - it's becoming less consequential.
AlexCoventry 9mo ago
> What is the logic here? Because if you can read code, I can’t imagine poking the result with black box testing being faster. It's producing seemingly working code faster than you can closely review it.
- torben-friis 9mo ago
  Your car can also move faster than what you can safely control. Knowing this, why go pedal to the metal?
eikenberry 9mo ago
I think many people are missing the overall meaning of these sorts of posts.. that is they are describing a new type of programmer that will only use agents and never read the underlying code. These vibe/agent coders will use natural(-ish) language to communicate with the agents and wouldn't look at the code anymore than, say, a PHP developer would look at the underlying assembly. It is not the level of abstraction they are working on. There are many use cases where this type of coding will work fine and it will let many people who previously couldn't really take advantage of computers to do so. This is great but in no way will do anything to replace the need for code that requires humans to understand (which, in turn, requires participation in the writing).
- straydusk 9mo ago
  I'm glad you wrote this comment because I completely agree with it. I don't think that there is not a need for software engineers to deeply consider architecture; who can fully understand the truly critical systems that exist at most software companies; who can help dream up the harness capabilities to make these agents work better. I just am describing what I'm doing now, and what I'm seeing at the leading edge of using these tools. It's a different approach - but I think it'll become the most common way of producing software.
- re-thc 9mo ago
  > that is they are describing a new type of programmer that will only use agents and never read the underlying code > and wouldn't look at the code anymore than, say, a PHP developer would look at the underlying assembly This really puts down the work that the PHP maintainers have done. Many people spend a lot of time crafting the PHP codebase so you don't have to look at the underlying assembly. There is a certain amount of trust that I as a PHP developer assume. Is this what the agents do? No. They scrape random bits of code everywhere and put something together with no craft. How do I know they won't hide exploits somewhere? How do I know they don't leak my credentials?
  - 6510 9mo ago
    That is true for all languages. Very high quality until you use a lib, a module or an api.
bloomca 9mo ago
I think this is the logical next step -- instead of manually steering the model, just rely on the acceptance criteria and some E2E test suite (that part is tricky since you need to verify that part). I personally think we are not that far from it, but it will need something built on top of current CLI tools.
andyferris 9mo ago
The output is the program behavior. You use it, like a user, and give feedback to the coding agent. If the app is too bright, you tweak the settings and build it again. Photography used to involve developing film in dark rooms. Now my iPhone does... god knows what to the photo - I just tweak in post, or reshoot. I _could_ get the raw, understand the algorithm to transform that into sRGB, understand my compression settings, etc - but I don't need to. Similarly, I think there will be people who create useful software without looking at what happens in between. And there will still be low-level software engineers for whom what happens in between is their job.
vidarh 9mo ago
Your product managers most likely are not reading your code. Your CEO is not. The vast majority of your company is unlikely to ever look at a line of code. If the process becomes reliable enough, then there is no reason. For now, that still requires developers to pay attention for important projects, but there are also a lot of AI written tools I rely on day to day that I don't, because the opportunity cost of spending time reading them is lower than the cost of accepting the risk that they do something wrong. There are also a whole lot of tools I do read thoroughly, because the risk profile is different. But that category is getting smaller day by day, not just with model improvements, but with improved harnesses.
llamajams 9mo ago
Same. I stopped reading after that. I get the sense that most of these people thing all code is web or mobile or something non critical. Granted im not a web or mobile guy so I cant presume the complexity, risk, cost of such things. But I assume its in a different category than safety/mission critical things. I do dev tools for ASIL-B systems devs now and even then I cant say im comfortable not reading the generated code. Some of my junior peers are though, and im very frustrated that I feel like I keep having to play AI janitor, dont think the bosses care.
laterium 9mo ago
You generate 20k LOC in a few hours. How long will it take you to read it? One week? You just keep going instead. I don't think this works great at large scale production codebases yet, but it's an approach that will have more and more applications going forward. It doesn't have to fit every use case.

manmal 9mo ago

> I can’t imagine any other example where people voluntarily move for a black box approach. Anyone overseeing work from multiple people has to? At some point you have to let go and trust people‘s judgement, or, well, let them go. Reading and understanding the whole output of 9 concurrently running agents is impossible. People who do that (I‘m not one of them btw) must rely on higher level reports. Maybe drilling into this or that piece of code occasionally.

re-thc 9mo ago
> Anyone overseeing work from multiple people has to? That's not a black box though. Someone is still reading the code. > At some point you have to let go and trust people‘s judgement Where's the people in this case? > People who do that (I‘m not one of them btw) must rely on higher level reports. Does such a thing exist here? Just "done".
- manmal 9mo ago
  > Someone is still reading the code. But you are not. That’s the point? > Where's the people in this case? Juniors build worse code than codex. Their superiors also can‘t check everything they do. They need to have some level of trust for doing dumb shit, or they can’t hire juniors. > Does such a thing exist here? Just "done". Not sure what you mean. You can definitely ask the agent what it built, why it built it, and what could be improved. You will get only part of the info vs when you read the output, but it won’t be zero info.
  - re-thc 9mo ago
    > You can definitely ask the agent what it built, why it built it, and what could be improved. If that was true we’d have what they call AGI. So no, it doesn’t actually give you those since it can’t reason and logic in such a way.
    - manmal 9mo ago
      What does that have to do with AGI?
      - re-thc 9mo ago
        What you asked for is AGI. How else does it think, reason and logic to answer your "why"? It doesn't do that currently even if you think it does.
  - just6979 9mo ago
    You: "Why did you build this?" LLM: "Because the embeddings in your prompt are close to some embeddings in my training data. Here's some seemingly explanatory text with that is just similar embeddings to other 'why?' questions." You: "What could be improved?" LLM: "Here's some different stuff based on other training data with embeddings close to the original embeddings, but different. --- It's near zero useful information. Example imformation might be "it builds" (baseline necessity, so useless info), "it passes some tests" (fairly baseline, more useful, but actually useless if you don't know what the tests are doing), or "it's different" (duh).
    - deleted 9mo ago
      [deleted]
    - manmal 9mo ago
      If I asked you, about a piece of code that you didn’t build, „What would you improve?“, how would that be fundamentally different?
      - 8mo ago
torben-friis 9mo ago
>At some point you have to let go and trust people‘s judgement. Indeed. People. With salaries, general intelligence, a stake in the matter and a negative outcome if they don’t take responsibility. >Reading and understanding the whole output of 9 concurrently running agents is impossible. I agree. It is also impossible for a person to drive two cars at once… so we don’t. Why is the starting point of the conversation that one should be able to use 9 concurring agents? I get it, writing code no longer has a physical bottleneck. So the bottleneck becomes the next thing, which is our ability to review outputs. It’s already a giant advancement, why are we ignoring that second bottleneck and dropping quality assurance as well? Eventually someone has to put their signature on the thing being shippable.
- deleted 9mo ago
  [deleted]
- wtetzner 9mo ago
  Is reviewing outputs really more efficient than writing the code? Especially if it's a code base you haven't written code in?
  - kuschku 9mo ago
    It is not. To review code you need to have an understanding of the problem that can only be built by writing code. Not necessarily the final product, but at least prototypes and experiments that then inform the final product.
ink_13 9mo ago
An AI agent cannot be held accountable
- manmal 9mo ago
  Neither can employees, in many countries.

CharlesW 9mo ago

AI-assisted coding is not a black box in the way that managing an engineering team of humans is. You see the model "thinking", you see diffs being created, and occasionally you intervene to keep things on track. If you're leveraging AI professionally, any coding has been preceded by planning (the breadth and depth of which scale with the task) and test suites.

csallen 9mo ago

> Imagine taking a picture on autoshot mode and refusing to look at it. If the client doesn’t like it because it’s too bright, tweak the settings and shoot again, but never look at the output. The output of code isn't just the code itself, it's the product. The code is a means to an end. So the proper analogy isn't the photographer not looking at the photos, it's the photographer not looking at what's going on under the hood to produce the photos. Which, of course, is perfectly common and normal.

alanbernstein 9mo ago
Right, it seems the appropriate analogy is the shift from analog-photograph-developers to digital camera photographers.
torben-friis 9mo ago
>The output of code isn't just the code itself, it's the product. The code is a means to an end. I’ll bite. Is this person manually testing everything that one would regularly unit test? Or writing black box tests that he does know are correct because of being manually written? If not, you’re not reviewing the product either. If yes, it’s less time consuming to actually read and test the damn code
- CuriouslyC 9mo ago
  I mostly ignore code, I lean on specs + tests + static analysis. I spot check tests depending on how likely I think it is for the agent to have messed up or misinterpreted my instructions. I push very high test coverage on all my projects (85%+), and part of the way I build is "testing ladders" where I have the agent create progressively bigger integration tests, until I hit e2e/manual validation.
  - torben-friis 9mo ago
    >I spot check tests depending on how likely I think it is for the agent to have messed up or misinterpreted my instructions So a percentage of your code, based on your gut feeling, is left unseen by any human by the moment you submit it. Do you agree that this rises the chance of bugs slipping by? I don’t see how you wouldn’t. And considering the fact that your code output is larger, the percentage of it that is buggy is larger, and (presumably) you write faster, have you considered the conclusion in terms of the compounding likelihood of incidents?

notepad0x90 9mo ago

people care about results. Better processes need to produce better results. this is programming not a belief system where you have to adhere to some view or else.

Aeolun 9mo ago

> What is the logic here? It is right often enough that your time is better spent testing the functionality than the code. Sometimes it’s not right, and you need to re-instruct (often) or dive in (not very often).

torben-friis 9mo ago
I can’t imagine retesting all the functionality of a well established product for possible regressions not being stupidly time consuming. This is the very reason why we have unit tests in the first place, and why they are far more numerous in tests than end-to-end ones.

Xirdus 9mo ago

> I can’t imagine any other example where people voluntarily move for a black box approach. I can think of a few. The last 78 pages of any 80-page business analysis report. The music tracks of those "12 hours of chill jazz music" YouTube videos. Political speeches written ahead of time. Basically - anywhere that a proper review is more work than the task itself, and the quality of output doesn't matter much.

ink_13 9mo ago
So... things where the producer doesn't respect the audience? Because any such analysis would be worth as much as a 4.5 hour atonal bass solo.
- sroerick 9mo ago
  You can get an AI to listen to that bass solo for you
  - just6979 9mo ago
    But can you get an AI to zone out on a fluffy couch at the center point of a dank hi-fi setup with the volume cranked to 11, while chillin' on 50mg of THC? And will you enjoy paying someone else to let the AI to do that?

ForHackernews 9mo ago

>Imagine taking a picture on autoshot mode Almost everyone does this. Hardly anyone taking pictures understands what f-stop or focal length are. Even those who do seldom adjust them. There dozens of other examples where people voluntarily move to a black box approach. How many Americans drive a car with a manual transmission?

weikju 9mo ago
You missed out on the rest of the analogy though, which is the part where the photo is not reviewed before handing it over to the client.
sigseg1v 9mo ago
Hey it's me! I shoot with manual focus lenses in RAW and drive a standard. There are dozens of us!

weikju 9mo ago

Don’t read the code, test for desired behavior, miss out on all the hidden undesired behavior injected by malicious prompts or AI providers. Brave new world!

thefz 9mo ago
You made me imagine AI companies maliciously injecting backdoors in generated code no one reads, and now I'm scared.
- gibsonsmog 9mo ago
  My understanding is that it's quite easy to poison the models with inaccurate data, I wouldn't be surprised if this exact thing has happened already. Maybe not an AI company itself, but it's definitely in the purview of a hostile actor to create bad code for this purpose. I suppose it's kind of already happened via supply chain attacks using AI generated package names that didn't exist prior to the LLM generating them.
- djeastm 9mo ago
  One mitigation might be to use one company's model to check the work of another company's code and depend on market competition to keep the checks and balances.
  - thefz 9mo ago
    What about writing the actual code yourself
    - just6979 9mo ago
      Nah, more fun to burn money.
  - just6979 9mo ago
    Then how many models deep do you go before it's more cost effective to just hire a junior dev, supply them with a list of common backdoors, and have them scan the code?

raincole 9mo ago

> Because if you can read code, I can’t imagine poking the result with black box testing being faster. I don't know... it depends on the use case. I can't imagine even the best front-end engineer ever can read HTML faster than looking at the rendered webpage to check if the layout is correct.

nubg 9mo ago
Good analogy.

seanmcdirmid 9mo ago

> What is the logic here? Because if you can read code, I can’t imagine poking the result with black box testing being faster. The AI also writes the black box tests, what am I missing here?

torben-friis 9mo ago
>The AI also writes the black box tests, what am I missing here? If the AI misinterpreted your intentions and/or missed something in productive code, tests are likely to reproduce rather than catch that behavior. In other words, if “the ai is checking as well” no one is.
- seanmcdirmid 9mo ago
  That's true. Never let the AI know about the code it wrote when writing the test for sure. Write multiple tests, have an arbitrator (also AI) figure out if implementation or tests are wrong when tests fail. Have the AI heavily comment code and heavily comment tests in the language of your spec so you can manually verify if the scenarios/parts of the implementations make sense when it matters. etc...etc... > In other words, if “the ai is checking as well” no one is. "I tried nothing, and nothing at all worked!"

hjoutfbkfd 9mo ago

your metaphor is wrong. code is not the output. functionality is the output, and you do look at that.

torben-friis 9mo ago
Explain then how testing the functionality (not the new one; regressions included, this is not a school exercise) is faster than checking the code. Are you writing black box testing by hand, or manually checking, everything that would normally be a unit test? We have unit tests precisely because of how unworkable the “every test is black box” approach is.

straydusk 9mo ago

No pun intended but - it's been more "vibes" than science that I've done this. It's more effective. When I focus my attention on the harness layer (tests, hooks, checks, etc), and the inputs, my overall velocity improves relative to reading & debugging the code directly. To be fair - it is not accurate to say I absolutely never read the code. It's just rare, and it's much more the exception than the rule. My workflow just focuses much more on the final product, and the initial input layer, not the code - it's becoming less consequential.

AlexCoventry 9mo ago

> What is the logic here? Because if you can read code, I can’t imagine poking the result with black box testing being faster. It's producing seemingly working code faster than you can closely review it.

torben-friis 9mo ago
Your car can also move faster than what you can safely control. Knowing this, why go pedal to the metal?

eikenberry 9mo ago

I think many people are missing the overall meaning of these sorts of posts.. that is they are describing a new type of programmer that will only use agents and never read the underlying code. These vibe/agent coders will use natural(-ish) language to communicate with the agents and wouldn't look at the code anymore than, say, a PHP developer would look at the underlying assembly. It is not the level of abstraction they are working on. There are many use cases where this type of coding will work fine and it will let many people who previously couldn't really take advantage of computers to do so. This is great but in no way will do anything to replace the need for code that requires humans to understand (which, in turn, requires participation in the writing).

straydusk 9mo ago
I'm glad you wrote this comment because I completely agree with it. I don't think that there is not a need for software engineers to deeply consider architecture; who can fully understand the truly critical systems that exist at most software companies; who can help dream up the harness capabilities to make these agents work better. I just am describing what I'm doing now, and what I'm seeing at the leading edge of using these tools. It's a different approach - but I think it'll become the most common way of producing software.
re-thc 9mo ago
> that is they are describing a new type of programmer that will only use agents and never read the underlying code > and wouldn't look at the code anymore than, say, a PHP developer would look at the underlying assembly This really puts down the work that the PHP maintainers have done. Many people spend a lot of time crafting the PHP codebase so you don't have to look at the underlying assembly. There is a certain amount of trust that I as a PHP developer assume. Is this what the agents do? No. They scrape random bits of code everywhere and put something together with no craft. How do I know they won't hide exploits somewhere? How do I know they don't leak my credentials?
- 6510 9mo ago
  That is true for all languages. Very high quality until you use a lib, a module or an api.

bloomca 9mo ago

I think this is the logical next step -- instead of manually steering the model, just rely on the acceptance criteria and some E2E test suite (that part is tricky since you need to verify that part). I personally think we are not that far from it, but it will need something built on top of current CLI tools.

andyferris 9mo ago

The output is the program behavior. You use it, like a user, and give feedback to the coding agent. If the app is too bright, you tweak the settings and build it again. Photography used to involve developing film in dark rooms. Now my iPhone does... god knows what to the photo - I just tweak in post, or reshoot. I _could_ get the raw, understand the algorithm to transform that into sRGB, understand my compression settings, etc - but I don't need to. Similarly, I think there will be people who create useful software without looking at what happens in between. And there will still be low-level software engineers for whom what happens in between is their job.

vidarh 9mo ago

Your product managers most likely are not reading your code. Your CEO is not. The vast majority of your company is unlikely to ever look at a line of code. If the process becomes reliable enough, then there is no reason. For now, that still requires developers to pay attention for important projects, but there are also a lot of AI written tools I rely on day to day that I don't, because the opportunity cost of spending time reading them is lower than the cost of accepting the risk that they do something wrong. There are also a whole lot of tools I do read thoroughly, because the risk profile is different. But that category is getting smaller day by day, not just with model improvements, but with improved harnesses.

llamajams 9mo ago

Same. I stopped reading after that. I get the sense that most of these people thing all code is web or mobile or something non critical. Granted im not a web or mobile guy so I cant presume the complexity, risk, cost of such things. But I assume its in a different category than safety/mission critical things. I do dev tools for ASIL-B systems devs now and even then I cant say im comfortable not reading the generated code. Some of my junior peers are though, and im very frustrated that I feel like I keep having to play AI janitor, dont think the bosses care.

laterium 9mo ago

You generate 20k LOC in a few hours. How long will it take you to read it? One week? You just keep going instead. I don't think this works great at large scale production codebases yet, but it's an approach that will have more and more applications going forward. It doesn't have to fit every use case.