4 ms·

Everytime I read something like this, I question myself, what I am doing wrong ? And I tried all kinds of AI tools. But I am not even close to claiming that AI

Everytime I read something like this, I question myself, what I am doing wrong ? And I tried all kinds of AI tools. But I am not even close to claiming that AI writes 50% of my code. My work which sometimes include feature enhancements and maintenance is where I get even less. I have to be extremely careful and make sure nothing unwanted or addition that I am unaware of has been added. Maybe it's me and I am not good yet to get to 100% AI code generation.

fhennig 1y ago
I'm in the same boat, I still find the model to make mistakes or solve things in a less than ideal way - maybe the future is to just not care - but for now I want to maintain the level of quality that the codebase currently has. I think it's good to keep up with what early adopters are doing, but I'm not too fussed about missing something. The plugins is a good example: A few weeks ago there was a post on HN where someone said they are using 18 or 25 or whatever plugins and it's the future, now this person says they are using none. I'm still waiting for the dust to settle, I'm not in a rush.
- CuriouslyC 1y ago
  The person using 25 plugins is giving bad advice. The agent isn't going to need all those tools at once, and each tool burns context and causes tool confusion. Enable MCPs for the specific task you're going to have your agent do. The trick is to create deterministic hurdles the LLM has to jump over. Tests, linting, benchmarks, etc. You can even do this with diff size to enforce simpler code, tell an agent to develop a feature and keep the character count of the diff below some threshold, and it'll iterate on pruning the solution.
troupo 1y ago
You're not doing anything wrong. You have to read past hyperbole. Here's how the article starts: "Agentic engineering has become so good that it now writes pretty much 100% of my code. And yet I see so many folks trying to solve issues and generating these elaborated charades instead of getting sh*t done." Here's how it continues: - I run between 3-8 in parallel - My agents do git atomic commits, I iterated a lot on the agents file: https://gist.github.com/steipete/d3b9db3fa8eb1d1a692b7656217d8655 https://gist.github.com/steipete/d3b9db3fa8eb1d1a692b7656217... - I currently have 4 OpenAI subs and 1 Anthropic sub, so my overall costs are around 1k/month for basically unlimited tokens. - My current approach is usually that I start a discussion with codex, I paste in some websites, some ideas, ask it to read code, and we flesh out a new feature together. - If you do a bigger refactor, codex often stops with a mid-work reply. Queue up continue messages if you wanna go away and just see it done - When things get hard, prompting and adding some trigger words like “take your time” “comprehensive” “read all code that could be related” “create possible hypothesis” makes codex solve even the trickiest problems. - My Agent file is currently ~800 lines long and feels like a collection of organizational scar tissue. I didn’t write it, codex did. It's the same magical incantations and elaborated charades as everyone does. The "the no-bs Way of Agentic Engineering" is full of bs and has nothing concrete except a single link to a bunch of incantations for agents. No idea what his actual "website + tauri app + mobile app" is that he build 100% with AI, but depending on actual functionality, after burning $1000 a month on tokens you may actually have a fully functioning app in React + Typescript with little human supervision.
- 1718627440 1y ago
  > $1000 a month Yeah at this point you could hire a software developer.
  - TheMrZZ 1y ago
    I don't know how much SWE get paid in your area, but I sure hope it's not 1000$/month. Though I'm aligned that I don't (yet) believe in this "AI writes all my code for me" statements.
    - 1718627440 1y ago
      It includes that with AI you still need someone to work. First to query the AI and then to fix up something and to bring it in a form you can release and use.
  - stocksinsmocks 1y ago
    $5.75/hr is well below outsourced rates. It’s $1.40/hr if the agent runs without stopping. If I hired a human consultant for a project of any size, I could easily spend $10,000 or more on just scoping and contract approval. Humans don’t win on cost.
    - 1718627440 1y ago
      Right now they still need someone typing prompts and verifying them. When they do what you intend it means that is no longer more work to handhold them than doing it yourself, but it is still work.
- deleted 1y ago
  [deleted]
- steipete 1y ago
  (OP) You know if I link to a half-finished project, people would take it apart as many don't understand the nuance between crap and simply not done yet. But if you follow me on twitter it'll take you a few minutes to figure out. I'm two months in, even with AI, shipping good stuff takes time.
tptacek 1y ago
Same! Half would be a lot for me. I'm also not close to the point where I'm comfortable merging LLM-authored PRs without line-by-line reviews.
- darkwater 1y ago
  Didn't you write a blog post a few months ago saying that you had agents preparing PRs for you while AFK doing things IRL? Your outlook back then seemed pretty optimistic, while this comment now seems way less so. Did something change for you or had I misunderstood your post back then?
  - tptacek 1y ago
    I still have agents (Sketch.dev mostly) produce PRs offline for me! I'm very optimistic. This is the second most important thing to have happened in my career (#1: the Internet; #3: mobile; #4: not writing everything in C). Nothing has changed. But yeah, I still line-by-line audit everything the agent spits out, and I still take the wheel myself about half the time. If everything stopped right here and no further progress was made on any of this technology and my workflow remained the same, this would remain the second most important thing.
pessimizer 1y ago
> I have to be extremely careful and make sure nothing unwanted or addition that I am unaware of has been added. I've started getting desperate to the point of saying 1) "never. never, ever add or remove features without consulting me first and getting approval." Then eventually, 2) appended to the previous "The last rule is the most important rule, because you keep doing it and I need you to stop doing it." Then finally 3), "THE LAST RULE IS THE MOST IMPORTANT RULE, BECAUSE YOU KEEP DOING IT AND I NEED YOU TO STOP DOING IT." 3/4 of my AI bugs are the AI making changes to the functionality of the code when I'm not looking, or repeatedly reinserting bugs that had been previously removed. The most valuable thing I'm getting it to do is to refactor the code it already wrote into shorter well-named functions (during which it still inevitably adds and removes behavior), because it means that I can just debug by hand and stop demanding over and over again that it not ignore what I said. But, of course, it's not ignoring me, it's not thinking at all. Trying to look for the magic words to keep it from ignoring me and lying about it is just an illusion of control. The thing that will knock it off it's dumb track is likely just a lucky random seed during the 13th attempt. Then, like a sports fan, I add the lucky underwear to my instructions. edit: the "I'll get AI to write the AI prompt so it will be perfect" stuff is so much voodoo. LLMs have no special insight into what will make LLMs work correctly. I probably should have stopped that last sentence after the word "insight." Feed them a sample prompt that you say doesn't work, and it will explain to you exactly why it's so bad, and could never work. Feed them the same prompt and ask why it works so well, and it will tell you how perfectly crafted it is and why. Then it will offer to tell you how it could be improved.
- Kim_Bruning 1y ago
  Hrrrm, do you write unit tests to check for the desired behaviour? Or does it 'optimize' those away too? %-/
  - joshribakoff 1y ago
    AI circumvents guardrails yes.

fhennig 1y ago

I'm in the same boat, I still find the model to make mistakes or solve things in a less than ideal way - maybe the future is to just not care - but for now I want to maintain the level of quality that the codebase currently has. I think it's good to keep up with what early adopters are doing, but I'm not too fussed about missing something. The plugins is a good example: A few weeks ago there was a post on HN where someone said they are using 18 or 25 or whatever plugins and it's the future, now this person says they are using none. I'm still waiting for the dust to settle, I'm not in a rush.

CuriouslyC 1y ago
The person using 25 plugins is giving bad advice. The agent isn't going to need all those tools at once, and each tool burns context and causes tool confusion. Enable MCPs for the specific task you're going to have your agent do. The trick is to create deterministic hurdles the LLM has to jump over. Tests, linting, benchmarks, etc. You can even do this with diff size to enforce simpler code, tell an agent to develop a feature and keep the character count of the diff below some threshold, and it'll iterate on pruning the solution.

troupo 1y ago

You're not doing anything wrong. You have to read past hyperbole. Here's how the article starts: "Agentic engineering has become so good that it now writes pretty much 100% of my code. And yet I see so many folks trying to solve issues and generating these elaborated charades instead of getting sh*t done." Here's how it continues: - I run between 3-8 in parallel - My agents do git atomic commits, I iterated a lot on the agents file: https://gist.github.com/steipete/d3b9db3fa8eb1d1a692b7656217d8655 https://gist.github.com/steipete/d3b9db3fa8eb1d1a692b7656217... - I currently have 4 OpenAI subs and 1 Anthropic sub, so my overall costs are around 1k/month for basically unlimited tokens. - My current approach is usually that I start a discussion with codex, I paste in some websites, some ideas, ask it to read code, and we flesh out a new feature together. - If you do a bigger refactor, codex often stops with a mid-work reply. Queue up continue messages if you wanna go away and just see it done - When things get hard, prompting and adding some trigger words like “take your time” “comprehensive” “read all code that could be related” “create possible hypothesis” makes codex solve even the trickiest problems. - My Agent file is currently ~800 lines long and feels like a collection of organizational scar tissue. I didn’t write it, codex did. It's the same magical incantations and elaborated charades as everyone does. The "the no-bs Way of Agentic Engineering" is full of bs and has nothing concrete except a single link to a bunch of incantations for agents. No idea what his actual "website + tauri app + mobile app" is that he build 100% with AI, but depending on actual functionality, after burning $1000 a month on tokens you may actually have a fully functioning app in React + Typescript with little human supervision.

1718627440 1y ago
> $1000 a month Yeah at this point you could hire a software developer.
- TheMrZZ 1y ago
  I don't know how much SWE get paid in your area, but I sure hope it's not 1000$/month. Though I'm aligned that I don't (yet) believe in this "AI writes all my code for me" statements.
  - 1718627440 1y ago
    It includes that with AI you still need someone to work. First to query the AI and then to fix up something and to bring it in a form you can release and use.
- stocksinsmocks 1y ago
  $5.75/hr is well below outsourced rates. It’s $1.40/hr if the agent runs without stopping. If I hired a human consultant for a project of any size, I could easily spend $10,000 or more on just scoping and contract approval. Humans don’t win on cost.
  - 1718627440 1y ago
    Right now they still need someone typing prompts and verifying them. When they do what you intend it means that is no longer more work to handhold them than doing it yourself, but it is still work.
deleted 1y ago
[deleted]
steipete 1y ago
(OP) You know if I link to a half-finished project, people would take it apart as many don't understand the nuance between crap and simply not done yet. But if you follow me on twitter it'll take you a few minutes to figure out. I'm two months in, even with AI, shipping good stuff takes time.

tptacek 1y ago

Same! Half would be a lot for me. I'm also not close to the point where I'm comfortable merging LLM-authored PRs without line-by-line reviews.

darkwater 1y ago
Didn't you write a blog post a few months ago saying that you had agents preparing PRs for you while AFK doing things IRL? Your outlook back then seemed pretty optimistic, while this comment now seems way less so. Did something change for you or had I misunderstood your post back then?
- tptacek 1y ago
  I still have agents (Sketch.dev mostly) produce PRs offline for me! I'm very optimistic. This is the second most important thing to have happened in my career (#1: the Internet; #3: mobile; #4: not writing everything in C). Nothing has changed. But yeah, I still line-by-line audit everything the agent spits out, and I still take the wheel myself about half the time. If everything stopped right here and no further progress was made on any of this technology and my workflow remained the same, this would remain the second most important thing.

pessimizer 1y ago

> I have to be extremely careful and make sure nothing unwanted or addition that I am unaware of has been added. I've started getting desperate to the point of saying 1) "never. never, ever add or remove features without consulting me first and getting approval." Then eventually, 2) appended to the previous "The last rule is the most important rule, because you keep doing it and I need you to stop doing it." Then finally 3), "THE LAST RULE IS THE MOST IMPORTANT RULE, BECAUSE YOU KEEP DOING IT AND I NEED YOU TO STOP DOING IT." 3/4 of my AI bugs are the AI making changes to the functionality of the code when I'm not looking, or repeatedly reinserting bugs that had been previously removed. The most valuable thing I'm getting it to do is to refactor the code it already wrote into shorter well-named functions (during which it still inevitably adds and removes behavior), because it means that I can just debug by hand and stop demanding over and over again that it not ignore what I said. But, of course, it's not ignoring me, it's not thinking at all. Trying to look for the magic words to keep it from ignoring me and lying about it is just an illusion of control. The thing that will knock it off it's dumb track is likely just a lucky random seed during the 13th attempt. Then, like a sports fan, I add the lucky underwear to my instructions. edit: the "I'll get AI to write the AI prompt so it will be perfect" stuff is so much voodoo. LLMs have no special insight into what will make LLMs work correctly. I probably should have stopped that last sentence after the word "insight." Feed them a sample prompt that you say doesn't work, and it will explain to you exactly why it's so bad, and could never work. Feed them the same prompt and ask why it works so well, and it will tell you how perfectly crafted it is and why. Then it will offer to tell you how it could be improved.

Kim_Bruning 1y ago
Hrrrm, do you write unit tests to check for the desired behaviour? Or does it 'optimize' those away too? %-/
- joshribakoff 1y ago
  AI circumvents guardrails yes.