6 ms·

You have to hold AI hand to do even simple vanilla JS correctly. Or do framework code which is well documented all over the net. I love AI and use it for prog

You have to hold AI hand to do even simple vanilla JS correctly. Or do framework code which is well documented all over the net. I love AI and use it for programming a lot, but the limitations are real.

deleted 7mo ago
[deleted]
keeganpoppen 7mo ago
that's just not even remotely my experience. and i am ~20k hours into my programming career. ai makes most things so much faster that it is hard to justify ever doing large classes of things yourself (as much as this hurts my aesthetic sensibilities, it simply is what it is).
- lumost 7mo ago
  Part of this depends on if you care that the AI wrote the code "your way." I've been in shops with rather exotic and specific style guides and standards which the AI would not or will not conform to.
  - localhost 7mo ago
    then have ai write a deterministic transformation tool that turns it into the specific style and standard that is needed
  - igor47 7mo ago
    Yeah, I also highly value consistency in my projects, which forces me to keep an eye on the LLM and steer it often. This limits my overall velocity especially on larger features. But I'm still much faster with the agent. Recent example, https://github.com/igor47/csheet/pull/68 https://github.com/igor47/csheet/pull/68 -- this took me a couple of hours pairing with Claude code, which is insane give the size of the work here. Though this PR creates a bunch of tables, routes, services -- it's not just greenfield CRUD work. We're figuring out how to model a complicated domain, integrating with existing code, thinking through complex integrations including with LLMs at run time. Claude is writing almost all the code, I'm just steering
- leptons 7mo ago
  I've never seen a human estimate their "programming career" in kilohours. Is that supposed to look more impressive than years? So, you've been programming only about 7 years? I guess I'm at about "170 kilohours".
  - kennywinker 7mo ago
    I think it’s probabky because of the malcom gladwell “ten thousand hours” idea.
  - ralferoo 7mo ago
    As well as the peer comment about Gladwell (10k hours is considered the point you've mastered a skill), it's also a far more honest metric about how much time you've spent actually programming. Maybe you were writing code, make design choices and debugging 8 hours a day. Maybe you were primarily doing something else and only writing code for an hour a day. Who would be the better programmer? The first guy with one year of experience or the second guy with 7 years? I personally would only measure my experience in years, because it's approaching 3 decades full-time in industry (plus an additional decade of cutting my teeth during school and university), but I can certainly see that earlier on in a career it's a useful metric in comparison to the 10,000 hours.
    - hrimfaxi 7mo ago
      > Maybe you were writing code, make design choices and debugging 8 hours a day. Maybe you were primarily doing something else and only writing code for an hour a day. Who would be the better programmer? The first guy with one year of experience or the second guy with 7 years? So your logic is that the grandparent specified hours because they spent that many hours specifically programming, and not by just multiplying the number of years by the number of hours in a year?
      - ralferoo 7mo ago
        I don't know exactly how they arrived at their 20k hours figure, all I'm saying is that it didn't seem a controversial way of expressing their experience level, and assumed it was intended to be a comparison to the typical 10k hours needed for mastery of a craft.
GalaxyNova 7mo ago
Not what I've experienced
seanmcdirmid 7mo ago
Not in my experience. But then again, lots of programmers are limited in how they use AI to write code. Those limitations are definitely real.
sp00chy 7mo ago
Exactly that is also my experience also with Claude Code. It can create a lot of stuff impressively but with LOTS of more code than necessary. It’s not really effective in the end. I have more than 35 years of coding experience and always dig into the newest stuff. Quality wise it’s still not more than junior dev stuff even with latest models, sorry. And I know how to talk to these machines.
- TuxSH 7mo ago
  I don't have as many years of professional experience as you do, but IMO code pissing is one of the areas LLMs and "agentic tools" shine the least. In both personal projects and $dayjob tasks, the highest time-saving AI tasks were: - "review this feature branch" (containing hand-written commits) - "trace how this repo and repo located at ~/foobar use {stuff} and how they interact with each other, make a Mermaid diagram" - "reverse engineer the attached 50MiB+ unstripped ELF program, trace all calls to filesystem functions; make a table with filepath, caller function, overview of what caller does" (the table is then copy-pasted to Confluence) - basic YAML CRUD Also while Anthropic has more market share in B2B, their model seems optimized for frontend, design, and literary work rather than rigorous work; I find it to be the opposite with their main competitor. Claude writes code rife with safety issues/vulns all the time, or at least more than other models.
- iamflimflam1 7mo ago
  Try the new /simplify command.
wek 7mo ago
This is not my experience either. If you put the work in upfront to plan the feature, write the test cases, and then loop until they pass... you can build a lot of high quality software quickly. The difference between a junior engineer using it and a great architect using it is significant. I think of it as an amplifier.
- Mars008 7mo ago
  > The difference between a junior engineer using it and a great architect using it is significant Yes, juniors are trying to use AI with the minimum input. This alone tells a lot..
- bluefirebrand 7mo ago
  This honestly reads to me like "if you spend a lot of time doing tedious monotonous shit you can save a lot of time on the interesting stuff" I have no interest being a "great architect" if architects don't actually build anything
  - hrimfaxi 7mo ago
    "If I had eight hours to chop down a tree, I'd spend six sharpening my axe" - Abraham Lincoln
- andrekandre 7mo ago
  > If you put the work in upfront to plan the feature, write the test cases, and then loop until they pass... it can be exhausting and time consuming front-loading things so deeply though; sometimes i feel like i would have been faster cutting all that out and doing it myself because in the doing you discover a lot of missing context (in the spec) anyways...
- grey-area 7mo ago
  I’m amazed at how many great architects and experts on AI we now have.
moezd 7mo ago
AI assisted code can't even stick to the API documentation, especially if the data structures are not consistent and have evolved over time. You would see Claude literally pulling function after function from thin air, desperately trying to fulfill your complicated business logic and even when it's complete, it doesn't look neat at all. Yes, it will have test coverage, but one more feature request will probably break the back of the camel. And if you raise that PR to the rest of your team, good luck trying to summarise it all to your colleagues. However if you just have an easy project, or a greenfield project, or don't care about who's going to maintain that stuff in 6 months, sure, go all in with AI.
- ccosky 7mo ago
  I definitely wonder if the people going all-in on AI harnessing are working on greenfield projects, because it seems overwhelming to try to get that set up on a brownfield codebase where the patterns aren't consistent and the code quality is mixed.
- tayo42 7mo ago
  So just iterate on it? Your complaint is that the model isn't one shotting the problem and reading your mind about style. It's like any coding workflow, make it work, then make it nice.
  - moezd 7mo ago
    No, I never expect AI to one-shot (if I see such a miracle, it's usually because I needed a one-liner or something really simple and well documented, which I can also write on the whiteboard from memory). Try iterating over well known APIs where the response payloads are already gigantic JSONs, there are multiple ways to get certain data and they are all inconsistent and Claude spits out function after function, laying waste to your codebase. I found no amount of style guideline documents to resolve this issue. I'd rather read the documentation myself and write the code by hand rather than reviewing for the umpteenth time when Claude splits these new functions between e.g. __init__.py and main.py and god knows where, mixing business logic with plumbing and transport layers as an art form. God it was atrocious during the first few months of FastMCP.
jcranmer 7mo ago
I must say, I do love how this comment has provoked such varying responses. My own observations about using AI to write code is that it changes my position from that of an author to a reviewer. And I find code review to be a much more exhausting task than writing code in the first place, especially when you have to work out how and why the AI-generated code is structured the way it is.
- thegrim33 7mo ago
  There's a very wide range of programming tasks of differing difficulty that people are using / trying to use it for, and a very wide range of intelligence amongst the people that are using / trying to use it, and who are evaluating its results. Hence, different people have very different takes.
- seanmcdirmid 7mo ago
  > especially when you have to work out how and why the AI-generated code is structured the way it is. You could just ask it? Or you don’t trust the AI to answer you honestly?
  - chmod775 7mo ago
    You're anthropomorphizing. LLMs can't lie nor can they tell the truth. These concepts just don't apply to them. They also cannot tell you what they were "thinking" when they wrote a piece of code. If you "ask" them what they were thinking, you just get a plausible response, not the "intention" that may or may not have existed in some abstract form in some layer when the system selected tokens*. That information is gone at that point and the LLM has no means to turn that information into something a human could understand anyways. They simply do not have what in a human might be called metacognition. For now. There's lots of ongoing experimental research in this direction though. Chances are that when you ask an LLM about their output, you'll get the response of either someone who now recognized an issue with their work, or the likeness of someone who believes they did great work and is now defending it. Obviously this is based on the work itself being fed back through the context window, which will inform the response, and thus it may not be entirely useless, but... this is all very far removed from what a conscious being might explain about their thoughts. The closest you can currently get to this is reading the "reasoning" tokens, though even those are just some selected system output that is then fed back to inform later output. There's nothing stopping the system from "reasoning" that it should say A, but then outputting B. Example: https://i.imgur.com/e8PX84Z.png https://i.imgur.com/e8PX84Z.png * One might say that the LLM itself always considers every possible token and assigns weights to them, so there wouldn't even be a single chain of thought in the first place. More like... every possible "thought" at the same time at varying intensities.
fudfomo 7mo ago
Most of this thread is debating whether models are good or bad at writing code... however, I think a more important question is what we feed the AI with because that dramatically determines the quality of the output. When your agent explores your codebase trying to understand what to build, it read schema files, existing routes, UI components etc... easily 50-100k tokens of implementation detail. It's basically reverse-engineering intent from code. With that level of ambiguous input, no wonder the results feel like junior work. When you hand it a structured spec instead including data model, API contracts, architecture constraints etc., the agent gets 3-5x less context at much higher signal density. Instead of guessing from what was built it knows exactly what to build. Code quality improves significantly. I've measured this across ~47 features in a production codebase with amedian ratio: 4x less context with specs vs. random agent code exploration. For UI-heavy features it's 8-25x. The agent reads 2-3 focused markdown files instead of grepping through hundreds of KB of components. To pick up @wek's point about planning from above: devs who get great results from agentic development aren't better prompt engineers... they're better architects. They write the spec before the code, which is what good engineering always was... AI just made the payoff for that discipline 10x more visible.
xtracto 7mo ago
The other day I (well, the AI) just wrote a Rust app to merge two (huge, GB of data) tables by discovering columns with data in common based on text distance (levenshtein and Dice) . It worked beautifully An i have NEVER made one line of Rust. I dont understand nay-sayers, to me the state of gen.AI is like the simpsons quote "worst day so far". Look were we are within 5 years of the first real GPT/LLM. The next 5 years are going to be crazy exciting. The "programmer" position will become a "builder". When we've got LLMs that generate Opus quality text at 100x speed (think, ASIC based models) , things will get crazy.
- npinsker 7mo ago
  Human minds are built to find patterns, and you should be careful not to assume the rate of improvement will continue forever based on nothing but a pattern.
  - fauchletenerum 7mo ago
    The overall trend in AI performance will still be up and to the right like everything else in computing over the past 50 years, improvement doesn't have to be linear
    - swingboy 7mo ago
      Assuming newer, more efficient architectures are discovered.
  - throwawaytea 7mo ago
    Just the fact that even retail quality hardware is still improving at local LLM significantly is still a great sign. If AI quality remained the same, and the cost for local hardware dropped to $1000, it would still be the greatest thing since the internet IMO. So even if the worst happens and all progress stops, I'm still very happy with what we got.
    - leptons 7mo ago
neversupervised 7mo ago
It’s crazy how some people feel the ai and others don’t. But one group is wrong. It’s a matter of time before everyone feels the AI.

deleted 7mo ago

[deleted]

keeganpoppen 7mo ago

that's just not even remotely my experience. and i am ~20k hours into my programming career. ai makes most things so much faster that it is hard to justify ever doing large classes of things yourself (as much as this hurts my aesthetic sensibilities, it simply is what it is).

lumost 7mo ago
Part of this depends on if you care that the AI wrote the code "your way." I've been in shops with rather exotic and specific style guides and standards which the AI would not or will not conform to.
- localhost 7mo ago
  then have ai write a deterministic transformation tool that turns it into the specific style and standard that is needed
- igor47 7mo ago
  Yeah, I also highly value consistency in my projects, which forces me to keep an eye on the LLM and steer it often. This limits my overall velocity especially on larger features. But I'm still much faster with the agent. Recent example, https://github.com/igor47/csheet/pull/68 https://github.com/igor47/csheet/pull/68 -- this took me a couple of hours pairing with Claude code, which is insane give the size of the work here. Though this PR creates a bunch of tables, routes, services -- it's not just greenfield CRUD work. We're figuring out how to model a complicated domain, integrating with existing code, thinking through complex integrations including with LLMs at run time. Claude is writing almost all the code, I'm just steering
leptons 7mo ago
I've never seen a human estimate their "programming career" in kilohours. Is that supposed to look more impressive than years? So, you've been programming only about 7 years? I guess I'm at about "170 kilohours".
- kennywinker 7mo ago
  I think it’s probabky because of the malcom gladwell “ten thousand hours” idea.
- ralferoo 7mo ago
  As well as the peer comment about Gladwell (10k hours is considered the point you've mastered a skill), it's also a far more honest metric about how much time you've spent actually programming. Maybe you were writing code, make design choices and debugging 8 hours a day. Maybe you were primarily doing something else and only writing code for an hour a day. Who would be the better programmer? The first guy with one year of experience or the second guy with 7 years? I personally would only measure my experience in years, because it's approaching 3 decades full-time in industry (plus an additional decade of cutting my teeth during school and university), but I can certainly see that earlier on in a career it's a useful metric in comparison to the 10,000 hours.
  - hrimfaxi 7mo ago
    > Maybe you were writing code, make design choices and debugging 8 hours a day. Maybe you were primarily doing something else and only writing code for an hour a day. Who would be the better programmer? The first guy with one year of experience or the second guy with 7 years? So your logic is that the grandparent specified hours because they spent that many hours specifically programming, and not by just multiplying the number of years by the number of hours in a year?
    - ralferoo 7mo ago
      I don't know exactly how they arrived at their 20k hours figure, all I'm saying is that it didn't seem a controversial way of expressing their experience level, and assumed it was intended to be a comparison to the typical 10k hours needed for mastery of a craft.

GalaxyNova 7mo ago

Not what I've experienced

seanmcdirmid 7mo ago

Not in my experience. But then again, lots of programmers are limited in how they use AI to write code. Those limitations are definitely real.

sp00chy 7mo ago

Exactly that is also my experience also with Claude Code. It can create a lot of stuff impressively but with LOTS of more code than necessary. It’s not really effective in the end. I have more than 35 years of coding experience and always dig into the newest stuff. Quality wise it’s still not more than junior dev stuff even with latest models, sorry. And I know how to talk to these machines.

TuxSH 7mo ago
I don't have as many years of professional experience as you do, but IMO code pissing is one of the areas LLMs and "agentic tools" shine the least. In both personal projects and $dayjob tasks, the highest time-saving AI tasks were: - "review this feature branch" (containing hand-written commits) - "trace how this repo and repo located at ~/foobar use {stuff} and how they interact with each other, make a Mermaid diagram" - "reverse engineer the attached 50MiB+ unstripped ELF program, trace all calls to filesystem functions; make a table with filepath, caller function, overview of what caller does" (the table is then copy-pasted to Confluence) - basic YAML CRUD Also while Anthropic has more market share in B2B, their model seems optimized for frontend, design, and literary work rather than rigorous work; I find it to be the opposite with their main competitor. Claude writes code rife with safety issues/vulns all the time, or at least more than other models.
iamflimflam1 7mo ago
Try the new /simplify command.

wek 7mo ago

This is not my experience either. If you put the work in upfront to plan the feature, write the test cases, and then loop until they pass... you can build a lot of high quality software quickly. The difference between a junior engineer using it and a great architect using it is significant. I think of it as an amplifier.

Mars008 7mo ago
> The difference between a junior engineer using it and a great architect using it is significant Yes, juniors are trying to use AI with the minimum input. This alone tells a lot..
bluefirebrand 7mo ago
This honestly reads to me like "if you spend a lot of time doing tedious monotonous shit you can save a lot of time on the interesting stuff" I have no interest being a "great architect" if architects don't actually build anything
- hrimfaxi 7mo ago
  "If I had eight hours to chop down a tree, I'd spend six sharpening my axe" - Abraham Lincoln
andrekandre 7mo ago
> If you put the work in upfront to plan the feature, write the test cases, and then loop until they pass... it can be exhausting and time consuming front-loading things so deeply though; sometimes i feel like i would have been faster cutting all that out and doing it myself because in the doing you discover a lot of missing context (in the spec) anyways...
grey-area 7mo ago
I’m amazed at how many great architects and experts on AI we now have.

moezd 7mo ago

AI assisted code can't even stick to the API documentation, especially if the data structures are not consistent and have evolved over time. You would see Claude literally pulling function after function from thin air, desperately trying to fulfill your complicated business logic and even when it's complete, it doesn't look neat at all. Yes, it will have test coverage, but one more feature request will probably break the back of the camel. And if you raise that PR to the rest of your team, good luck trying to summarise it all to your colleagues. However if you just have an easy project, or a greenfield project, or don't care about who's going to maintain that stuff in 6 months, sure, go all in with AI.

ccosky 7mo ago
I definitely wonder if the people going all-in on AI harnessing are working on greenfield projects, because it seems overwhelming to try to get that set up on a brownfield codebase where the patterns aren't consistent and the code quality is mixed.
tayo42 7mo ago
So just iterate on it? Your complaint is that the model isn't one shotting the problem and reading your mind about style. It's like any coding workflow, make it work, then make it nice.
- moezd 7mo ago
  No, I never expect AI to one-shot (if I see such a miracle, it's usually because I needed a one-liner or something really simple and well documented, which I can also write on the whiteboard from memory). Try iterating over well known APIs where the response payloads are already gigantic JSONs, there are multiple ways to get certain data and they are all inconsistent and Claude spits out function after function, laying waste to your codebase. I found no amount of style guideline documents to resolve this issue. I'd rather read the documentation myself and write the code by hand rather than reviewing for the umpteenth time when Claude splits these new functions between e.g. __init__.py and main.py and god knows where, mixing business logic with plumbing and transport layers as an art form. God it was atrocious during the first few months of FastMCP.

jcranmer 7mo ago

I must say, I do love how this comment has provoked such varying responses. My own observations about using AI to write code is that it changes my position from that of an author to a reviewer. And I find code review to be a much more exhausting task than writing code in the first place, especially when you have to work out how and why the AI-generated code is structured the way it is.

thegrim33 7mo ago
There's a very wide range of programming tasks of differing difficulty that people are using / trying to use it for, and a very wide range of intelligence amongst the people that are using / trying to use it, and who are evaluating its results. Hence, different people have very different takes.
seanmcdirmid 7mo ago
> especially when you have to work out how and why the AI-generated code is structured the way it is. You could just ask it? Or you don’t trust the AI to answer you honestly?
- chmod775 7mo ago
  You're anthropomorphizing. LLMs can't lie nor can they tell the truth. These concepts just don't apply to them. They also cannot tell you what they were "thinking" when they wrote a piece of code. If you "ask" them what they were thinking, you just get a plausible response, not the "intention" that may or may not have existed in some abstract form in some layer when the system selected tokens*. That information is gone at that point and the LLM has no means to turn that information into something a human could understand anyways. They simply do not have what in a human might be called metacognition. For now. There's lots of ongoing experimental research in this direction though. Chances are that when you ask an LLM about their output, you'll get the response of either someone who now recognized an issue with their work, or the likeness of someone who believes they did great work and is now defending it. Obviously this is based on the work itself being fed back through the context window, which will inform the response, and thus it may not be entirely useless, but... this is all very far removed from what a conscious being might explain about their thoughts. The closest you can currently get to this is reading the "reasoning" tokens, though even those are just some selected system output that is then fed back to inform later output. There's nothing stopping the system from "reasoning" that it should say A, but then outputting B. Example: https://i.imgur.com/e8PX84Z.png https://i.imgur.com/e8PX84Z.png * One might say that the LLM itself always considers every possible token and assigns weights to them, so there wouldn't even be a single chain of thought in the first place. More like... every possible "thought" at the same time at varying intensities.

fudfomo 7mo ago

Most of this thread is debating whether models are good or bad at writing code... however, I think a more important question is what we feed the AI with because that dramatically determines the quality of the output. When your agent explores your codebase trying to understand what to build, it read schema files, existing routes, UI components etc... easily 50-100k tokens of implementation detail. It's basically reverse-engineering intent from code. With that level of ambiguous input, no wonder the results feel like junior work. When you hand it a structured spec instead including data model, API contracts, architecture constraints etc., the agent gets 3-5x less context at much higher signal density. Instead of guessing from what was built it knows exactly what to build. Code quality improves significantly. I've measured this across ~47 features in a production codebase with amedian ratio: 4x less context with specs vs. random agent code exploration. For UI-heavy features it's 8-25x. The agent reads 2-3 focused markdown files instead of grepping through hundreds of KB of components. To pick up @wek's point about planning from above: devs who get great results from agentic development aren't better prompt engineers... they're better architects. They write the spec before the code, which is what good engineering always was... AI just made the payoff for that discipline 10x more visible.

xtracto 7mo ago

The other day I (well, the AI) just wrote a Rust app to merge two (huge, GB of data) tables by discovering columns with data in common based on text distance (levenshtein and Dice) . It worked beautifully An i have NEVER made one line of Rust. I dont understand nay-sayers, to me the state of gen.AI is like the simpsons quote "worst day so far". Look were we are within 5 years of the first real GPT/LLM. The next 5 years are going to be crazy exciting. The "programmer" position will become a "builder". When we've got LLMs that generate Opus quality text at 100x speed (think, ASIC based models) , things will get crazy.

npinsker 7mo ago
Human minds are built to find patterns, and you should be careful not to assume the rate of improvement will continue forever based on nothing but a pattern.
- fauchletenerum 7mo ago
  The overall trend in AI performance will still be up and to the right like everything else in computing over the past 50 years, improvement doesn't have to be linear
  - swingboy 7mo ago
    Assuming newer, more efficient architectures are discovered.
- throwawaytea 7mo ago
  Just the fact that even retail quality hardware is still improving at local LLM significantly is still a great sign. If AI quality remained the same, and the cost for local hardware dropped to $1000, it would still be the greatest thing since the internet IMO. So even if the worst happens and all progress stops, I'm still very happy with what we got.
  - leptons 7mo ago

neversupervised 7mo ago

It’s crazy how some people feel the ai and others don’t. But one group is wrong. It’s a matter of time before everyone feels the AI.