5 ms·

In case the author is reading this, I have the receipts on how there's a real step function in how much software I build, especially lately. I am not going to p

by larve 1y ago

In case the author is reading this, I have the receipts on how there's a real step function in how much software I build, especially lately. I am not going to put any number on it because that makes no sense, but I certainly push a lot of code that reasonably seems to work. The reason it doesn't show up online is that I mostly write software for myself and for work, with the primary goal of making things better, not faster. More tooling, better infra, better logging, more prototyping, more experimentation, more exploration. Here's my opensource work: https://github.com/orgs/go-go-golems/repositories https://github.com/orgs/go-go-golems/repositories . These are not just one-offs (although there's plenty of those in the vibes/ and go-go-labs/ repositories), but long-lived codebases / frameworks that are building upon each other and have gone through many many iterations.

trenchpilgrim 1y ago
Same. On many days 90% of my code output by lines is Claude generated and things that took me a day now take well under an hour. Also, a good chunk of my personal OSS projects are AI assisted. You probably can't tell from looking at them, because I have strict style guides that suppress the "AI style", and I don't really talk about how I use AI in the READMEs. Do you also expect I mention that I used Intellisense and syntax highlighting too?
- droidjj 1y ago
  The author’s main point is that there hasn’t been an uptick in total code shipped, as you would expect if people are 10x-ing their productivity. Whether folks admit to using AI in their workflow is irrelevant.
  - warkdarrior 1y ago
    Maybe people are working less and enjoying life more, while shipping the same amount of code as before. If someone builds a faster car tomorrow, I am not going to go to the office more often.
    - leoc 1y ago
      "In this economy?", as the saying goes.
    - jplusequalt 1y ago
      Jevon's paradox.
  - trenchpilgrim 1y ago
    The bottleneck on how much I ship has never been how fast I can write and deploy code :)
  - larve 1y ago
    Their main point is "AI coding claims don't add up", as shown by the amount of code shipped. I personally do think some of the more incredible claims about AI coding add up, and am happy to talk about it based on my "evidence", ie the software I am building. 99.99% of my code is ai generated at this point, with the occasional one line I fill in because it'd be stupid to wait for an LLM to do it. For example, I've built 5-6 iphone apps, but they're kind of one-offs and I don't know why I would put them up on the app store, since they only scratch my own itches.
    - trenchpilgrim 1y ago
      Oh yeah, I love building one off tools with it. I am working on a game mod with a friend, we are hand writing the code that runs when you play it, but we vibe code all sorts of dev tools to help us test and iterate on it faster. Do internal, narrow purpose dev tools count as shipped code?
      - daxfohl 1y ago
        This seems to be a common thread. For personal projects where most details aren't important, they are good at meeting the couple things that are important to you and filling in the rest with reasonable, mostly-good-enough guesses. But the more detailed the requirements are, the less filler code there is, and the more each line of code matters. In those situations it's probably faster to type the line of code than to type the English equivalent and hand-hold the assistant through the editing process.
        larve 1y ago
        I don't think so, although I think at that point experience heavily comes into play. With GPT-5 especially, I can basically point cursor/codex at a repo and say "refactor this to this pattern" and come back 25 minutes later to a pretty much impeccable result. In fact that's become my favourite past time lately. I linked some examples higher up, but I've been maintaining a lot of packages that I started slightly before chatgpt and then refactored and worked on as I progressively moved to the "entirely AI generated" workflow I have today. I don't think it's an easy skill (not saying that to make myself look good, I spent an ungodly amount of time exploring programming with LLMs and still do), akin to thinking at a strategic level vs at a "code" level. Certain design patterns also make it much easier to deal with LLM code: state reducers (redux/zustand for example), event-driven architectures, component-based design systems, building many CLI tools that the agent can invoke to iterate and correct things, as do certain "tools" like sqlite/tmux (by that I mean just telling the LLM "btw you can use tmux/sqlite", you allow it to pass hurdles that would otherwise just make it spiral into slop-ratatouille). I also think that a language like go was a really good coincidence, because it is so amenable to LLM-ification.
  - Aeolun 1y ago
    I don’t think this is necessarily true. People that didn’t ship before still don’t ship. My ‘unshipped projects’ backlog is still nearly as large. It’s just got three new entries in the past two months instead of one.
- jplusequalt 1y ago
  >Do you also expect I mention that I used Intellisense and syntax highlighting too? No, but I expect my software to have been verified for correctness, and soundness by a human being with a working mental model of how the code works. But, I guess that's not a priority anymore if you're willing to sacrifice $2400 a year to Anthropic.
  - trenchpilgrim 1y ago
    $2400? Mate, I have a free GitHub Copilot subscription (Microsoft hands them out to active OSS developers), and work pays for my Claude Code via our cloud provider backend (and it costs less per working day than my morning Monster can). LLM inference is _cheap_ and _getting cheaper every month_. > No, but I expect my software to have been verified for correctness, and soundness by a human being with a working mental model of how the code works. This is not exclusive with AI tools: - Use AI to write dev tools to help you write and verify your handwritten code. Throw the one-off dev tools in the bin when you're done. - Handwrite your code, generate test data, review the test data like you would a junior engineer's work. - Handwrite tests, AI generate an implementation, have the agent run tests in a loop to refine itself. Works great for code that follows a strict spec. Again, review the code like you would a junior engineer's work.
    - jplusequalt 1y ago
      Writing the tests by hand, but letting the AI write the code sounds horribly dull.
      - trenchpilgrim 1y ago
        I'm an infrastructure/platform engineer. If the code is boring, that probably means I'm doing my job well. This isn't hobby coding.
nerevarthelame 1y ago
How are you sure it's increasing your productivity if it "makes no sense" to even quantify that? What are the receipts you have?
- larve 1y ago
  I have linked my github above. I don't know how that fares in the bigger scope of things, but I went from 0 opensource to hundreds of tools and frameworks and libraries. Putting a number on "productivity" makes no sense to me, I would have no idea what that means. I generate between 10-100k lines of code per day these days. But is that a measure of productivity? Not really...
  - sarchertech 1y ago
    >I generate between 10-100k lines of code per day these days. That’s absolute nonsense.
    - irthomasthomas 1y ago
      He said "generate". This is trivial to do. And probably this is what Amodei meant when he said 90% of code would be AI by now. It doesn't meant that generated code is actually useful and gets checked in.
      - larve 1y ago
        Trivial is a pretty big word in this context. Expanding an idea into some sort of code is indeed a matter of waiting. The idea, the prompt, the design of the overall workflow to leverage the capabilities of llms/agents in a professional/long-lived codebase context is far from trivial, imo.
    - larve 1y ago
      You can look at my GitHub, and I stream full unedited sessions on https://youtube.com/@program-with-ai https://youtube.com/@program-with-ai
noidesto 1y ago
Agree. In the hands of a seasoned dev not only does productivity improve but the quality of outputs. If I’m working against a deadline I feel more comfortable spending time on research and design knowing I can spend less time on implementation. In the end, it took the same amount of time, though hopefully with an increase of reliability, observability, and extendibility. None of these things show up in the author’s faulty dataset and experiment.
deleted 1y ago
[deleted]
philipwhiuk 1y ago
I mean it's definitely shovelware, I'll give you that. https://github.com/go-go-golems/ai-in-action-app/blob/main/cmd/server/main.go https://github.com/go-go-golems/ai-in-action-app/blob/main/c...
- larve 1y ago
  Not sure what you mean? This was a demo in a live session that took about 30 minutes, including ui ideation (see pngs). It’s a reasonably well featured app and the code is fairly minimal. I wouldn’t be able to write something like that in 30 minutes by hand.
ryanobjc 1y ago
The author is pointing out that aggregate productivity hasn't really gone up. The graphs are fairly compelling. There are many reasons for your experience, and I am glad you are having them! That's great! But the fact remains, overall we aren't seeing an exponential or even step function in how much software is being delivered!
xenobeb 1y ago
What is even the point in having this argument? At this point, one is gaining with each model release or they are not. Lets see in 2035 who was right and who was wrong. My bet is the people who are not gaining right now are not going to like the situation in 2035.

by larve 1y ago

trenchpilgrim 1y ago

Same. On many days 90% of my code output by lines is Claude generated and things that took me a day now take well under an hour. Also, a good chunk of my personal OSS projects are AI assisted. You probably can't tell from looking at them, because I have strict style guides that suppress the "AI style", and I don't really talk about how I use AI in the READMEs. Do you also expect I mention that I used Intellisense and syntax highlighting too?

droidjj 1y ago
The author’s main point is that there hasn’t been an uptick in total code shipped, as you would expect if people are 10x-ing their productivity. Whether folks admit to using AI in their workflow is irrelevant.
- warkdarrior 1y ago
  Maybe people are working less and enjoying life more, while shipping the same amount of code as before. If someone builds a faster car tomorrow, I am not going to go to the office more often.
  - leoc 1y ago
    "In this economy?", as the saying goes.
  - jplusequalt 1y ago
    Jevon's paradox.
- trenchpilgrim 1y ago
  The bottleneck on how much I ship has never been how fast I can write and deploy code :)
- larve 1y ago
  Their main point is "AI coding claims don't add up", as shown by the amount of code shipped. I personally do think some of the more incredible claims about AI coding add up, and am happy to talk about it based on my "evidence", ie the software I am building. 99.99% of my code is ai generated at this point, with the occasional one line I fill in because it'd be stupid to wait for an LLM to do it. For example, I've built 5-6 iphone apps, but they're kind of one-offs and I don't know why I would put them up on the app store, since they only scratch my own itches.
  - trenchpilgrim 1y ago
    Oh yeah, I love building one off tools with it. I am working on a game mod with a friend, we are hand writing the code that runs when you play it, but we vibe code all sorts of dev tools to help us test and iterate on it faster. Do internal, narrow purpose dev tools count as shipped code?
    - daxfohl 1y ago
      This seems to be a common thread. For personal projects where most details aren't important, they are good at meeting the couple things that are important to you and filling in the rest with reasonable, mostly-good-enough guesses. But the more detailed the requirements are, the less filler code there is, and the more each line of code matters. In those situations it's probably faster to type the line of code than to type the English equivalent and hand-hold the assistant through the editing process.
      - larve 1y ago
        I don't think so, although I think at that point experience heavily comes into play. With GPT-5 especially, I can basically point cursor/codex at a repo and say "refactor this to this pattern" and come back 25 minutes later to a pretty much impeccable result. In fact that's become my favourite past time lately. I linked some examples higher up, but I've been maintaining a lot of packages that I started slightly before chatgpt and then refactored and worked on as I progressively moved to the "entirely AI generated" workflow I have today. I don't think it's an easy skill (not saying that to make myself look good, I spent an ungodly amount of time exploring programming with LLMs and still do), akin to thinking at a strategic level vs at a "code" level. Certain design patterns also make it much easier to deal with LLM code: state reducers (redux/zustand for example), event-driven architectures, component-based design systems, building many CLI tools that the agent can invoke to iterate and correct things, as do certain "tools" like sqlite/tmux (by that I mean just telling the LLM "btw you can use tmux/sqlite", you allow it to pass hurdles that would otherwise just make it spiral into slop-ratatouille). I also think that a language like go was a really good coincidence, because it is so amenable to LLM-ification.
- Aeolun 1y ago
  I don’t think this is necessarily true. People that didn’t ship before still don’t ship. My ‘unshipped projects’ backlog is still nearly as large. It’s just got three new entries in the past two months instead of one.
jplusequalt 1y ago
>Do you also expect I mention that I used Intellisense and syntax highlighting too? No, but I expect my software to have been verified for correctness, and soundness by a human being with a working mental model of how the code works. But, I guess that's not a priority anymore if you're willing to sacrifice $2400 a year to Anthropic.
- trenchpilgrim 1y ago
  $2400? Mate, I have a free GitHub Copilot subscription (Microsoft hands them out to active OSS developers), and work pays for my Claude Code via our cloud provider backend (and it costs less per working day than my morning Monster can). LLM inference is _cheap_ and _getting cheaper every month_. > No, but I expect my software to have been verified for correctness, and soundness by a human being with a working mental model of how the code works. This is not exclusive with AI tools: - Use AI to write dev tools to help you write and verify your handwritten code. Throw the one-off dev tools in the bin when you're done. - Handwrite your code, generate test data, review the test data like you would a junior engineer's work. - Handwrite tests, AI generate an implementation, have the agent run tests in a loop to refine itself. Works great for code that follows a strict spec. Again, review the code like you would a junior engineer's work.
  - jplusequalt 1y ago
    Writing the tests by hand, but letting the AI write the code sounds horribly dull.
    - trenchpilgrim 1y ago
      I'm an infrastructure/platform engineer. If the code is boring, that probably means I'm doing my job well. This isn't hobby coding.

nerevarthelame 1y ago

How are you sure it's increasing your productivity if it "makes no sense" to even quantify that? What are the receipts you have?

larve 1y ago
I have linked my github above. I don't know how that fares in the bigger scope of things, but I went from 0 opensource to hundreds of tools and frameworks and libraries. Putting a number on "productivity" makes no sense to me, I would have no idea what that means. I generate between 10-100k lines of code per day these days. But is that a measure of productivity? Not really...
- sarchertech 1y ago
  >I generate between 10-100k lines of code per day these days. That’s absolute nonsense.
  - irthomasthomas 1y ago
    He said "generate". This is trivial to do. And probably this is what Amodei meant when he said 90% of code would be AI by now. It doesn't meant that generated code is actually useful and gets checked in.
    - larve 1y ago
      Trivial is a pretty big word in this context. Expanding an idea into some sort of code is indeed a matter of waiting. The idea, the prompt, the design of the overall workflow to leverage the capabilities of llms/agents in a professional/long-lived codebase context is far from trivial, imo.
  - larve 1y ago
    You can look at my GitHub, and I stream full unedited sessions on https://youtube.com/@program-with-ai https://youtube.com/@program-with-ai

noidesto 1y ago

Agree. In the hands of a seasoned dev not only does productivity improve but the quality of outputs. If I’m working against a deadline I feel more comfortable spending time on research and design knowing I can spend less time on implementation. In the end, it took the same amount of time, though hopefully with an increase of reliability, observability, and extendibility. None of these things show up in the author’s faulty dataset and experiment.

deleted 1y ago

[deleted]

philipwhiuk 1y ago

I mean it's definitely shovelware, I'll give you that. https://github.com/go-go-golems/ai-in-action-app/blob/main/cmd/server/main.go https://github.com/go-go-golems/ai-in-action-app/blob/main/c...

larve 1y ago
Not sure what you mean? This was a demo in a live session that took about 30 minutes, including ui ideation (see pngs). It’s a reasonably well featured app and the code is fairly minimal. I wouldn’t be able to write something like that in 30 minutes by hand.

ryanobjc 1y ago

The author is pointing out that aggregate productivity hasn't really gone up. The graphs are fairly compelling. There are many reasons for your experience, and I am glad you are having them! That's great! But the fact remains, overall we aren't seeing an exponential or even step function in how much software is being delivered!

xenobeb 1y ago

What is even the point in having this argument? At this point, one is gaining with each model release or they are not. Lets see in 2035 who was right and who was wrong. My bet is the people who are not gaining right now are not going to like the situation in 2035.