6 ms·

I've been using Claude Code heavily for about 3 months now, and I'm pretty sure I'm between 10 and 20 times more productive while using it. How I measure perfo

• fuomag9 1y ago
  Same, I’ve done stuff that should have taken me 2-3 weeks in days
  • fourthark 1y ago
    I’ve done this without AI. The thing was not as hard as I thought it would be.
    • tracker1 1y ago
      Same, often a missing feature or library had become available since I last looked into a subject.
• zsoltkacsandi 1y ago
  I have exactly the same experience.
• benreesman 1y ago
  I find the swings to be wild, when you win with it, you win really big. But when you lose with it, it's a real bite out of your week too. And I think 10x to 20x has to be figurative right, you can do 20x by volume maybe, but to borrow an expression from Steve Ballmer, that's like measuring an airplane by kilograms. Someone already operating at the very limit of their abilities doing stuff that is for them high complexity, high cognitive load, detail intense, and tactically non-obvious? Even a machine that just handed you the perfect code can't 20x your real output, even if it gave you the source file at 20x your native sophistication you wouldn't be able to build and deploy it, let alone make changes to it. But even if it's the last 5-20% after you're already operating at your very limit and trying to hit your limit every single day is massive, it makes a bunch of stuff on the bubble go from "not realistic" to "we did that".
  • calrain 1y ago
    There are definitely swings. Last night it took about 2 hours to get Monaco into my webpack built bootstrap template, it came down to CSS being mishandled and Claude couldn't see the light. I just pasted the code into ChatGPT o3 and it fixed it first try. I pasted the output of ChatGPT into Claude and viola, all done. A key skill is to sense when the AI is starting to guess for solutions (no different to human devs) and then either lean into another AI or reset context and start over. I'm finding the code quality increase greatly with the addition of the text 'and please follow best practices because will be pen tested on this!' and wow.. it takes it much more seriously.
    • cluckindan 1y ago
      Is there a way to have two agentic AIs do pair programming?
      • calrain 1y ago
        I did experiment with this where Claude Code was the 'programmer' and ChatGPT was the Software Architect. The outcome was really solid and I made it clear that each was talking to an AI and they really seemed to collaborate and respect the key points of each side. It would be interesting to set up a MCP style interface, but even me copy/pasting between windows was constructive. The time this worked best was when I was building a security model for an API that had to be flexible and follow best practices. It was interesting seeing ChatGPT compare and contrast against major API vendors, and Claude Code asking the detailed implementation questions. The final output was a pragmatic middle-ground between simplistic and way too complex.
      • rsanheim 1y ago
        yes, definitely. https://github.com/BeehiveInnovations/zen-mcp-server https://github.com/BeehiveInnovations/zen-mcp-server is one example of people going off on this, but i'm sure there are many others
    • nottorp 1y ago
      Doesn't sound like you were writing actual functionality code, just integrating libraries?
• sph87 1y ago
  Where I have found Claude most helpful is on problems with very specific knowledge requirements. Like: Why isn’t this working? Here Claude read this like 90 page PDF and tell me where I went wrong interfacing with this SDK. Ohh I accidentally passed async_context_background_threading_safe instead of async_context_thread_safe_poll and it’s so now it’s panicking. Wow that would have taken me forever.
• DeepYogurt 1y ago
  Have any open source work you can show off?
  • calrain 1y ago
    Unfortunately not, but ensuring the final code quality will be well written is a challenge I am putting off for now. I'm leaning into the future growth of AI capabilities to help me here, otherwise I'll have to do it myself. That is a tomorrow problem, too much project structure/functionality to get right first.
    • KodeNinjaDev 1y ago
      So you are more productive, as long as you don't have to consider code quality.
      • calrain 1y ago
        Possibly, but not really. With most projects where innovation is a key requirement, the goal isn't to write textbook quality code, it's to prove your ideas work and quickly evolve the project. Once you have an idea of how it's going to work, you can then choose to start over from scratch or continue on and clean up all the bits you skipped over. Right now I'm in the innovation cycle, and having AI able to pick up whole API path strategies and pivot them, is incredibly amazing. How many times have you used large API's and seen clear hands of different developers and URI strategies, with an AI, you just pivot. Code quality and pen tests are critical, but they can come later.
        • skydhash 1y ago
          > Code quality and pen tests are critical, but they can come later. In my experience, no. These kind of shortcuts taken at the beginning of the project is why velocity have a sharp descent after some times. Because you’re either spending time undoing all of it (unlikely to be allowed) or you’re fighting in the code jungle trying to get some feature out.
      • 1y ago
• gtsop 1y ago
  I cringe when I see these numbers. 20 times better means that you can accomplish in two months what you would do in 4 years, which is ridiculus when said out loud. We can make it even more ridiculous by pointing out you would do in 3 years the work of working lifetime (60 years) I am wondering, what sort of tasks are you seeing these x20 boost?
  • baq 1y ago
    It isn’t ridiculous, it’s easily true, especially when you’re experienced in general, but have little to no knowledge of this particular big piece of tech, like say you’ve stopped doing frontend when jquery was all there was and you’re coming back. I’m doing things with react in hours I would have no business doing in weeks a couple years ago.
    • gtsop 1y ago
      I am waiting to see your 4 year human-equivalent project in a couple of months. Words without actions are junk. You are asserting something you have no proof for. Proove it then. Amaze us all with your productivity, out in the open. Shred those pilled up open issues on open source projects and then give us a report of how fast-easy it. If it is "easily true" you'll be done by next month
      • baq 1y ago
        I don’t need to prove to you something I experience daily, both professionally and in hobby side projects. You can take my anecdotes or you can keep believing me and people like me are wrong, your choice.
        • gtsop 1y ago
          If you don't need to prove it to me then you you wouldn't have replied to my initial comment to begin with. But I agree, you don't need to prove anything to me. You need to proove it to yourself first.
  • calrain 1y ago
• congaliminal 1y ago
  You're getting 6 months worth of work done in a week?
  • shinycode 1y ago
    I bet with a co-worker that a migration from angular 15 to angular 19 could be done really fast avoiding months. I spent a whole evening on it and Claude code have never been able to pull off a migration from 15 to 16 on its own. A total waste of time and nothing worked. I had the surprise that it cost me 275$ for nothing. So maybe for greenfield projects it’s smooth and saves time but it’s not a silver bullet on projects with problems.
    • deleted 1y ago
      [deleted]
    • calrain 1y ago
      I've had a lot of issues with Claude and web development. I ended up asking it how it wanted to work and would an 'AdminKit Template' work to get things moving. It recommended AdminKit and that was a good move. For me, custom UI's aren't a big part of the solution, I just need web pages to manage CRUD endpoints to manage the product. AdminKit has been a good fit so far, but it was a fresh start, no migration.
      • lucianbr 1y ago
        You asked Claude if AdminKit would work and in answer it recommended AdminKit? Seriously? Wow, what an unexpected turn of events. I am flabbergasted.
        • calrain 1y ago
          Apologies, it was a typo. I asked what 'Admin Template' because there are so many, and rather than build something from scratch, I wanted one it seemed to have understanding of. It mentioned AdminKit and it worked out pretty well.
    • 1y ago
• iammrpayments 1y ago
  I can’t believe such numbers. If this was true why don’t you quit your job and vibe code 10 ios apps
  • calrain 1y ago
    I wish I could. Some problems are difficult to solve and I still need to pay the bills. So I work 8 hours a day (to get money to eat) and code another 4 hours at home at night. Weekends are both 10 hour days, and then rinse / repeat. Unfortunately some projects are just hard to do and until now, they were too hard to attempt to solve solo. But with AI assistance, I am literally moving mountains. The project may still be a failure but at least it will fail faster, no different to the pre-AI days.
    • gbalduzzi 1y ago
      I don't think you are understanding how big 10x and 20x are. It means you can replace a whole team of developers alone. I can believe that some tasks are speed up by 10x or even 20x, but I find very hard to believe it's the average of your productivity (maintaining good code quality)
      • calrain 1y ago
        I mean from a time perspective, your mileage may vary. So me finishing a carded up block of work that is expected to take 2 weeks (80 hours) and I get it done in 1 day (8 hours) then that would be a 10x boost. There are always tar pits of time where you are no better off with AI, but sometimes it's 20x. I've setup development teams in the past, and have have been coding since the late 70's, so I am sort of aware of my capabilities. It super depends on the type of work you're doing.
        • lucianbr 1y ago
          This is satire, right? You're 60ish years old, and hyper optimistic about AI, it's making you tens of times more productive, paste code from one AI to another, one is the dev and the other is the architect... I mean, it's literally unbelievable.
• thinkingemote 1y ago
  I agree I feel more productive. AI tools do actually make it easier and makes my brain use less energy. You would think that would be more productive but maybe it just feels that way. Stage magicians say that the magic is done in the audiences memory after the trick is done. It's the effect of the activity. AI coding tools makes developers happier and able to spend more brain power on actually difficult things. But overall perhaps the amount of work isn't in orders of magnitudes it just feels like it. Waze the navigation app routes you in non standard routes so that you are not stuck in traffic, so it feels fast that you are making progress. But the time taken may be longer and the distance travelled may be further! Being in stuck traffic and not moving even for a little bit makes you feel that time has stopped, it's boring and frustrating. Now developers need never be stuck. Their roads will be clear, but they may take longer routes. We get little boosts of dopamine using AI tools to do stuff. Perhaps we used these signals as indicators of productivity "Ahh that days work felt good, I did a lot"
  • TeMPOraL 1y ago
    > Waze the navigation app routes you in non standard routes so that you are not stuck in traffic, so it feels fast that you are making progress. But the time taken may be longer and the distance travelled may be further! You're not "stuck in traffic", you are the traffic. If the app distributes users around and this makes it so they don't end up in traffic jams, it's effectively preventing traffic jams from forming I liked your washing machine vs. sink example that I see you just edited out. The machine may do it slower and less efficiently than you'd do in the sink, but the machine runs in parallel, freeing you to do something else. So is with good use of LLMs.
    • thinkingemote 1y ago
      Yeah I totally agree. It's like washing by hand vs using a mangle possibly. The metaphor of agents to machines was also what I thought but didn't write as it's about companion tools mainly. (I got confused and put in a high level comment but somehow didn't actually post that!) For Waze, even if you are traffic and others go around you, you still may get there quicker and your car use less energy than taking the suggested route that feels faster. Others may feel happier and feel like they were faster though. Indeed they were faster but might have taken a longer journey. Also, generally most people don't use the app around here to effect significant road use changes. But if they did im not sure (but I'm having fun trying to think) what metaphor we can apply to the current topic :)
• greatwhitenorth 1y ago
  I'm between 73 and 86 times more productive using claude code. You're not using it well.
  • luckylion 1y ago
    Can you show some of those problems and their solutions?
  • alternatex 1y ago
    Those are rookie numbers, you gotta pump those numbers up.
• xigoi 1y ago
  > How I measure performance is how many features I can implement in a given period of time. When a measure becomes a target, it ceases to be a good measure.
• halamadrid 1y ago
  For the sake of argument 20x means you have basically suddenly got access to 19 people with the same skill set as you. You can build a new product company with 20 people. Probably in the same domain as you are in right now.
  • discordance 1y ago
    Output doesn't necessarily scale linearly with as you add more people. Look up mythical man.
• latexr 1y ago
  > I'm pretty sure So were the people taking the study. Which is why we do these, to understand where our understanding of ourselves is lacking. Maybe you are special and do get extra gains. Or maybe you are as wrong about yourself as everyone else and are overestimating the gains you think you have.
• leptons 1y ago
  You're only getting 10x to 20x more productive? For me it's more like 10,000x to 50,000x, at minimum. YMMV.