4 ms·

Could you elaborate a bit on the tasks,languages,domain etc you’re using it with? People have such widely varying experiences and I’m wondering why.

criddell 1y ago
I haven't had great luck with Claude writing Windows Win32 (using MFC) in C++. It invents messages and APIs all the time that read like exactly what I want it to do. I'd think Win32 development would be something AIs are very strong at because it's so old, so well documented, and there's a ton of code out there for it to read. Yet it still struggles with the differences between Windows messages, control notification messages, and command messages.
thegrim33 1y ago
I find it pretty interesting that it's a roughly 2,500 word article on "using Claude Code" and they never once actually explain what they're using it for, what type of project they're coding. It's all just so generic. I read some of it then realize that there was absolutely no substance in what I just read. It's also another in my growing list of data points towards my opinion that if an author posts meme pictures in their article, it's probably not an article I'm interested in reading.
- kraftman 1y ago
  Yeah I got about half way through before thinking "wow theres no information in this" and giving up.
- sensanaty 1y ago
  Always good to be cognizant that there are MANY people out there, especially on HN/YC circles, with large vested interests in LLM tooling. Just check out the YC batches lately, you'll be hard pressed to find a single one that doesn't mention AI or LLMs in some way.
- vitaflo 1y ago
  It’s always telling when people don’t show their work. I’m not saying LLMs can’t do a good job but if you’re not even explaining the steps you used or showing the code that was generated or fixed then I have to assume what was really produced was unmaintainable spaghetti code that just happened to compile.
  - danielbln 1y ago
    I literally cannot share my code, as that belongs to the company or the client. So what would I show? My prompts? Pseudo code? Unless you're an open source developer or build personal projects, requests for "show your code, bro" are hard to satisfy.
    - leptons 1y ago
      That's fine, but then don't write an article about it if you can't show the code. The vagueness just makes the article look unsupported by facts.
_se 1y ago
It's always POC apps in js or python, or very small libraries in other popular languages with good structure from the start. There are ways to make them somewhat better in other cases (automated testing/validation/linting being a big one), but for the type of thing that 95% of developers are doing day to day (working on a big, sprawling code base where none of those attributes apply), it's not close to being there. The tools really do shine where they're good though. They're amazing. But the moment you try to do the more "serious" work with them, it falls apart rapidly. I say this as someone that uses the tools every day. The only explanation that makes sense to me is that the "you don't get it, they're amazing at everything" people just aren't working on anything even remotely complicated. Or it's confirmation bias that they're only remembering the good results - as we saw with last week's study on the impact of these tools on open source development (perceived productivity was up, real productivity was down). Until we start seeing examples to the contrary, IMO it's not worth thinking that much about. Use them at what they're good at, don't use them for other tasks. LLMs don't have to be "all or nothing". They absolutely are not good at everything, but that doesn't mean they aren't good at anything.
- Herring 1y ago
  I like them for refactoring and “explain this massive codebase please”. Basically polishing or investigating things that already work. But I think we should expect the scope of LLM work to improve rapidly in the next few years. https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...
  - oblio 1y ago
    The bad news is that mostly, as far as we can see, that doubling of performance also requires (at least) doubling of resource usage, plus we're getting close to a point where planetary resources for doubling LLM resources are getting kind of low...
    - Herring 1y ago
      This species is going extinct. I finally accepted that when my dad died rather than change his lifestyle, despite being warned 10000x. My mom survived a heart attack, saw what happened to my dad, still hasn't changed her lifestyle.
rr808 1y ago
I opinion is that the AI is the distilled average of all the code it can scrape. For the stuff I'm good at and work on every day it doesn't help much beyond some handy code completions. For stuff I'm below average at like bash commands and JS it helps me get up to average. The most valuable to me is if I can use it to learn something - it gives some good alternatives and ideas if you have something mainstream.
iambateman 1y ago
I’m a TALL developer, so Laravel, Livewire, Tailwind, Alpine. It’s nice because 3/4 of those are well-known but not “default” industry choices and it still handles them very well. So there’s a Laravel CRM builder called Filament which is really fun to work in. Claude does a great job with that. It’s a tremendous amount of boilerplate with clear documentation, so it makes sense that Claude would do well. The thing I appreciate though is that CC as an agent is able to do a lot in one go. I’ve also hooked CC up to a read-only API for a client, and I need to consume all of the data on that API for the transition to a Filament app. Claude is currently determining the schema, replicating it in Laravel, and doing a full pull of API records into Laravel models, all on its own. It’s been running for 10 minutes with no interruption and I expect will perform flawlessly at that. I invest a lot of energy in prompt preparation. My prompts are usually about 200 words for a feature, and I’ll go back and forth with an LLM to make sure it thinks it’s clear enough.
William_BB 1y ago
The reason is probably complexity and the task at hand. In my experience, LLMs are great at small tasks (bash or python scripts); good at simple CRUD stuff (js, ts, html, css, python); good at prototyping; good at documentation; okay at writing unit tests; okay at adding simple features in more complex databases; Anything more complex and I find it pretty much unusable, even with Claude 4. More complex C++ codebases; more niche libraries; ML, CV, more mathsy domains that require reasoning.
ModernMech 1y ago
When I read these LLM in coding discussions, I'm reminded a lot of online dating discussions. Someone will post "I'm really having a tough time dating. I tried X (e.g. a dating app) but I had a tough experience." Someone will respond "I tried X and had great success. Highly recommended." Seems confounding, but when you click into their profiles to see pictures of each person, it becomes abundantly clear why these people report different experiences. Not to dog the author too hard, but a look at their Github profile says a lot about the projects they've worked on and what kind of dev they are. Not much there in terms of projects or code output, but they do have 15k followers on Twitter, where they post frequently about LLMs to their audience. They aren't talking about the tasks and the domains they're using because that's incidental; what they really want to do is just talk about LLMs to their online audience, not ship code.

criddell 1y ago

I haven't had great luck with Claude writing Windows Win32 (using MFC) in C++. It invents messages and APIs all the time that read like exactly what I want it to do. I'd think Win32 development would be something AIs are very strong at because it's so old, so well documented, and there's a ton of code out there for it to read. Yet it still struggles with the differences between Windows messages, control notification messages, and command messages.

thegrim33 1y ago

I find it pretty interesting that it's a roughly 2,500 word article on "using Claude Code" and they never once actually explain what they're using it for, what type of project they're coding. It's all just so generic. I read some of it then realize that there was absolutely no substance in what I just read. It's also another in my growing list of data points towards my opinion that if an author posts meme pictures in their article, it's probably not an article I'm interested in reading.

kraftman 1y ago
Yeah I got about half way through before thinking "wow theres no information in this" and giving up.
sensanaty 1y ago
Always good to be cognizant that there are MANY people out there, especially on HN/YC circles, with large vested interests in LLM tooling. Just check out the YC batches lately, you'll be hard pressed to find a single one that doesn't mention AI or LLMs in some way.
vitaflo 1y ago
It’s always telling when people don’t show their work. I’m not saying LLMs can’t do a good job but if you’re not even explaining the steps you used or showing the code that was generated or fixed then I have to assume what was really produced was unmaintainable spaghetti code that just happened to compile.
- danielbln 1y ago
  I literally cannot share my code, as that belongs to the company or the client. So what would I show? My prompts? Pseudo code? Unless you're an open source developer or build personal projects, requests for "show your code, bro" are hard to satisfy.
  - leptons 1y ago
    That's fine, but then don't write an article about it if you can't show the code. The vagueness just makes the article look unsupported by facts.

_se 1y ago

It's always POC apps in js or python, or very small libraries in other popular languages with good structure from the start. There are ways to make them somewhat better in other cases (automated testing/validation/linting being a big one), but for the type of thing that 95% of developers are doing day to day (working on a big, sprawling code base where none of those attributes apply), it's not close to being there. The tools really do shine where they're good though. They're amazing. But the moment you try to do the more "serious" work with them, it falls apart rapidly. I say this as someone that uses the tools every day. The only explanation that makes sense to me is that the "you don't get it, they're amazing at everything" people just aren't working on anything even remotely complicated. Or it's confirmation bias that they're only remembering the good results - as we saw with last week's study on the impact of these tools on open source development (perceived productivity was up, real productivity was down). Until we start seeing examples to the contrary, IMO it's not worth thinking that much about. Use them at what they're good at, don't use them for other tasks. LLMs don't have to be "all or nothing". They absolutely are not good at everything, but that doesn't mean they aren't good at anything.

Herring 1y ago
I like them for refactoring and “explain this massive codebase please”. Basically polishing or investigating things that already work. But I think we should expect the scope of LLM work to improve rapidly in the next few years. https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...
- oblio 1y ago
  The bad news is that mostly, as far as we can see, that doubling of performance also requires (at least) doubling of resource usage, plus we're getting close to a point where planetary resources for doubling LLM resources are getting kind of low...
  - Herring 1y ago
    This species is going extinct. I finally accepted that when my dad died rather than change his lifestyle, despite being warned 10000x. My mom survived a heart attack, saw what happened to my dad, still hasn't changed her lifestyle.

rr808 1y ago

I opinion is that the AI is the distilled average of all the code it can scrape. For the stuff I'm good at and work on every day it doesn't help much beyond some handy code completions. For stuff I'm below average at like bash commands and JS it helps me get up to average. The most valuable to me is if I can use it to learn something - it gives some good alternatives and ideas if you have something mainstream.

iambateman 1y ago

I’m a TALL developer, so Laravel, Livewire, Tailwind, Alpine. It’s nice because 3/4 of those are well-known but not “default” industry choices and it still handles them very well. So there’s a Laravel CRM builder called Filament which is really fun to work in. Claude does a great job with that. It’s a tremendous amount of boilerplate with clear documentation, so it makes sense that Claude would do well. The thing I appreciate though is that CC as an agent is able to do a lot in one go. I’ve also hooked CC up to a read-only API for a client, and I need to consume all of the data on that API for the transition to a Filament app. Claude is currently determining the schema, replicating it in Laravel, and doing a full pull of API records into Laravel models, all on its own. It’s been running for 10 minutes with no interruption and I expect will perform flawlessly at that. I invest a lot of energy in prompt preparation. My prompts are usually about 200 words for a feature, and I’ll go back and forth with an LLM to make sure it thinks it’s clear enough.

William_BB 1y ago

The reason is probably complexity and the task at hand. In my experience, LLMs are great at small tasks (bash or python scripts); good at simple CRUD stuff (js, ts, html, css, python); good at prototyping; good at documentation; okay at writing unit tests; okay at adding simple features in more complex databases; Anything more complex and I find it pretty much unusable, even with Claude 4. More complex C++ codebases; more niche libraries; ML, CV, more mathsy domains that require reasoning.

ModernMech 1y ago

When I read these LLM in coding discussions, I'm reminded a lot of online dating discussions. Someone will post "I'm really having a tough time dating. I tried X (e.g. a dating app) but I had a tough experience." Someone will respond "I tried X and had great success. Highly recommended." Seems confounding, but when you click into their profiles to see pictures of each person, it becomes abundantly clear why these people report different experiences. Not to dog the author too hard, but a look at their Github profile says a lot about the projects they've worked on and what kind of dev they are. Not much there in terms of projects or code output, but they do have 15k followers on Twitter, where they post frequently about LLMs to their audience. They aren't talking about the tasks and the domains they're using because that's incidental; what they really want to do is just talk about LLMs to their online audience, not ship code.