5 ms·

I'm completely equally lost the other way. I've went through multiple phases of LLM usage for development. GPT3.5 era: wow this is amazing, oh. everything is

I'm completely equally lost the other way. I've went through multiple phases of LLM usage for development. GPT3.5 era: wow this is amazing, oh. everything is hallucinated. not actually as useful as I first thought GPT4 era: very helpful as stackoverflow on steroids. Claude 3.5 Sonnet: have it open pretty much all the time, constantly asking questions and getting it to generate simple code (in the web UI) when it goes down actually feels very old school googling stuff. Tried a lot of in IDE AI "chat" stuff but hugely underwhelmed. Now: rarely open IDE as I can do (nearly) absolutely everything in Claude Code. I do have to refactor stuff every so often "manually", but this is more for my sanity and understanding of the codebase.. To give an example of a task I got Claude code to do today in a few minutes which would take me hours. Had a janky looking old admin panel in bootstrap styles that I wanted to make look nice. Told Claude code to fetch the marketing site for the project. Got it to pull CSS, logos, fonts from there using curl and apply similar styling to the admin panel project. Within 10 mins it was looking far, far better than I would have ever got it looking (at least without a designers help). Then got it to go through the entire project (dozens of screens) and update "explanation" copy - most of which was TODO placeholders to explain what everything did properly. I then got it to add an e2e test suite to the core flows. This took less than an hour while I was watching TV. I would have almost certainly _never_ got around to this before. I'd been meaning to do all this and I always sigh when I go into this panel at how clunky it all is and hard to explain to people.

kaashif 1y ago
Yeah, as a primarily backend engineer dealing with either weird technical problems Claude can't get quite right or esoteric business domain problems Claude has no idea about (and indeed, it may be only a few people in one company could help with) - Claude isn't that useful. But random stuff like make a web app that automates this thing or make an admin panel with auto complete on these fields and caching data pulled from this table. It is like infinity times faster on this tedious boilerplate because some of this stuff I'd just have never done before. Or I'd have needed to get some headcount in some web dev team to do it, but I just don't need to. Not that I'd have ever actually bothered to do that anyway...
- martinald 1y ago
  One thing I'd recommend for weird business domain projects as a starter is getting it to create a "wiki" of markdown files of all the logic (I suspect this may have been on your to-do list anyway!). You may be pleasantly surprised at how well it does it, and then you can update your claude.md file to point to them (or even put it in there, but it is maybe overkill).
  - PaulHoule 1y ago
    Asking for documentation is good. I had a data structure in Java that was a bunch of nested Map<String, Object>(s) that I partially understood and asked my agent to write me some sample JSON documents and... it just did it and it was a big help for manual coding against that structure. When I can't figure out something about a library I have often loaded the git repository into my IDE and checked out the version I was using, then used the IDE to study the code. Now I do that and ask the agent questions about the code like "How do I do X?" (often it sees the part in the doumentation that I missed) or "Where is the code that does Y?" It greatly accelerates the process of understanding code.
  - kaashif 1y ago
    Agreed - LLMs are great at reading and synthesising docs. At various points I've gotten pushback when I ask people to document because no-one will look at the docs, but now we have LLMs to read and link to docs for us. And traditional search still works, of course, but worse.
- viccis 1y ago
  I've found that it's mainly good at doing coding work that basically anyone who's memorized the content of the relevant docs could do. So I mostly only use it as glorified scaffolding, as it crashes and burns on complex stuff in big legacy codebases.
throwawaysleep 1y ago
> while I was watching TV This to me is one of the real benefits. I can vibe code watching TV. I can vibe code in bed. I can vibe code on the plane waiting for takeoff with GitHub Copilot Agents.
- VladVladikoff 1y ago
  To me this is sad. I love programming. It was always a fun job. I don’t want to watch TV instead of programming.
  - johnfn 1y ago
    Then don't. You'll likely outperform those who do. Not making any value judgements; both have their place. Sometimes I want to be locked in and sometimes I want to let Claude spin while I take a walk.
  - throwawaysleep 1y ago
    Do you love finding every weird CSS important hack, every RBAC refactor, or marching through the codebase to put a key where every piece of text is when i18n comes along? Lots of coding work is interesting, but plenty is just tedious.
  - jama211 1y ago
    Then code personal projects while you vibe code your job. The thing about having more free time, is you can use it on whatever you want.
jvanderbot 1y ago
I'm convinced the vast difference in outcome with LLM use is a product of the vast difference in jobs. For front end work it's just amazing. Spits out boilerplate and makes alterations without any need of help. For domain specific backend, for example robotics, it's bad. Tries to puke bespoke a-star, or invents libraries and functions. I'm way better off hand coding these things. The problem is this is classic Gell Mann Amnesia. I can have it restyle my website with zero work, even adding StarCraft 2 or NBA Jam themes, but ask it to work in a planning or estimation problem and I'm annoyed by its quality. Its probably bad at both but I don't notice. If we have 10 specializations required on an app, I'm only mad about 10℅. If I want to make an app entirely outside my domain, yeah sure it's the best ever.
- thrown-0825 1y ago
  If there is an abundance of medium / stack overflow articles it does well.
- DGCA 1y ago
  Frontend dev for 13+ years here. In my experience it’s pretty bad at anything but basic frontend stuff. I’ve tried all the models, editors, Claude code etc. It’s constantly wrong about the simplest things. Just now it was hallucinating about CSS grid properties. I’m convinced that how good AI seems is inversely correlated to someone’s knowledge in that domain, as you also mentioned. Still super useful in my day to day but I have to hand hold and correct it constantly.
  - anonzzzies 1y ago
    I am very bad at frontend and I know frontend peeps keep saying claude/llms are bad at it, but now, for the first time in my 40 year career, I deliver frontend that are looking better and functioning better than whatever I see in hip products people 'designed with love in blah'. And our clients agree. If the css is correct or not, I don't know as I never really found the energy to learn it and now I don't need to and can focus on the business logic. The endresults look and work phenomenal: not sure why I would care if the css is the best. Especially nice when I have been using something for a long time and some things I always found lacking: now I add them in in minutes: auto completion here, move creation inline there, add drag and drop: ah but on mobile make it like that. Everything just works and in hours vs days or weeks it was before.
socalgal2 1y ago
> This took less than an hour while I was watching TV. I certainly could not review all of those changes in an uninterupted hour. I'd need to test the design changes on multiple browsers, check they respond to zoom and window sizing. I'd have to read through the tests and check that they were not just nonsense and returning true to pass. There's no way I could do all that while watching TV in 1 hour.
- jama211 1y ago
  This is for a janky backend admin panel, not a customer facing product. If it’s way better than it was before, which likely wasn’t responsive either, it’s a win. Understanding the context of a project is important.
  - socalgal2 1y ago
    And the tests? To me, if it was something important enough to have tests for in which case it's important to check the tests are valid so I'd have to review them
    - martinald 1y ago
      It takes me 5 mins to review e2e UI tests and watch it go through them. It's literally filling fields and making sure certain things happen.
    - jama211 1y ago
      I think you’re overestimating the complexity of the tests required here. But I take it from the rest of your comment that you understand my point.

kaashif 1y ago

Yeah, as a primarily backend engineer dealing with either weird technical problems Claude can't get quite right or esoteric business domain problems Claude has no idea about (and indeed, it may be only a few people in one company could help with) - Claude isn't that useful. But random stuff like make a web app that automates this thing or make an admin panel with auto complete on these fields and caching data pulled from this table. It is like infinity times faster on this tedious boilerplate because some of this stuff I'd just have never done before. Or I'd have needed to get some headcount in some web dev team to do it, but I just don't need to. Not that I'd have ever actually bothered to do that anyway...

martinald 1y ago
One thing I'd recommend for weird business domain projects as a starter is getting it to create a "wiki" of markdown files of all the logic (I suspect this may have been on your to-do list anyway!). You may be pleasantly surprised at how well it does it, and then you can update your claude.md file to point to them (or even put it in there, but it is maybe overkill).
- PaulHoule 1y ago
  Asking for documentation is good. I had a data structure in Java that was a bunch of nested Map<String, Object>(s) that I partially understood and asked my agent to write me some sample JSON documents and... it just did it and it was a big help for manual coding against that structure. When I can't figure out something about a library I have often loaded the git repository into my IDE and checked out the version I was using, then used the IDE to study the code. Now I do that and ask the agent questions about the code like "How do I do X?" (often it sees the part in the doumentation that I missed) or "Where is the code that does Y?" It greatly accelerates the process of understanding code.
- kaashif 1y ago
  Agreed - LLMs are great at reading and synthesising docs. At various points I've gotten pushback when I ask people to document because no-one will look at the docs, but now we have LLMs to read and link to docs for us. And traditional search still works, of course, but worse.
viccis 1y ago
I've found that it's mainly good at doing coding work that basically anyone who's memorized the content of the relevant docs could do. So I mostly only use it as glorified scaffolding, as it crashes and burns on complex stuff in big legacy codebases.

throwawaysleep 1y ago

> while I was watching TV This to me is one of the real benefits. I can vibe code watching TV. I can vibe code in bed. I can vibe code on the plane waiting for takeoff with GitHub Copilot Agents.

VladVladikoff 1y ago
To me this is sad. I love programming. It was always a fun job. I don’t want to watch TV instead of programming.
- johnfn 1y ago
  Then don't. You'll likely outperform those who do. Not making any value judgements; both have their place. Sometimes I want to be locked in and sometimes I want to let Claude spin while I take a walk.
- throwawaysleep 1y ago
  Do you love finding every weird CSS important hack, every RBAC refactor, or marching through the codebase to put a key where every piece of text is when i18n comes along? Lots of coding work is interesting, but plenty is just tedious.
- jama211 1y ago
  Then code personal projects while you vibe code your job. The thing about having more free time, is you can use it on whatever you want.

jvanderbot 1y ago

I'm convinced the vast difference in outcome with LLM use is a product of the vast difference in jobs. For front end work it's just amazing. Spits out boilerplate and makes alterations without any need of help. For domain specific backend, for example robotics, it's bad. Tries to puke bespoke a-star, or invents libraries and functions. I'm way better off hand coding these things. The problem is this is classic Gell Mann Amnesia. I can have it restyle my website with zero work, even adding StarCraft 2 or NBA Jam themes, but ask it to work in a planning or estimation problem and I'm annoyed by its quality. Its probably bad at both but I don't notice. If we have 10 specializations required on an app, I'm only mad about 10℅. If I want to make an app entirely outside my domain, yeah sure it's the best ever.

thrown-0825 1y ago
If there is an abundance of medium / stack overflow articles it does well.
DGCA 1y ago
Frontend dev for 13+ years here. In my experience it’s pretty bad at anything but basic frontend stuff. I’ve tried all the models, editors, Claude code etc. It’s constantly wrong about the simplest things. Just now it was hallucinating about CSS grid properties. I’m convinced that how good AI seems is inversely correlated to someone’s knowledge in that domain, as you also mentioned. Still super useful in my day to day but I have to hand hold and correct it constantly.
- anonzzzies 1y ago
  I am very bad at frontend and I know frontend peeps keep saying claude/llms are bad at it, but now, for the first time in my 40 year career, I deliver frontend that are looking better and functioning better than whatever I see in hip products people 'designed with love in blah'. And our clients agree. If the css is correct or not, I don't know as I never really found the energy to learn it and now I don't need to and can focus on the business logic. The endresults look and work phenomenal: not sure why I would care if the css is the best. Especially nice when I have been using something for a long time and some things I always found lacking: now I add them in in minutes: auto completion here, move creation inline there, add drag and drop: ah but on mobile make it like that. Everything just works and in hours vs days or weeks it was before.

socalgal2 1y ago

> This took less than an hour while I was watching TV. I certainly could not review all of those changes in an uninterupted hour. I'd need to test the design changes on multiple browsers, check they respond to zoom and window sizing. I'd have to read through the tests and check that they were not just nonsense and returning true to pass. There's no way I could do all that while watching TV in 1 hour.

jama211 1y ago
This is for a janky backend admin panel, not a customer facing product. If it’s way better than it was before, which likely wasn’t responsive either, it’s a win. Understanding the context of a project is important.
- socalgal2 1y ago
  And the tests? To me, if it was something important enough to have tests for in which case it's important to check the tests are valid so I'd have to review them
  - martinald 1y ago
    It takes me 5 mins to review e2e UI tests and watch it go through them. It's literally filling fields and making sure certain things happen.
  - jama211 1y ago
    I think you’re overestimating the complexity of the tests required here. But I take it from the rest of your comment that you understand my point.