6 ms·

I'm seeing the exact opposite on a large C++ project. I have friends at other companies with similar projects, they say the same thing. It's like we're living

I'm seeing the exact opposite on a large C++ project. I have friends at other companies with similar projects, they say the same thing. It's like we're living in different worlds. Still, LLMs are nice for well defined small projects, microservices, tools and research.

YZF 5mo ago
What tools have you tried? Are we talking Codex GPT 5.5 and Opus 4.7? Would you say the project is well architected? Clear boundaries? Or ball of mud? How large is large? Are there AGENT.md files giving good information that helps LLMs get context when looking at a certain area of the code? Is it all in one repo? multiple repos? Are there good tests? I feel like these are some of the many variables that can make a difference. I work on a pretty large project/code base, written mostly in Go, and I have pretty positive experience with LLMs. I take on fairly small chunks, I review and understand the changes. I also use LLMs to explore options and prototype quickly. They're also very good at fixing bugs, failing tests etc.
- mittensc 5mo ago
  > What tools have you tried? Are we talking Codex GPT 5.5 and Opus 4.7? Yes, with generous budgets. > They're also very good at fixing bugs, Seeing opposite here too, they are like eager juniors 'oh the issue is here and here's a 5 page report why', and it's wrong... then you add more info and it goes to a different spot... repeat until you get tired and solve it yourseld, it is useful as a rubber ducky i guess. > I work on a pretty large project/code base, written mostly in Go, and I have pretty positive experience with LLMs. I take on fairly small chunks, I review and understand the changes. Great that it's working for you, I'm just pointing out there's a massive disconnect. I would assume your work can be done by a junior engineer without any prior knowledge (except LLM md files) with same quality but less speed? If yes, then great, perhaps that's where the disconnect is, complexity. Also, if yes, which would be cheaper?, junior engineer or LLM?
  - brabel 5mo ago
    > Seeing opposite here too, they are like eager juniors 'oh the issue is here and here's a 5 page report why', and it's wrong... then you add more info and it goes to a different spot... repeat until you get tired and solve it yourseld, it is useful as a rubber ducky i guess. It's really amazing how different people have completely different experiences. I work on a massive code base and I thought AI would not be able to fix anything in at least a few years since the application is very complex and does not use well known frameworks. I was very wrong. In my experience, it fixes bugs better than I could, at least given a short time budget (which is always the case, if we spend too much time on each bug we just fix bugs slower than they get reported and we'd enter a death spiral). I have worked on this code base for more than 10 years, touched every part of it, and I wrote large chunks of most systems, despite around 20 people working on it right now. Still, when I need to figure out something, now, I often ask AI as it is absolutely wonderful in understanding and explaining code, no matter how big the code base is. My team consists of 20 very senior developers, and I am their technical lead, so I think I know what I am talking about. A junior would require at least 6 months of guidance to become productive in our code base, unfortunately, just because it's so big and it integrates with all sorts of external services, databases etc. I do understand that saying this is not really a flex, I would've actually preferred that my code base was so good even a junior developer could be immediately productive in it, but that's sadly just not the case. But perhaps, with the help of a AI tutor, that's actually possible now?! If you think AI is at the level of a junior developer right now, I'm afraid you're kidding yourself. In case you're wondering: we use Claude Code.
    - mittensc 5mo ago
      > given a short time budget (which is always the case, if we spend too much time on each bug we just fix bugs slower than they get reported and we'd enter a death spiral). This is something I don't understand. - If you have a bug, you need to fix it well as well as proper root cause. - That way the bug never surfaces again and safeguards are added for that class of bugs. - if done well over time it builds discipline and bugs only surface from new features or integrations. I've never had an experience of a 'death spiral' that you mention. > Still, when I need to figure out something, now, I often ask AI as it is absolutely wonderful in understanding and explaining code, no matter how big the code base is. Sure, but you still dig into the code afterwards I assume, you don't blindly trust what the AI summarization tells you. > If you think AI is at the level of a junior developer right now, I'm afraid you're kidding yourself. It depends, small projects with well defined scope, yeah, it knocks them out of the park, what I'm working on, it's a bit disappointing, not for lack of trying. Still, one other thing I'm noticing now... if my account were not anonymous I would likely need to think of possible repercussions for my 'lack of faith' and would probably post comments very similar to yours or not at all. So I'll stop here.
- iLoveOncall 5mo ago
  That's a lot of "ifs" for something supposed to revolutionize the industry.
nkapias 5mo ago
Noticed different results from friends, we have similar projects and tools. We're guessing it comes from organizational behavior (culture, governance, management, etc.), we work in diverse teams / regions / companies.
- cyclopeanutopia 5mo ago
  Or just when one person sees "great result", the other sees "garbage".
- aprilthird2021 5mo ago
  It's due to the jagged edge of AI experience. Because it's not deterministic the results don't play out deterministically (e.g. similar scenarios will have different and potentially drastically different results)

YZF 5mo ago

What tools have you tried? Are we talking Codex GPT 5.5 and Opus 4.7? Would you say the project is well architected? Clear boundaries? Or ball of mud? How large is large? Are there AGENT.md files giving good information that helps LLMs get context when looking at a certain area of the code? Is it all in one repo? multiple repos? Are there good tests? I feel like these are some of the many variables that can make a difference. I work on a pretty large project/code base, written mostly in Go, and I have pretty positive experience with LLMs. I take on fairly small chunks, I review and understand the changes. I also use LLMs to explore options and prototype quickly. They're also very good at fixing bugs, failing tests etc.

mittensc 5mo ago
> What tools have you tried? Are we talking Codex GPT 5.5 and Opus 4.7? Yes, with generous budgets. > They're also very good at fixing bugs, Seeing opposite here too, they are like eager juniors 'oh the issue is here and here's a 5 page report why', and it's wrong... then you add more info and it goes to a different spot... repeat until you get tired and solve it yourseld, it is useful as a rubber ducky i guess. > I work on a pretty large project/code base, written mostly in Go, and I have pretty positive experience with LLMs. I take on fairly small chunks, I review and understand the changes. Great that it's working for you, I'm just pointing out there's a massive disconnect. I would assume your work can be done by a junior engineer without any prior knowledge (except LLM md files) with same quality but less speed? If yes, then great, perhaps that's where the disconnect is, complexity. Also, if yes, which would be cheaper?, junior engineer or LLM?
- brabel 5mo ago
  > Seeing opposite here too, they are like eager juniors 'oh the issue is here and here's a 5 page report why', and it's wrong... then you add more info and it goes to a different spot... repeat until you get tired and solve it yourseld, it is useful as a rubber ducky i guess. It's really amazing how different people have completely different experiences. I work on a massive code base and I thought AI would not be able to fix anything in at least a few years since the application is very complex and does not use well known frameworks. I was very wrong. In my experience, it fixes bugs better than I could, at least given a short time budget (which is always the case, if we spend too much time on each bug we just fix bugs slower than they get reported and we'd enter a death spiral). I have worked on this code base for more than 10 years, touched every part of it, and I wrote large chunks of most systems, despite around 20 people working on it right now. Still, when I need to figure out something, now, I often ask AI as it is absolutely wonderful in understanding and explaining code, no matter how big the code base is. My team consists of 20 very senior developers, and I am their technical lead, so I think I know what I am talking about. A junior would require at least 6 months of guidance to become productive in our code base, unfortunately, just because it's so big and it integrates with all sorts of external services, databases etc. I do understand that saying this is not really a flex, I would've actually preferred that my code base was so good even a junior developer could be immediately productive in it, but that's sadly just not the case. But perhaps, with the help of a AI tutor, that's actually possible now?! If you think AI is at the level of a junior developer right now, I'm afraid you're kidding yourself. In case you're wondering: we use Claude Code.
  - mittensc 5mo ago
    > given a short time budget (which is always the case, if we spend too much time on each bug we just fix bugs slower than they get reported and we'd enter a death spiral). This is something I don't understand. - If you have a bug, you need to fix it well as well as proper root cause. - That way the bug never surfaces again and safeguards are added for that class of bugs. - if done well over time it builds discipline and bugs only surface from new features or integrations. I've never had an experience of a 'death spiral' that you mention. > Still, when I need to figure out something, now, I often ask AI as it is absolutely wonderful in understanding and explaining code, no matter how big the code base is. Sure, but you still dig into the code afterwards I assume, you don't blindly trust what the AI summarization tells you. > If you think AI is at the level of a junior developer right now, I'm afraid you're kidding yourself. It depends, small projects with well defined scope, yeah, it knocks them out of the park, what I'm working on, it's a bit disappointing, not for lack of trying. Still, one other thing I'm noticing now... if my account were not anonymous I would likely need to think of possible repercussions for my 'lack of faith' and would probably post comments very similar to yours or not at all. So I'll stop here.
iLoveOncall 5mo ago
That's a lot of "ifs" for something supposed to revolutionize the industry.

nkapias 5mo ago

Noticed different results from friends, we have similar projects and tools. We're guessing it comes from organizational behavior (culture, governance, management, etc.), we work in diverse teams / regions / companies.

cyclopeanutopia 5mo ago
Or just when one person sees "great result", the other sees "garbage".
aprilthird2021 5mo ago
It's due to the jagged edge of AI experience. Because it's not deterministic the results don't play out deterministically (e.g. similar scenarios will have different and potentially drastically different results)