4 ms·

Sorry, but this is a ridiculous rant. Hallucinations have become much less common among the SOTA models, especially for coding with good guidance. I've been u

Sorry, but this is a ridiculous rant. Hallucinations have become much less common among the SOTA models, especially for coding with good guidance. I've been using Claude Sonnet 3.7 a lot the last two days, and to my knowledge, there have been zero hallucinations. It just does what I ask. But when they do make coding mistakes - so what? So do humans. I use it for non-coding questions all the time. The idea that the utility I get is completely negated because of occasional hallucinations is absurd. Have you used any tools like Aider, Cursor or Code?

jcgrillo 2y ago
> But when they do make coding mistakes - so what? So do humans. Surely we can do better than that. "Coding mistakes" are OK, so long as they're caught by review. However, engineers who continually make tons of mistakes without improving over time are liable to annoy their colleagues to the point of quitting, or alternatively (and hopefully more likely) asking management to remove the defective engineer. So the open question is: Do these tools make PRs that are irritating to review? Another related and arguably more important open question is: Does widespread use of "AI" tools in an engineering department result in significantly more defects being deployed in production? Another important question: Are codebases which have evolved over a period of years in an organization which makes heavy use of "AI" tools more or less easy to understand? Another question: Is the MTTR of incidents affected by the use of "AI" coding tools? I could go on, but the point is absolutely not "oh well devs make mistakes so do LLMs whatever who cares?" Until these questions are answerable concretely, this is nothing but a research topic. It's worth zero dollars. It's a fuckin' NFT.
- SamPatt 2y ago
  You sound like an old engineer who believes that the process you've mastered is the only conceivable way to do things. Saying LLMs are worth zero dollars shows a disconnection from reality so profound that I doubt you'll ever be able to admit you were wrong. I wish you all the best with your remaining career.
  - jcgrillo 2y ago
    You know, I hope you're right. I'm tired, and I'd love to retire. If your techno-rapture is indeed imminent I'll soon be able to do that. Unfortunately, I can't join you in this cultish belief system. I have had the benefit of boom upon bust and hype cycles upon hype cycles, "AI" summers, "AI" winters, and more clever little spacecamp boondoggles and silly con valley grifter scams than I can count. And I'm not even 40 yet. So I'll believe it when I see it. You think you can change the world with chatcoin or whatever, go do it! Prove me wrong! I'm not betting on you. Those who do not know history are doomed to repeat it has been playing on a loop in my mind for the past few months... I wonder why?
    - SamPatt 2y ago
      Surely there's a middle ground between culty techno-raptures and literally zero value? Of course there's hype. Turns out I'm older than you are. I've seen hype cycles. I've also seen incredible progress. Why not acknowledge that hype exists and LLMs aren't perfect, but that many people find value in them already and there appears to be a positive trend in their capability? Knowledge of history is knowledge of technological progress. Hype cycles don't negate that. I hear being tired though. Let's watch and see what happens.
      - jcgrillo 2y ago
        Oh I agree LLMs are interesting, and many people are excited about them, but I don't think the utility has been demonstrated. I'll grant you plenty of people state they've been made more productive, but I really prefer objective measures especially in this overhyped fomo-riddled information environment. So until someone demonstrates objectively that these tools actually make the business of producing and running software more efficient, there's zero value. Even then, it remains to be seen whether these efficiency gains (if any) are lasting and sufficiently great to offset the material cost of running the models, and the social cost in the organizations that run them. It's hard to imagine how they could be worth the trillions the market expects. Sure, they seem to be getting cheaper and to some degree more accurate, but it's extremely dangerous to extrapolate. So I won't. Neither should you or anyone else. If these things are actually any good for something show, don't tell. EDIT: My "investment thesis" on this is that it's a huge bubble, and this "AI" hype will ultimately erase many times more value than it will create. I really hope I'm wrong, we'll see.

jcgrillo 2y ago

> But when they do make coding mistakes - so what? So do humans. Surely we can do better than that. "Coding mistakes" are OK, so long as they're caught by review. However, engineers who continually make tons of mistakes without improving over time are liable to annoy their colleagues to the point of quitting, or alternatively (and hopefully more likely) asking management to remove the defective engineer. So the open question is: Do these tools make PRs that are irritating to review? Another related and arguably more important open question is: Does widespread use of "AI" tools in an engineering department result in significantly more defects being deployed in production? Another important question: Are codebases which have evolved over a period of years in an organization which makes heavy use of "AI" tools more or less easy to understand? Another question: Is the MTTR of incidents affected by the use of "AI" coding tools? I could go on, but the point is absolutely not "oh well devs make mistakes so do LLMs whatever who cares?" Until these questions are answerable concretely, this is nothing but a research topic. It's worth zero dollars. It's a fuckin' NFT.

SamPatt 2y ago
You sound like an old engineer who believes that the process you've mastered is the only conceivable way to do things. Saying LLMs are worth zero dollars shows a disconnection from reality so profound that I doubt you'll ever be able to admit you were wrong. I wish you all the best with your remaining career.
- jcgrillo 2y ago
  You know, I hope you're right. I'm tired, and I'd love to retire. If your techno-rapture is indeed imminent I'll soon be able to do that. Unfortunately, I can't join you in this cultish belief system. I have had the benefit of boom upon bust and hype cycles upon hype cycles, "AI" summers, "AI" winters, and more clever little spacecamp boondoggles and silly con valley grifter scams than I can count. And I'm not even 40 yet. So I'll believe it when I see it. You think you can change the world with chatcoin or whatever, go do it! Prove me wrong! I'm not betting on you. Those who do not know history are doomed to repeat it has been playing on a loop in my mind for the past few months... I wonder why?
  - SamPatt 2y ago
    Surely there's a middle ground between culty techno-raptures and literally zero value? Of course there's hype. Turns out I'm older than you are. I've seen hype cycles. I've also seen incredible progress. Why not acknowledge that hype exists and LLMs aren't perfect, but that many people find value in them already and there appears to be a positive trend in their capability? Knowledge of history is knowledge of technological progress. Hype cycles don't negate that. I hear being tired though. Let's watch and see what happens.
    - jcgrillo 2y ago
      Oh I agree LLMs are interesting, and many people are excited about them, but I don't think the utility has been demonstrated. I'll grant you plenty of people state they've been made more productive, but I really prefer objective measures especially in this overhyped fomo-riddled information environment. So until someone demonstrates objectively that these tools actually make the business of producing and running software more efficient, there's zero value. Even then, it remains to be seen whether these efficiency gains (if any) are lasting and sufficiently great to offset the material cost of running the models, and the social cost in the organizations that run them. It's hard to imagine how they could be worth the trillions the market expects. Sure, they seem to be getting cheaper and to some degree more accurate, but it's extremely dangerous to extrapolate. So I won't. Neither should you or anyone else. If these things are actually any good for something show, don't tell. EDIT: My "investment thesis" on this is that it's a huge bubble, and this "AI" hype will ultimately erase many times more value than it will create. I really hope I'm wrong, we'll see.