4 ms·

Definitely not my experience. No matter the model, if I'm working on something important (and there is little reason on working on something not important) I do

Definitely not my experience. No matter the model, if I'm working on something important (and there is little reason on working on something not important) I do care about correctness and understandability. While LLMs are great for throwaway one-time code (although that's also debatable), they cannot compete with code written by a seasoned professional. No matter how many times I've tried delegating writing code to LLMs I've always regretted it in a few months' time, because it is more buggy and I don't really know what is happening there. The future is using LLMs for what they are good for. What that is still being found out. I've had great experience with LLMs reviewing the code (matches with Primagean's ~50% accuracy at finding bugs, which is really good) and for explaining unfamiliar concepts to me. I firmly believe that the code itself needs to be 100% organic, and if it's not and you're relying on LLMs to generate tons of code, you haven't built enough abstraction to make it unnecessary.

noodlescb 3mo ago
After 18+ months of this, LLMs are incredible for getting rid of the work I hated most: Updating JIRA tickets, tedious little cleanup jobs, refactoring, etc. They are great at finding infosec issues. They are also GREAT at writing code based on standards and patterns established by an actual human engineers. They are STILL trash at anything meaningfully complex from the ground up. I will go through hours of planning where I have to kick the hell out of its plans in ways that I don't think someone without a ton of real world experience could do. Even in the fancy new Fable models that remains true. They are not creative and any engineering worth doing requires creativity.
- vorticalbox 3mo ago
  > I have to kick the hell out of its plans in ways that I don't think someone without a ton of real world experience could do I think that’s the real point the article’s making. Unless you already know the ropes, you can’t tell if a plan is actually any good. I use LLMs all the time for work, but only because I get the system well enough to spot the nonsense or catch when a plan duplicates something that already exists. They really do keep rewriting the same functions over and over in the code.
echelon 3mo ago
You're moving too slow. Over the last four days I've few shotted a 1:1 clone (and improvement) of Fal, OpenRouter, and Replicate in Rust. It's going to be open source. I've lined up $300k of monthly commitments for it that land at launch. Might apply to YC with it. Moving "organic" is too slow for the world today.
- ozim 3mo ago
  Should have get some AI for bait posts :D
- overgard 3mo ago
  Oh cool, you're remaking things that already exist. You know why you can few shot those things? Because they already exist.
  - echelon 3mo ago
    Everything already exists. We only have incremental changes. The average moves over time. The ball is moving towards open source and this will include fine tunes and LoRAs, and even more fertile growth after that. There's an abundant future market in that direction. Several billion dollar businesses started as open source clones of existing things. Some found unique business lines, others simply did a better job.
    - CyberDildonics 3mo ago
      Everything already exists Then why are you trying to make something?
      - echelon 3mo ago
        Why make new music or new movies? The same things have been said by every previous generation. Just listen to your parents' music, right? Use the same technology, the same fashion, everything. Why? Because it's incremental improvement. Because it's making a dent in the world. Because creation feels better than consumption. Because we're maybe 10-20 iterations away (250 years, tops?) from brain uploads and everlasting life harvesting the energy of distant stars. Or at least our descendants are. And that's pretty cool.
overgard 3mo ago
One thing that I find frustrating/disturbing about LLM generated code is that wrong stuff doesn't usually look wrong, so you almost have to review it even closer than what you'd review from a person. Like in the past when a developer was being sloppy, usually you could kind of tell superficially, but with LLM code it's usually well documented and superficially well structured, all while doing batshit insane things. Like today I noticed some code that it had written was just silently returning on errors without logging a warning or returning an error code. It "worked" but it was a major foot-gun!
- m3kw9 3mo ago
  double edge sword of LLM code generation, you cannot catch 100%, nor do you want to always ready every single word. Gotta develop a feel.
  - bluefirebrand 3mo ago
    Engineering by feels... Is this what we've come to?
    - m3kw9 3mo ago
      Yes, feel, not your type of feel, my type where you can scan a section and know it's this architecture, compartmentized, know the general performance impact. Yes.
- learningstud 3mo ago
  Too true. I realized that the only way to do an actual review is to have wrote it yourself in the first place. The relatively slow speed of writing code reflects the amount of subtleties and effort to think through all of them. Only when LLMs are used as hint givers, are they actually doing an essential speedup. This goes to show how shallow programming and code review had always been. The right impression is that of studying real analysis.
m3kw9 3mo ago
I can generate LLM code and it will be very stable long term, but you must be QA minded and non-stop testing, ask and make sure it generate maintainable code, watch for code smell etc. Maybe it won't match a very seasoned dev, but LLM's generate sht code because the dev allows it at the review stage.
gwbas1c 3mo ago
Some things to consider: 1: The quality of code varies widely from model to model. In Visual Studio Copilot, I get so-so code from GPT, great code from Claude 4.x, lousy results from Claude 5.x 2: "Novel work" is harder to get from an LLM than when it's following an example. For example, I can get great Blazor / HTML / css from an LLM (and then spend time iterating making small adjustments,) but if I'm writing a complicated DB query or implementing a novel algorithm, it's best to do by hand. 3: But, in the case of a novel algorithm or query, I get great results saying "please write unit tests for XXX". That alone makes me develop code twice as fast!

noodlescb 3mo ago

After 18+ months of this, LLMs are incredible for getting rid of the work I hated most: Updating JIRA tickets, tedious little cleanup jobs, refactoring, etc. They are great at finding infosec issues. They are also GREAT at writing code based on standards and patterns established by an actual human engineers. They are STILL trash at anything meaningfully complex from the ground up. I will go through hours of planning where I have to kick the hell out of its plans in ways that I don't think someone without a ton of real world experience could do. Even in the fancy new Fable models that remains true. They are not creative and any engineering worth doing requires creativity.

vorticalbox 3mo ago
> I have to kick the hell out of its plans in ways that I don't think someone without a ton of real world experience could do I think that’s the real point the article’s making. Unless you already know the ropes, you can’t tell if a plan is actually any good. I use LLMs all the time for work, but only because I get the system well enough to spot the nonsense or catch when a plan duplicates something that already exists. They really do keep rewriting the same functions over and over in the code.

echelon 3mo ago

You're moving too slow. Over the last four days I've few shotted a 1:1 clone (and improvement) of Fal, OpenRouter, and Replicate in Rust. It's going to be open source. I've lined up $300k of monthly commitments for it that land at launch. Might apply to YC with it. Moving "organic" is too slow for the world today.

ozim 3mo ago
Should have get some AI for bait posts :D
overgard 3mo ago
Oh cool, you're remaking things that already exist. You know why you can few shot those things? Because they already exist.
- echelon 3mo ago
  Everything already exists. We only have incremental changes. The average moves over time. The ball is moving towards open source and this will include fine tunes and LoRAs, and even more fertile growth after that. There's an abundant future market in that direction. Several billion dollar businesses started as open source clones of existing things. Some found unique business lines, others simply did a better job.
  - CyberDildonics 3mo ago
    Everything already exists Then why are you trying to make something?
    - echelon 3mo ago
      Why make new music or new movies? The same things have been said by every previous generation. Just listen to your parents' music, right? Use the same technology, the same fashion, everything. Why? Because it's incremental improvement. Because it's making a dent in the world. Because creation feels better than consumption. Because we're maybe 10-20 iterations away (250 years, tops?) from brain uploads and everlasting life harvesting the energy of distant stars. Or at least our descendants are. And that's pretty cool.

overgard 3mo ago

One thing that I find frustrating/disturbing about LLM generated code is that wrong stuff doesn't usually look wrong, so you almost have to review it even closer than what you'd review from a person. Like in the past when a developer was being sloppy, usually you could kind of tell superficially, but with LLM code it's usually well documented and superficially well structured, all while doing batshit insane things. Like today I noticed some code that it had written was just silently returning on errors without logging a warning or returning an error code. It "worked" but it was a major foot-gun!

m3kw9 3mo ago
double edge sword of LLM code generation, you cannot catch 100%, nor do you want to always ready every single word. Gotta develop a feel.
- bluefirebrand 3mo ago
  Engineering by feels... Is this what we've come to?
  - m3kw9 3mo ago
    Yes, feel, not your type of feel, my type where you can scan a section and know it's this architecture, compartmentized, know the general performance impact. Yes.
learningstud 3mo ago
Too true. I realized that the only way to do an actual review is to have wrote it yourself in the first place. The relatively slow speed of writing code reflects the amount of subtleties and effort to think through all of them. Only when LLMs are used as hint givers, are they actually doing an essential speedup. This goes to show how shallow programming and code review had always been. The right impression is that of studying real analysis.

m3kw9 3mo ago

I can generate LLM code and it will be very stable long term, but you must be QA minded and non-stop testing, ask and make sure it generate maintainable code, watch for code smell etc. Maybe it won't match a very seasoned dev, but LLM's generate sht code because the dev allows it at the review stage.

gwbas1c 3mo ago

Some things to consider: 1: The quality of code varies widely from model to model. In Visual Studio Copilot, I get so-so code from GPT, great code from Claude 4.x, lousy results from Claude 5.x 2: "Novel work" is harder to get from an LLM than when it's following an example. For example, I can get great Blazor / HTML / css from an LLM (and then spend time iterating making small adjustments,) but if I'm writing a complicated DB query or implementing a novel algorithm, it's best to do by hand. 3: But, in the case of a novel algorithm or query, I get great results saying "please write unit tests for XXX". That alone makes me develop code twice as fast!