6 ms·

I'm a convert. I was 100% skeptical about LLM code generation, now over 80% of the professional code I write is generated. That said, the limitations are kind

• runhelm 4mo ago
  [flagged]
• dominotw 4mo ago
  [flagged]
  • apsec112 4mo ago
    LLMs recently solved a major, famous open mathematical problem in combinatorial geometry: https://www.reddit.com/r/math/comments/1tj534d/openais_internal_model_disproves_unit_distance/ https://www.reddit.com/r/math/comments/1tj534d/openais_inter...
  • loeg 4mo ago
    There is nothing new under the sun.
    • son_of_gloin 4mo ago
      But there are other suns :)
      • yard2010 4mo ago
        There are other suns that nothing new under them
    • goatlover 4mo ago
      Everything changes, nothing remains without change.
      • coldtea 4mo ago
        Nah, most things stay the same withing quite narrow margins. The moon gets hit by a meteor now and then, but it's been essentially the same rock for some billion years.
        • goatlover 4mo ago
          Life didn't stay mostly the same over that time span. Evolution is a good counter to everything remaining the same under the sun. Anyway, Buddhists and Heraclitus aren't wrong. It's just a matter of enough time and the moon will no longer be a moon.
  • giwook 4mo ago
    And how much of pre-LLM code was just copy pasta from Stack Overflow? Code doesn't need to be novel to be useful. There's a reason why design patterns are a thing in software.
    • archagon 4mo ago
      That’s why we abstract the useful code away as libraries, frameworks, etc. AI is not an abstraction.
      • ryan_lane 4mo ago
        You generally need to wire libraries in to your service, and you may be using the library in a slightly different way than normal. AIs are perfectly capable of doing this. Back to the original point, though: most software engineering work isn't novel. Most people are working on slightly different iterations of the same thing, but with the aim of different products. You can have completely different products that use nearly the same patterns as most other services. To put it bluntly: we don't need AI to generate novel code for the vast majority of the software being built.
        • abalashov 4mo ago
          It depends on the programming. However, you're right: another Angular reinsurance accounting application doesn't pose much of a challenge for LLMs.
      • coldtea 4mo ago
        Of course it is. It abstracts away the code generation to a much more compressed natural language prompt. That's the very definition of a high abstraction...
        • hansmayer 4mo ago
          It abstracts shit mate. How many Rs in the strawberry and if you want to go to a car wash, should you walk, run or drive? They're just fucking text generators, which happen to spit out something half-usable about 60% of the time.
  • maytc 4mo ago
    Most code written are not novel. Actually most code written should not be novel. Eg: the number of lines of code written isn't spent on writing git, it's spent on writing that landing page no one ever sees.
  • pkulak 4mo ago
    If we were doing novel things, we’d be scientists. I’m an engineer though. I don’t think I’ve been writing slop for 30 years.
  • hmmokidk 4mo ago
    they can take novel things. so my novel
  • naveen99 4mo ago
    Unless you mean remixing the alphabet / tokens, this is mathematically false. 2^256 gets you to unique very fast, and that’s like 252 bytes or like 80 tokens. Remember almost all numbers are irrational. Complexity is infinite.
  • lurking_swe 4mo ago
    > you are merely remixing whats out there So basically 90% of programming in an enterprise environment? lol. Sounds useful to me...
  • coldtea 4mo ago
    I don't think you understand how LLMs work. They're not merely re-arranging pre-existing blocks of code. And they have been shown to develop emergent properties that weren't in their training set time and again. They generate novel things as much as the average programmer (which works after himself having practice, exposure to codebases, and training, and reading API documentation, and such) generates novel things.
    • sirsinsalot 4mo ago
      Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe. What makes the behavior emergent is that it can't be predicted at training time. The emergent and unpredictable output is the result of massive vector complexity being encoded.
      • coldtea 4mo ago
        >Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe. You are either being pedantic or missing the point of emergent however. Yes, it's not some novel unforeseen thing, like a magical Marvel Universe material or some unknown to humanity mode of thinking. Same way when people make something new they still recombine known words, or colors, or physical things in the universe. It is however new capabilities that is not explicitly in the training set and can't be predicted by it. Like teaching something only calculus training materials and it figures out boolean algebra. >The emergent and unpredictable output is the result of massive vector complexity being encoded As opposed to what in humans? God given revelation?
        • sirsinsalot 4mo ago
          My point was that if training data + encoding/training = model with emergent behaviour The emergent behaviour is in the training data and/or encoding/training. So while I agree it is emergent from the complexity, it isn't some unknown mechanism. Just complexity at scale.
  • phantompeace 4mo ago
    Source?
  • __mharrison__ 4mo ago
    99% of coders don't need to generate anything novel.
    • dominotw 4mo ago
      then what are they doing with the time savings from llm. generating more remixing ? is there really so much demand for remix slop. i dont think so.
      • tpmoney 4mo ago
        I don't think I've ever worked on a project where there wasn't more work to be done than there was time to do it in.
        • dominotw 4mo ago
          ai can remix things on the fly so lots of widgets that were being stamped on are not needed anymore. we are in middle state where ai tools to generate on the fly widget work arent accessible in the form that most ppl need. So programmers are currently doing the manual step by managing remix into easily consumable form.
      • __mharrison__ 4mo ago
        My job is different from most developers these days. But I'm writing more code than I ever did as a developer. YMMV
• mike_hock 4mo ago
  > moving complexity from the more formal and deterministic world of programming languages to the informal and non-deterministic world of natural language It's like using a compiler that generates semantically different code every time you run it. Basically like compiling a program that's full of UB but "seems to work" most of the time. > business sees this as productivity gains Back to LoC/s as a measure of "productivity."
  • somewhatgoated 4mo ago
    > Back to LoC/s as a measure of "productivity." IMO this doesn’t follow from what OP wrote. I personally measure it with a more abstract “how long does it take me to ship something that is useful in production and solving a real problem” and the increase in speed there has been massive for me. But of course I’m not a bigbrain 10x coder that is doing bleeding edge novel stuff like most people here, so gains might be more obvious for me than for others.
    • sdevonoes 4mo ago
      > how long does it take me to ship something that is useful in production and solving a real problem But that’s only half of the problem. What about “and how easy it is to maintain long-term”. If you say that maintenance can be done via LLM, I would argue that there is zero guarantees that LLMs are backwards compatible and that the markdown you wrote now will work just as fine in 1,2,3 years
      • coldtea 4mo ago
        >I would argue that there is zero guarantees that LLMs are backwards compatible and that the markdown you wrote now will work just as fine in 1,2,3 years That this would be the case is even more guaranteed than some programming language being backwards compatible and the code we wrote working just as fine in 1,2,3, years. Languages do get non-backwards compatible changes, dependencies break, stuff is deprecated, etc. But the job of LLMs will remain to generate something from a prompt, and the markdown we wrote, as it's high level and not tied to language versions, APIs, and implementation details, will be just as good a prompt for that in 2050 as it is in 2026.
• bob1029 4mo ago
  I'm not having much trouble with very large (>50mb raw source) and complex codebases. The fact that it's all strongly typed probably helps a lot, but I don't think that's the whole story. I think the harness and code patching technique starts to matter a lot more once you get outside the trivial range of codebases that fit within the first ~20% of the context window and can otherwise be iterated completely in a single inference pass. The apply_patch technique that OAI has polished their models on seems to be the best approach for monster scale codebases. Anything based on line ranges and simple find-replace will disintegrate at the edges. You need multiple spatial anchors to deal with nasty things like cshtml files. The prepare/commit behavior is ideal for iterating through ambiguous contexts across many large files and refining anchors.
• TeMPOraL 4mo ago
  > yet the trade off seems to be clear and a lot of people are just ignoring it. There's plenty of focus on the negative side of the tradeoff. Less so on why we're making it anyway, or why it somehow works out even if "this starts to look like we're all just moving complexity from the more formal and deterministic world of programming languages to the informal and non-deterministic world of natural language". And the answer to that can be condensed to a one-liner, which I quote after[0]: sizeof(docs) << sizeof(code) -- [0] - https://drensin.medium.com/elephants-goldfish-and-the-new-golden-age-of-software-engineering-c33641a48874 https://drensin.medium.com/elephants-goldfish-and-the-new-go... - article may be a bit fluffy here and there, but that one line was a big insight for me.
  • sdevonoes 4mo ago
    I don’t think that’s true based on experience. Maybe “<“ instead of “<<“, yeah. But even in that case, it’s an awful trade off for any serious codebase that needs to be maintained over the years (and you don’t know what LLMs are gonna look like next year, so there are zero guarantees all your MD is gonna work as good as it’s “working” right now)
    • coldtea 4mo ago
      As long as LLMs remains at the same skill level at coding, or better, there's 100% guarantee an MD (a glorified prompt) is gonna work as good as it’s “working” right now.
      • andwur 4mo ago
        This is quite a claim without any evidence to substantiate it. LLMs are nondeterministic models, whose behaviour is reliant on training data, model architecture and context (both in the general and domain specific sense). There is absolutely no guarantee llm1(MD) == llm2(MD), by design. With the current batch you need to explicitly constrain a number of parameters, far more than simply the prompt, to get identical output from the _same_ model, let alone another model that has varied training data and/or architecture.
      • 4mo ago
• sdevonoes 4mo ago
  > At some point this starts to look like we're all just moving complexity from the more formal and deterministic world of programming languages to the informal and non-deterministic world of natural language. This is the problem nobody is talking about. I see codebases growing in MD files with instructions and guidelines and requests that are also LLM generated… and it’s all piling up. No one is reviewing it 100% , and even when we do, it’s all very subjective. What’s the difference between “Follow a RESTful approach”, “We use REST, not graphql”, “90% of our endpoints are resource oriented, but we have a couple of endpoints that look rpc-ish; please ignore the latter”… It’s all very stupid.
  • tcoff91 4mo ago
    This is why you need to be generating more linter rules instead of just having things be in markdown files. I had never written an eslint rule until i started having agents pump them out for me and now I've encoded a bunch of important rules as lint rules that will fail CI if violated.
    • deadbabe 4mo ago
      Who lints the linters
      • jckahn 4mo ago
        Linter linters, obviously
        • tcoff91 4mo ago
          It's linters all the way down
    • hansmayer 4mo ago
      A linter won't prevent your idiot LLM from going bonkers and suddenly switching to GQL instead of REST just for that one endpoint, because it confabulated something or putting your stripe secret into your react frontend - all cases of slop I've seen happen.
• kang 4mo ago
  hehe, this is by design. next model needs to eat your natural language
• tpmoney 4mo ago
  One question I have is are these "constraints, style guides, corner cases, error handling, optimization guidelines" extra things that you wouldn't need otherwise, or are they formal documentation of the baked in assumptions and knowledge accumulated over the years? Every project I've ever worked on has had heaps of shared knowledge that's just part of stuff the team just "knows" and no one ever really writes down. Things like "sure you can use java's built in assert for tests, but we don't compile or run the application with the flags that enable them. Use junit's assertions/use the assertj library." or "prefer using auto generated accessors instead of manually writing them out". Even things like "if you change the structure of this ID string, you need to change all the code in modules A, B, and C because they all rely on the ID being in a certain format". If you're really lucky, maybe a lot of this is documented in some wiki page somewhere, but everyone knows the documentation is never as complete as you'd like it to be. The longer a team works together without new people coming on board, the more likely it is that the documentation of these soft requirements and knowledge has drifted from reality. IME nothing shows how much you've failed to document than revisiting your onboarding process documents for the first time 2-3 years after you wrote them. As I've experimented with the various AI tools, I feel like a lot of these extra documents I've written are documenting a lot of these things "everyone knows". But I'm also not at the "80% of the professional code I write is generated" stage yet. So I'm curious if you're finding that you're creating documentation that goes beyond just documenting what we used to just keep in our heads and are now getting into "writing a book about how to code" territory?
  • bitexploder 4mo ago
    There is a great thing. Because the agents can do so much toil you can add things like formal verification, fuzzing, and other feedback mechanisms and quality gates to your projects cheaply. In a human written project you still needed those things, but it cost a lot. Agents require these quality gates and they can implement them for you. The problem with AI documentation is it will just write a lot of useless bullshit unless you guide it on what is important. You can also get agents to identify transitive dependencies via testing and other things. I adopt the mindset of docs are for humans, tests are for agents. They document formal dependencies and leave a measurable artifact behind. If you identify some behavior or transitive dep in your system, agents document it first with a test codifying the expected behavior. Tests are the source of truth about expected system behavior and you can convince agents to write decent behavioral tests if you ask them to with the right structure. Docs are now cheap and a render, not a long term thing. There is some token efficiency to consider, but still, they are quick and cheap if you don't understand some module or its purpose.
• hansmayer 4mo ago
  > I have to start adding more and more constraints, style guides, corner cases, error handling, optimization guidelines and all this good stuff to my Markdown specifications, rules and skills So kind of like maintaining a growing codebase? But this time around you cannot guarantee what the outputs will be?
• tomascantor 4mo ago
  [flagged]