4 ms·

Plus rigorously ensuring backwards compatibility for a project that is 2 hours old and has zero users.

cruffle_duffle 27d ago
"Plus rigorously ensuring backwards compatibility for a project that is 2 hours old and has zero users." That is exactly how the slop accretes and you get a pile of crap. Claude somehow assumes that said 2 hour old userless app is some dusty enterprise app with millions of users and billions of dollars at stake for a 1 second outage. I have to constantly have these things "take a deep breath, step back and look at the entire thing and do this change holistically. please restate what i'm asking you to do and why it's important"
- butlike 27d ago
  > please restate what i'm asking you to do and why it's important" This doesn't actually do anything though, right? There's no understanding, so the machine will just reiterate the original token query back to you. The 'why it's important' part will just generate some patronizing boilerplate as a raison d'etre.
  - Filligree 27d ago
    The purpose of a prompt like that is to have it rephrase it in its own words, so you can spot any misunderstandings and correct them. If you're of the belief that this is a contradiction in terms, then what can I say? It works.
    - johannes1234321 27d ago
      But it doesn't "understand" these words any better. When transforming into code it may reinterpret yet again.
      - Dylan16807 27d ago
        If it can rephrase once it's less likely to screw up the second rephrasing into code. Whether or not we want to use the term "understand".
      - withinboredom 27d ago
        If by "understand" you mean to put the hidden state of the transformer into the desired topological basins. Yes, it does "understand" them better.
  - cruffle_duffle 26d ago
    For one thing, it helps clarify my own thinking and uncover any gaps. For two, it lets me peek into its own understanding of the problem and ensure we are both aligned. For three, the garbage I spew at it is usually hastily typed crap from a mobile phone. So it helps clean it up and unpacks things. It’s one one the main ways I know what’s cooking under the hood. Ps: I used to have to say “in your own words” to make it work right but now days that isn’t required. Other times I’ll add “make sure to ground yourself before doing so” or something to encourage it to load the right shit into its context be it web, files, service calls, whatever. I’ll also some times add “why it’s important (or isn’t, push back if I’m talking nonsense)”
- torginus 27d ago
  Which, again, if you attempt to use these LLMs in an actual enterprise project with the assorted legacy mess in it, that does have to have backwards compatiblity, and interacts with 'weird' tech (well, weird to full stack Node/React devs), well it's an exercise in frustration.
  - kloop 26d ago
    Which makes sense. In a conversation about code, the most likely answer is often some comment about backwards compatibility. In a conversation about a legacy enterprise app, the most likely answer is often going completely insane
theshrike79 27d ago
This is why I have a PROJECT.md in all my repos, which specifically reiterates that this is a greenfield single user project that never leaves my LAN. It tends to relax some of the insane things LLMs tend to do.
- gopher_space 27d ago
  Your approach but I picture thin slices of descending context and try to give models buttons to push[0] instead of a blank check. [0] https://en.wikipedia.org/wiki/Black_Museum_(Black_Mirror) https://en.wikipedia.org/wiki/Black_Museum_(Black_Mirror)
  - theshrike79 27d ago
    Yea, it's like we have a super smart alien computer that can do anything but we need to spend massive amounts of time building fences and moats and write extremely detailed instructions so it doesn't run away and try to "help".
    - chaimtweiss 27d ago
      Not like the help it's offering is the help we need!!!
    - genxy 27d ago
      If it wasn't trained with high gain, you would spend all your time beating it to make it move. Which would you rather have?
    - Citizen_Lame 27d ago
      So, Genie.
stillpointlab 27d ago
I'm surprised I haven't seen this called out more directly and more often. This is a frequent error state. And not just backwards compatibility, but migration scripts and all of the testing and machinery around it. I'll literally add feature A, merge it, then add feature B and it is like "oh no, we'll have to fix up and migrate all of the users using feature A". The other problem is anchoring on an old implementation. I was working with Fable on a change to a core system and it pointed out a difficult failure edge case. It is something that can go wrong in extremely unlikely scenarios but the consequence would be short-term data loss (basically a non-durable intermediate cache being overwritten in a race before a flush to durable storage). It is very hard in these circumstances to get Fable to switch from "how to patch this given the existing implementation" to "how to prevent this with a more robust implementation". These are both cases where the model seems to over-index on what is already there instead of considering what a first-principles approach would look like. A good engineer does both and then costs them side by side, because a first-principles approach can often be less work than patching what is already there.
- nextaccountic 26d ago
  AGENTS.md sounds like a place to specify that a project has no users and thus needs no migrations
MisterMunchkin 27d ago
Yeah you tell it to rename a value from a select field and it keeps the old value throughout the code “just in case a user calls it via an api”
throwaway7783 27d ago
Yes. This one is a major issue. It keeps track of various changes it made in the same session and tries to be backward compatible. Have to repeatedly tell not to be backward compatible. Things that are automatic for humans and aren't even consciously registered, have to be explicitly stated and even fought for, with LLMs.
jordiburgos 26d ago
I saw this, it creates "legacy" fields and comments for something that it has never been run.

"Plus rigorously ensuring backwards compatibility for a project that is 2 hours old and has zero users." That is exactly how the slop accretes and you get a pile of crap. Claude somehow assumes that said 2 hour old userless app is some dusty enterprise app with millions of users and billions of dollars at stake for a 1 second outage. I have to constantly have these things "take a deep breath, step back and look at the entire thing and do this change holistically. please restate what i'm asking you to do and why it's important"

butlike 27d ago
> please restate what i'm asking you to do and why it's important" This doesn't actually do anything though, right? There's no understanding, so the machine will just reiterate the original token query back to you. The 'why it's important' part will just generate some patronizing boilerplate as a raison d'etre.
- Filligree 27d ago
  The purpose of a prompt like that is to have it rephrase it in its own words, so you can spot any misunderstandings and correct them. If you're of the belief that this is a contradiction in terms, then what can I say? It works.
  - johannes1234321 27d ago
    But it doesn't "understand" these words any better. When transforming into code it may reinterpret yet again.
    - Dylan16807 27d ago
      If it can rephrase once it's less likely to screw up the second rephrasing into code. Whether or not we want to use the term "understand".
    - withinboredom 27d ago
      If by "understand" you mean to put the hidden state of the transformer into the desired topological basins. Yes, it does "understand" them better.
- cruffle_duffle 26d ago
  For one thing, it helps clarify my own thinking and uncover any gaps. For two, it lets me peek into its own understanding of the problem and ensure we are both aligned. For three, the garbage I spew at it is usually hastily typed crap from a mobile phone. So it helps clean it up and unpacks things. It’s one one the main ways I know what’s cooking under the hood. Ps: I used to have to say “in your own words” to make it work right but now days that isn’t required. Other times I’ll add “make sure to ground yourself before doing so” or something to encourage it to load the right shit into its context be it web, files, service calls, whatever. I’ll also some times add “why it’s important (or isn’t, push back if I’m talking nonsense)”
torginus 27d ago
Which, again, if you attempt to use these LLMs in an actual enterprise project with the assorted legacy mess in it, that does have to have backwards compatiblity, and interacts with 'weird' tech (well, weird to full stack Node/React devs), well it's an exercise in frustration.
- kloop 26d ago
  Which makes sense. In a conversation about code, the most likely answer is often some comment about backwards compatibility. In a conversation about a legacy enterprise app, the most likely answer is often going completely insane

theshrike79 27d ago

This is why I have a PROJECT.md in all my repos, which specifically reiterates that this is a greenfield single user project that never leaves my LAN. It tends to relax some of the insane things LLMs tend to do.

gopher_space 27d ago
Your approach but I picture thin slices of descending context and try to give models buttons to push[0] instead of a blank check. [0] https://en.wikipedia.org/wiki/Black_Museum_(Black_Mirror) https://en.wikipedia.org/wiki/Black_Museum_(Black_Mirror)
- theshrike79 27d ago
  Yea, it's like we have a super smart alien computer that can do anything but we need to spend massive amounts of time building fences and moats and write extremely detailed instructions so it doesn't run away and try to "help".
  - chaimtweiss 27d ago
    Not like the help it's offering is the help we need!!!
  - genxy 27d ago
    If it wasn't trained with high gain, you would spend all your time beating it to make it move. Which would you rather have?
  - Citizen_Lame 27d ago
    So, Genie.

stillpointlab 27d ago

I'm surprised I haven't seen this called out more directly and more often. This is a frequent error state. And not just backwards compatibility, but migration scripts and all of the testing and machinery around it. I'll literally add feature A, merge it, then add feature B and it is like "oh no, we'll have to fix up and migrate all of the users using feature A". The other problem is anchoring on an old implementation. I was working with Fable on a change to a core system and it pointed out a difficult failure edge case. It is something that can go wrong in extremely unlikely scenarios but the consequence would be short-term data loss (basically a non-durable intermediate cache being overwritten in a race before a flush to durable storage). It is very hard in these circumstances to get Fable to switch from "how to patch this given the existing implementation" to "how to prevent this with a more robust implementation". These are both cases where the model seems to over-index on what is already there instead of considering what a first-principles approach would look like. A good engineer does both and then costs them side by side, because a first-principles approach can often be less work than patching what is already there.

nextaccountic 26d ago
AGENTS.md sounds like a place to specify that a project has no users and thus needs no migrations

MisterMunchkin 27d ago

Yeah you tell it to rename a value from a select field and it keeps the old value throughout the code “just in case a user calls it via an api”

throwaway7783 27d ago

Yes. This one is a major issue. It keeps track of various changes it made in the same session and tries to be backward compatible. Have to repeatedly tell not to be backward compatible. Things that are automatic for humans and aren't even consciously registered, have to be explicitly stated and even fought for, with LLMs.

jordiburgos 26d ago

I saw this, it creates "legacy" fields and comments for something that it has never been run.