4 ms·

> Set up a nightly cron job that executes the prompt: fetch upstream changes to the <software> and rebase all local changes on top of upstream. Check that the s

by theamk 2mo ago

> Set up a nightly cron job that executes the prompt: fetch upstream changes to the <software> and rebase all local changes on top of upstream. Check that the software works as intended and replace the current version. This sounds like hell. You have unreliable actor redoing the software every night, and every day there is a chance you wake up and find your workflow broken. And no, "Check that the software works as intended" is not going to cut it, as AI are very, very good at obeying the letter but not the spirit of the ask. Yes, your diffs appear but they lack filenames. You've added this requirements to the prompt? Ok, filenames are back but they are font size 4, unreadably small. You want them well-seen? next week they become size 54, taking entire screen.. This is not a problem with regular AI development - you review the changes and test them a bit. But doing it day-to-day with no overview is just asking for trouble.

zephen 2mo ago
> This sounds like hell. You have unreliable actor... But, at least you know the cost up-front. Oh, wait...
arjie 2mo ago
I don’t auto-rebase but I definitely vendor in and choose to update on my own cadence.
- lanstin 2mo ago
  yeah, vendor in is the only way to stay sane over time, if you understand the cost of dependencies (and have a reasonable security model). And forking some Python or Go dep on Github and using that instead of the canonical one is pretty ergonomic. I don't know how Rust folks manage with so many dependencies via Cargo - my friend who loves Rust says they are smaller and more "one thing done well" but the complexity of hundreds or thousands of deps just scares me.
  - arjie 2mo ago
    My number one problem with Rust (which I otherwise like) is that Cargo allows build-time code execution so I need to run in a sandbox (unergonomic for me) or provide agent instructions to inspect `build.rs` style code in libraries etc. prior to compile. In practice it's easy to build and then ship a dev binary into a sandbox, but it's not so pleasant to build in a sandbox, so I don't want to get build-time pwned. Makes me very unhappy. Right now, I just have a build server that I ship things to and does lots of build-caching etc. so it's fine, but I would have preferred not to have a system that actively allows build-time thievery of my ~/.ssh
20k 2mo ago
I suspect the people advocating for this don't actually do this, its one of those things that you say because it sounds modern and up to date, and makes for a good sounding blogpost. The actual reality of any of what they're talking about is absolutely terrible Even just the cost of this would be a disaster, especially once everything goes to token based pricing
- raincole 2mo ago
  Some people even claim that they don't commit the code into source control, only the prompts. The best part about the internet is that you can publish any random shower thought.
  - CamperBob2 2mo ago
    Around 2020 I started getting downvoted for predicting that while the most popular programming languages at the time were Python, Java, and JavaScript, or whatever they were, the most popular programming languages in 2035 would be English and Mandarin. Looks more like 2030, after all. So the downvoters had a point.
    - jeremyjh 2mo ago
      I think having some real examples of this 2035 is a lot more feasible now than it would have looked to most people in 2020, but the majority of projects in 2030 is absurd. Can you name even one piece of software today that is actually built this way? This means: Every release, the only thing done is they edit a prompt and then ship the build that comes out of LLM -> CI/CD ? Then they delete all the source that was generated, along with the context, and only the prompt is carried forward to the next release?
      - CamperBob2 2mo ago
        What I'm talking about has only become possible in the last few months. Within a year you'll start to see new products built this way. Within five years you will see very few new products built any other way. Today's source code is tomorrow's object code.
g3f32r 2mo ago
> "Check that the software works as intended" is not going to cut it, as AI are very, very good at obeying the letter but not the spirit of the ask. Surely you don't think that he literally meant to use those seven words as a bullet proof prompt, right? I will never understand this way of responding literally and pedantically to verbal psudocode. He's leaving it as an exercise to the reader to specify "check that the software works as intended" because he could never possibly properly articulate a perfectly comprehensive QA and validation flow for every piece of software that would satisfy every individual's need.
- xmprt 2mo ago
  It's still the same problem. Now instead of checking if the code is correct, you have to check every day if your prompt is still validating the correct features that you expect to work.
- theamk 2mo ago
  And my point is, no matter how advanced you are going to make that prompt, AI will find a way to mess it up. If you say, "A, B and C must be true", then eventually an agent will find property D which you did not thing about, making the program unusable. Again, this is normal if you are interactively working on software and can fix any problems you see. It is only problematic if you do what OP says, and make it on cron, daily and unattended.
apitman 2mo ago
If your coding agent workflow isn't sandboxed, backed up, and rollback-able, it's fundamentally broken. And yes I realize most of us aren't working this way today. But I think we'll have the tooling pretty soon. There's currently a cambrian explosion of solutions in this space.
9dev 2mo ago
I don't know; I've been rebasing huge branches in really mean divergent states with Claude Code, and it handled it flawlessly pretty much every single time. So I don't think the actor is as unreliable as you think.
QGQBGdeZREunxLe 2mo ago
Workflow being broken is the least of your worries. This is a security nightmare.

by theamk 2mo ago

zephen 2mo ago

> This sounds like hell. You have unreliable actor... But, at least you know the cost up-front. Oh, wait...

arjie 2mo ago

I don’t auto-rebase but I definitely vendor in and choose to update on my own cadence.

lanstin 2mo ago
yeah, vendor in is the only way to stay sane over time, if you understand the cost of dependencies (and have a reasonable security model). And forking some Python or Go dep on Github and using that instead of the canonical one is pretty ergonomic. I don't know how Rust folks manage with so many dependencies via Cargo - my friend who loves Rust says they are smaller and more "one thing done well" but the complexity of hundreds or thousands of deps just scares me.
- arjie 2mo ago
  My number one problem with Rust (which I otherwise like) is that Cargo allows build-time code execution so I need to run in a sandbox (unergonomic for me) or provide agent instructions to inspect `build.rs` style code in libraries etc. prior to compile. In practice it's easy to build and then ship a dev binary into a sandbox, but it's not so pleasant to build in a sandbox, so I don't want to get build-time pwned. Makes me very unhappy. Right now, I just have a build server that I ship things to and does lots of build-caching etc. so it's fine, but I would have preferred not to have a system that actively allows build-time thievery of my ~/.ssh

20k 2mo ago

I suspect the people advocating for this don't actually do this, its one of those things that you say because it sounds modern and up to date, and makes for a good sounding blogpost. The actual reality of any of what they're talking about is absolutely terrible Even just the cost of this would be a disaster, especially once everything goes to token based pricing

raincole 2mo ago
Some people even claim that they don't commit the code into source control, only the prompts. The best part about the internet is that you can publish any random shower thought.
- CamperBob2 2mo ago
  Around 2020 I started getting downvoted for predicting that while the most popular programming languages at the time were Python, Java, and JavaScript, or whatever they were, the most popular programming languages in 2035 would be English and Mandarin. Looks more like 2030, after all. So the downvoters had a point.
  - jeremyjh 2mo ago
    I think having some real examples of this 2035 is a lot more feasible now than it would have looked to most people in 2020, but the majority of projects in 2030 is absurd. Can you name even one piece of software today that is actually built this way? This means: Every release, the only thing done is they edit a prompt and then ship the build that comes out of LLM -> CI/CD ? Then they delete all the source that was generated, along with the context, and only the prompt is carried forward to the next release?
    - CamperBob2 2mo ago
      What I'm talking about has only become possible in the last few months. Within a year you'll start to see new products built this way. Within five years you will see very few new products built any other way. Today's source code is tomorrow's object code.

g3f32r 2mo ago

> "Check that the software works as intended" is not going to cut it, as AI are very, very good at obeying the letter but not the spirit of the ask. Surely you don't think that he literally meant to use those seven words as a bullet proof prompt, right? I will never understand this way of responding literally and pedantically to verbal psudocode. He's leaving it as an exercise to the reader to specify "check that the software works as intended" because he could never possibly properly articulate a perfectly comprehensive QA and validation flow for every piece of software that would satisfy every individual's need.

xmprt 2mo ago
It's still the same problem. Now instead of checking if the code is correct, you have to check every day if your prompt is still validating the correct features that you expect to work.
theamk 2mo ago
And my point is, no matter how advanced you are going to make that prompt, AI will find a way to mess it up. If you say, "A, B and C must be true", then eventually an agent will find property D which you did not thing about, making the program unusable. Again, this is normal if you are interactively working on software and can fix any problems you see. It is only problematic if you do what OP says, and make it on cron, daily and unattended.

apitman 2mo ago

If your coding agent workflow isn't sandboxed, backed up, and rollback-able, it's fundamentally broken. And yes I realize most of us aren't working this way today. But I think we'll have the tooling pretty soon. There's currently a cambrian explosion of solutions in this space.

9dev 2mo ago

I don't know; I've been rebasing huge branches in really mean divergent states with Claude Code, and it handled it flawlessly pretty much every single time. So I don't think the actor is as unreliable as you think.

QGQBGdeZREunxLe 2mo ago

Workflow being broken is the least of your worries. This is a security nightmare.