6 ms·

I used to assume they pushed people into the prompt-only workflows because you’re paying them for the tokens, and not paying them for the scaffolding you built.

I used to assume they pushed people into the prompt-only workflows because you’re paying them for the tokens, and not paying them for the scaffolding you built. However, I think that they’re really worried about is that a person needs to design and implement that stuff… It throws a wet blanket on their insistence that this will replace entire people in entire workflows or even projects, and I just don’t buy it. I do think it’s going to increase productivity enough to disastrously affect developer job market/pay scale, but I just don’t think this particular version of this particular technology is going to actually do what they say it will. If they said they were spending this much money bootstrapping a super useful thingy that can reduce a big chunk of the busy work of a human dev team— what most developers really want, and most executives really don’t— a bunch of investors would make them walk the plank. I also think having granular, tightly controlled steps is much friendlier to implementing smaller, cheaper, more specialized models rather than using some ginormous behemoth of a model that can automate your tests, or crank out 5 novels of CSI fan fic in a snap.

pishpash 5mo ago
Aren't they just buying time to build you whatever harness you need? They want to be the only software engineering shop in the world.
cogman10 5mo ago
> However, I think that they’re really worried about is that a person needs to design and implement that stuff… It throws a wet blanket on their insistence that this will replace entire people in entire workflows or even projects, and I just don’t buy it. I think you are on to something. But I also think this sort of system lends itself to not needing really good LLMs to do impressive things. I've noticed that the quality of a lot of these LLMs just gets worse the more datapoints they need to track. But, if you break it up into smaller and easier to consume chunks all the sudden you need a much less capable LLM to get results comparable or better than the SOTA. Why pay extra money for Opus 4.7 when you could run Qwen 3.6 35b for free and get similar results?
- tempest_ 5mo ago
  It is also interesting because you get people with very different use cases arguing about the effectiveness of various models but doing very different things with them. Its one things for a model to be very clearly instructed to add a REST endpoint to an existing Django app and add a button connected to it on the front vs "Design me a youtube". The smaller models can pretty dependably do the first and fall flat on the second.
- devin 5mo ago
  And then you realize that what you’re using the smaller models for is ALSO decomposable and part of it is just a few if statements, and then you realize that for this feature you don’t actually need or want a model because the performance, reliability, reproducibility are cheaper and better for you and your users.
  - jimbokun 5mo ago
    So you have the model write the if statements and put itself out of a job.
    - gman2093 5mo ago
      Alternatively, and sometimes more cost-efficient: you can find a developer who can write bespoke if statements. There are dozens of us!
      - saltcured 5mo ago
        So, are we going to end up with a mechanical Turk that pretends it is an LLM but just farms out tasks to gig workers?
      - DrewADesign 5mo ago
        Additionally, developers tend to become less expensive as venture capitalists turn off the spigot, while access to giant frontier models becomes way more expensive. Beyond that, a developer might go out and have a beer with you after work, which appeals to the sickos that have the gall to prioritize humanity over fanatical efficiency for corporate gains.
user34283 5mo ago
The designing and implementing of a code harness in your workflow can be as simple as running something like /skill-builder. You prompt for what you want it to do, and it will write eg. python scripts as needed for the looping part, and for example use claude -p for the LLM call. You can build this in 10 minutes. I don’t use a cloud platform, so I can’t comment on that part. I‘d say just run it on your own hardware, it’s probably cheaper too.
zozbot234 5mo ago
> However, I think that they’re really worried about is that a person needs to design and implement that stuff… It throws a wet blanket on their insistence that this will replace entire people in entire workflows or even projects You can have the AI design the custom harness in advance. It's not especially hard work! In fact, the AI could even come up with the workflow itself; it's a different and much simpler problem than trying to stick to it after-the-fact, with a filled-in context.
- AndyNemmity 5mo ago
  That's what my system does. It uses a workflow if one already exists, if not, it just creates one on the fly from the primitives. https://github.com/notque/vexjoy-agent https://github.com/notque/vexjoy-agent I would prefer that be deterministic though. This thread has me considering what if anything I can do to make it forced. Like, I could do it with hooks, but that's not elegant at all.
- mf_kevintruong 5mo ago
  yeah , that is what I am do in with the DAG-aware TUI hypervisor agents https://getspur.dev https://getspur.dev

pishpash 5mo ago

Aren't they just buying time to build you whatever harness you need? They want to be the only software engineering shop in the world.

cogman10 5mo ago

> However, I think that they’re really worried about is that a person needs to design and implement that stuff… It throws a wet blanket on their insistence that this will replace entire people in entire workflows or even projects, and I just don’t buy it. I think you are on to something. But I also think this sort of system lends itself to not needing really good LLMs to do impressive things. I've noticed that the quality of a lot of these LLMs just gets worse the more datapoints they need to track. But, if you break it up into smaller and easier to consume chunks all the sudden you need a much less capable LLM to get results comparable or better than the SOTA. Why pay extra money for Opus 4.7 when you could run Qwen 3.6 35b for free and get similar results?

tempest_ 5mo ago
It is also interesting because you get people with very different use cases arguing about the effectiveness of various models but doing very different things with them. Its one things for a model to be very clearly instructed to add a REST endpoint to an existing Django app and add a button connected to it on the front vs "Design me a youtube". The smaller models can pretty dependably do the first and fall flat on the second.
devin 5mo ago
And then you realize that what you’re using the smaller models for is ALSO decomposable and part of it is just a few if statements, and then you realize that for this feature you don’t actually need or want a model because the performance, reliability, reproducibility are cheaper and better for you and your users.
- jimbokun 5mo ago
  So you have the model write the if statements and put itself out of a job.
  - gman2093 5mo ago
    Alternatively, and sometimes more cost-efficient: you can find a developer who can write bespoke if statements. There are dozens of us!
    - saltcured 5mo ago
      So, are we going to end up with a mechanical Turk that pretends it is an LLM but just farms out tasks to gig workers?
    - DrewADesign 5mo ago
      Additionally, developers tend to become less expensive as venture capitalists turn off the spigot, while access to giant frontier models becomes way more expensive. Beyond that, a developer might go out and have a beer with you after work, which appeals to the sickos that have the gall to prioritize humanity over fanatical efficiency for corporate gains.

user34283 5mo ago

The designing and implementing of a code harness in your workflow can be as simple as running something like /skill-builder. You prompt for what you want it to do, and it will write eg. python scripts as needed for the looping part, and for example use claude -p for the LLM call. You can build this in 10 minutes. I don’t use a cloud platform, so I can’t comment on that part. I‘d say just run it on your own hardware, it’s probably cheaper too.

zozbot234 5mo ago

> However, I think that they’re really worried about is that a person needs to design and implement that stuff… It throws a wet blanket on their insistence that this will replace entire people in entire workflows or even projects You can have the AI design the custom harness in advance. It's not especially hard work! In fact, the AI could even come up with the workflow itself; it's a different and much simpler problem than trying to stick to it after-the-fact, with a filled-in context.

AndyNemmity 5mo ago
That's what my system does. It uses a workflow if one already exists, if not, it just creates one on the fly from the primitives. https://github.com/notque/vexjoy-agent https://github.com/notque/vexjoy-agent I would prefer that be deterministic though. This thread has me considering what if anything I can do to make it forced. Like, I could do it with hooks, but that's not elegant at all.
mf_kevintruong 5mo ago
yeah , that is what I am do in with the DAG-aware TUI hypervisor agents https://getspur.dev https://getspur.dev