7 ms·

I meant reference Toby Ord's work here. I think his framing of the performance/cost frontier hasn't gotten enough attention https://www.tobyord.com/writing/hour

I meant reference Toby Ord's work here. I think his framing of the performance/cost frontier hasn't gotten enough attention https://www.tobyord.com/writing/hourly-costs-for-ai-agents https://www.tobyord.com/writing/hourly-costs-for-ai-agents

fragmede 6mo ago
That post doesn't address the human factor of cost, and I don't mean that in a good way. Even if AI costs more than a human, it's tireless, doesn't need holidays, is never going to have to go to HR for sexual harassment issues, won't show up hungover or need an advance to pay for a dying relative's surgery. It can be turned on and off with the flip of a switch. Hire 30 today, fire 25 of them next week. Spin another 5 up just before the trade show demo needs to go out and fire them with no remorse afterwards.
- pona-a 6mo ago
  I think the word you're looking for is contractors. But yes, you still have to treat those with _some_ human decency.
- piker 6mo ago
  That was a great promise before the models starting becoming "moody" due to their proprietors arbitrarily modifying their performance capabilities and defaults without transparency or recourse.
  - mh- 6mo ago
    I still haven't seen any statistically sound data supporting that this is happening on the API (per-token pricing.) If you've got something to share I'd love to see it.
    - jessermejia 6mo ago
      There's an interesting analysis here: https://github.com/anthropics/claude-code/issues/42796 https://github.com/anthropics/claude-code/issues/42796 >The most striking row is user prompts: 5,608 in February vs 5,701 in March. The human put in the same effort. But the model consumed 80x more API requests and 64x more output tokens to produce demonstrably worse results.
      - deleted 6mo ago
        [deleted]
      - mh- 6mo ago
        Sorry, "this" referred to the parent comment's claim. > models starting becoming "moody" due to their proprietors arbitrarily modifying their performance capabilities The tokenizer changes are measurable, the above is quite difficult to quantify. There are a few sites floating around that purport to, but all of them have fatal flaws in their methodology.
        fragmede 6mo ago
        Unfortunately, LLM performance isn't an exact science and some observations are going to be subjective. Observations like ChatGPT being "lazy" in the Winter. Wanting to form opinions based on hard data, aka science, and not vibes is entirely reasonable but doesn't make the vibes a figment of imagination. Or as Jeff Bezos put it, "When the data and the anecdotes disagree, the anecdotes are usually right." And while he's not a scientist, his success does put some weight behind that quote. (as does digging deeper in what he meant by that.)
- louiereederson 6mo ago
  I think it's difficult to say agentic and human developer labor are fungible in the real world at this point. Agents may succeed in discrete tasks, like those in a benchmark assessment, but those requiring a larger context window (i.e. working in brownfield systems, which is arguably the bulk of development work) favor developers for now. Not to mention that at this point a lot of necessary context is not encoded in an enterprise system, but lives in people's heads. I'd also flip your framing on its head. One of the advantages of human labor over agents is accountability. Someone needs to own the work at the end of the day, and the incentive alignment is stronger for humans given that there is a real cost to being fired.
  - kennywinker 6mo ago
    For some the appeal of agent over human is the lack of accountability. “Agent, find me ten targets in iran to blow up” - “Okay, great idea! This military strike isn’t just innovative - it’s game changing! A reddit comment from ten years ago says that military often uses schools to hide weapons, so here is a list of the ten most crowded schools in Iran”
    - Our_Benefactors 6mo ago
      It must be wild to actually go through life believing the things written in this post and also thinking you have a rational worldview.
- cyanydeez 6mo ago
  it just will delete production database when flustered. no biggie. we learning how to socialize again. cant let all that history go to waste.
  - cindyllm 6mo ago
    [dead]
- krainboltgreene 6mo ago
  Ah-ha, the perfect slave.
- michaelbuckbee 6mo ago
  More importantly it collapses mythical-man-month communication overhead.
  - jonfromsf 6mo ago
    This is an amazing frame /reframe.
  - tclancy 6mo ago
    Hang on, tell me how, because I am not picking up what you are putting down. At a minimum, wouldn’t this require working from a perfectly written spec that has already accounted for the discovery of changes that would need to be made from the original perfect spec?
    - fragmede 6mo ago
      The way I see it, because you can spin up additional AI employees at will (and spin them back down), when the problem with the spec is found, it's no big deal to redo all of that work from before, adjusting for that change.
      - mainguy 6mo ago
        Ironically, people keep saying this, but then gloss over the core problem of coordinated between these agents... For completely independent codebases with no dependencies, sure thing right on...go for it. But the vast majority of F500 companies I work with have wild and undocumented dependencies between almost every system that will take years to "agentify" (assuming they ever figure out that it's an organization and governance problem.... which they might not ever realize)
    - michaelbuckbee 6mo ago
      So we have two things here: 1. "Mythical Man Month" which is the shorthand for a whole book + concept that you can't just throw more people at a software development project and get linear productivity improvements as the communications overhead (meetings, emails, mistakes due to poor assumptions, etc.) deeply eat into the raw number of productive hours that a new person added to the team brings. 2. AI automation tools (Claude Code) are often described as a "junior developer" which is an imperfect comparison as while you could potentially sort of set them up that way many people use them as more of a singular force multiplier. I use them to work on many more projects in many more ways and ship far more than I could even if I had a "junior developer" sitting alongside of me as there's not the same level of communication needed.
- lbreakjai 6mo ago
  The cost to hire a human is highly predictable. The cost of AI isn't. I, as a human, need food and shelter, which puts a ceiling to my bargaining power. I can't withdraw my labour indefinitely. The power dynamics are also vastly against me. I represent a fraction of my employer's labour, but my employer represents 100% of my income. That dynamic is totally inverted with AI. You are a rounding error on their revenue sheet, they have a monopoly on your work throughput. How do you budget an workforce that could turn 20% more expensive overnight?
  - bornfreddy 6mo ago
    By continuously testing competitors and local LLMs? The reason for rising prices is that they (Anthropic) probably realized that they have reached a ceiling of what LLMs are capable of, and while it's a lot, it is still not a big moat and it's definitely not intelligence.
    - noosphr 6mo ago
      Anything but the simplest tooling is not transferable between model generations, let alone completely different families.
      - skissane 6mo ago
        > Anything but the simplest tooling is not transferable between model generations, let alone completely different families. It is transferable-yes, you will get issues if you take prompts and workflows tuned for one model and send them to another unchanged. But, most of the time, fixing it is just tinkering with some prompt templates People port solutions between models all the time. It takes some work, but the amount of work involved is tractable Plus: this is absolutely the kind of task a coding agent can accelerate The biggest risk is if your solution is at the frontier of capability, and a competing model (even another frontier model) just can’t do it. But a lot of use cases, that isn’t the case. And even if that is the case today, decent odds in a few more months it won’t be
        6mo ago
- thrwaway55 6mo ago
  Why do you think it can't sexually harass someone or drive people to suicide. There are already lawsuits coming in on it causing suicides. This is an architecture that people are increasing begging to give network connectivity that can't differentiate its system prompt from user input
  - siffin 6mo ago
    I thought it already was used to sexually harass people by creating naked versions of them.
  - inquirerGeneral 6mo ago
    [dead]
  - fragmede 6mo ago
    As the NRA puts it, guns don't kill people, people kill people. Yes, a harasser now has access to more powerful tools to sexual harass somebody, but Claude, the AI, isn't going to randomly grab the secretary's ass and have to be fired for having done so at the holiday Christmas party.
dang 6mo ago
Let's give that one a SCP* re-up: https://news.ycombinator.com/item?id=47778922 https://news.ycombinator.com/item?id=47778922 (* explained at https://news.ycombinator.com/item?id=26998308 https://news.ycombinator.com/item?id=26998308)

fragmede 6mo ago

That post doesn't address the human factor of cost, and I don't mean that in a good way. Even if AI costs more than a human, it's tireless, doesn't need holidays, is never going to have to go to HR for sexual harassment issues, won't show up hungover or need an advance to pay for a dying relative's surgery. It can be turned on and off with the flip of a switch. Hire 30 today, fire 25 of them next week. Spin another 5 up just before the trade show demo needs to go out and fire them with no remorse afterwards.

pona-a 6mo ago
I think the word you're looking for is contractors. But yes, you still have to treat those with _some_ human decency.
piker 6mo ago
That was a great promise before the models starting becoming "moody" due to their proprietors arbitrarily modifying their performance capabilities and defaults without transparency or recourse.
- mh- 6mo ago
  I still haven't seen any statistically sound data supporting that this is happening on the API (per-token pricing.) If you've got something to share I'd love to see it.
  - jessermejia 6mo ago
    There's an interesting analysis here: https://github.com/anthropics/claude-code/issues/42796 https://github.com/anthropics/claude-code/issues/42796 >The most striking row is user prompts: 5,608 in February vs 5,701 in March. The human put in the same effort. But the model consumed 80x more API requests and 64x more output tokens to produce demonstrably worse results.
    - deleted 6mo ago
      [deleted]
    - mh- 6mo ago
      Sorry, "this" referred to the parent comment's claim. > models starting becoming "moody" due to their proprietors arbitrarily modifying their performance capabilities The tokenizer changes are measurable, the above is quite difficult to quantify. There are a few sites floating around that purport to, but all of them have fatal flaws in their methodology.
      - fragmede 6mo ago
        Unfortunately, LLM performance isn't an exact science and some observations are going to be subjective. Observations like ChatGPT being "lazy" in the Winter. Wanting to form opinions based on hard data, aka science, and not vibes is entirely reasonable but doesn't make the vibes a figment of imagination. Or as Jeff Bezos put it, "When the data and the anecdotes disagree, the anecdotes are usually right." And while he's not a scientist, his success does put some weight behind that quote. (as does digging deeper in what he meant by that.)
louiereederson 6mo ago
I think it's difficult to say agentic and human developer labor are fungible in the real world at this point. Agents may succeed in discrete tasks, like those in a benchmark assessment, but those requiring a larger context window (i.e. working in brownfield systems, which is arguably the bulk of development work) favor developers for now. Not to mention that at this point a lot of necessary context is not encoded in an enterprise system, but lives in people's heads. I'd also flip your framing on its head. One of the advantages of human labor over agents is accountability. Someone needs to own the work at the end of the day, and the incentive alignment is stronger for humans given that there is a real cost to being fired.
- kennywinker 6mo ago
  For some the appeal of agent over human is the lack of accountability. “Agent, find me ten targets in iran to blow up” - “Okay, great idea! This military strike isn’t just innovative - it’s game changing! A reddit comment from ten years ago says that military often uses schools to hide weapons, so here is a list of the ten most crowded schools in Iran”
  - Our_Benefactors 6mo ago
    It must be wild to actually go through life believing the things written in this post and also thinking you have a rational worldview.
cyanydeez 6mo ago
it just will delete production database when flustered. no biggie. we learning how to socialize again. cant let all that history go to waste.
- cindyllm 6mo ago
  [dead]
krainboltgreene 6mo ago
Ah-ha, the perfect slave.
michaelbuckbee 6mo ago
More importantly it collapses mythical-man-month communication overhead.
- jonfromsf 6mo ago
  This is an amazing frame /reframe.
- tclancy 6mo ago
  Hang on, tell me how, because I am not picking up what you are putting down. At a minimum, wouldn’t this require working from a perfectly written spec that has already accounted for the discovery of changes that would need to be made from the original perfect spec?
  - fragmede 6mo ago
    The way I see it, because you can spin up additional AI employees at will (and spin them back down), when the problem with the spec is found, it's no big deal to redo all of that work from before, adjusting for that change.
    - mainguy 6mo ago
      Ironically, people keep saying this, but then gloss over the core problem of coordinated between these agents... For completely independent codebases with no dependencies, sure thing right on...go for it. But the vast majority of F500 companies I work with have wild and undocumented dependencies between almost every system that will take years to "agentify" (assuming they ever figure out that it's an organization and governance problem.... which they might not ever realize)
  - michaelbuckbee 6mo ago
    So we have two things here: 1. "Mythical Man Month" which is the shorthand for a whole book + concept that you can't just throw more people at a software development project and get linear productivity improvements as the communications overhead (meetings, emails, mistakes due to poor assumptions, etc.) deeply eat into the raw number of productive hours that a new person added to the team brings. 2. AI automation tools (Claude Code) are often described as a "junior developer" which is an imperfect comparison as while you could potentially sort of set them up that way many people use them as more of a singular force multiplier. I use them to work on many more projects in many more ways and ship far more than I could even if I had a "junior developer" sitting alongside of me as there's not the same level of communication needed.
lbreakjai 6mo ago
The cost to hire a human is highly predictable. The cost of AI isn't. I, as a human, need food and shelter, which puts a ceiling to my bargaining power. I can't withdraw my labour indefinitely. The power dynamics are also vastly against me. I represent a fraction of my employer's labour, but my employer represents 100% of my income. That dynamic is totally inverted with AI. You are a rounding error on their revenue sheet, they have a monopoly on your work throughput. How do you budget an workforce that could turn 20% more expensive overnight?
- bornfreddy 6mo ago
  By continuously testing competitors and local LLMs? The reason for rising prices is that they (Anthropic) probably realized that they have reached a ceiling of what LLMs are capable of, and while it's a lot, it is still not a big moat and it's definitely not intelligence.
  - noosphr 6mo ago
    Anything but the simplest tooling is not transferable between model generations, let alone completely different families.
    - skissane 6mo ago
      > Anything but the simplest tooling is not transferable between model generations, let alone completely different families. It is transferable-yes, you will get issues if you take prompts and workflows tuned for one model and send them to another unchanged. But, most of the time, fixing it is just tinkering with some prompt templates People port solutions between models all the time. It takes some work, but the amount of work involved is tractable Plus: this is absolutely the kind of task a coding agent can accelerate The biggest risk is if your solution is at the frontier of capability, and a competing model (even another frontier model) just can’t do it. But a lot of use cases, that isn’t the case. And even if that is the case today, decent odds in a few more months it won’t be
      - 6mo ago
thrwaway55 6mo ago
Why do you think it can't sexually harass someone or drive people to suicide. There are already lawsuits coming in on it causing suicides. This is an architecture that people are increasing begging to give network connectivity that can't differentiate its system prompt from user input
- siffin 6mo ago
  I thought it already was used to sexually harass people by creating naked versions of them.
- inquirerGeneral 6mo ago
  [dead]
- fragmede 6mo ago
  As the NRA puts it, guns don't kill people, people kill people. Yes, a harasser now has access to more powerful tools to sexual harass somebody, but Claude, the AI, isn't going to randomly grab the secretary's ass and have to be fired for having done so at the holiday Christmas party.

dang 6mo ago

Let's give that one a SCP* re-up: https://news.ycombinator.com/item?id=47778922 https://news.ycombinator.com/item?id=47778922 (* explained at https://news.ycombinator.com/item?id=26998308 https://news.ycombinator.com/item?id=26998308)