4 ms·

> All of these features are about breaking the coupling between a human sitting at a terminal or chat window and interacting turn-by-turn with the agent. This

by aledevv 6mo ago

> All of these features are about breaking the coupling between a human sitting at a terminal or chat window and interacting turn-by-turn with the agent. This means: - less and less "man-in-the-loop" - less and less interaction between LLMs and humans - more and more automation - more and more decision-making autonomy for agents - more and more risk (i.e., LLMs' responsibility) - less and less human responsibility Problem: Tasks that require continuous iteration and shared decision-making with humans have two possible options: - either they stall until human input - or they decide autonomously at our risk Unfortunately, automation comes at a cost: RISK.

dist-epoch 6mo ago
AI driven cars have better risk profiles than humans. Why do you think the same will not also be true for AI steerers/managers/CEO? In a year of two, having a human in the loop, will all of their biases and inconsistencies will be considered risky and irresponsible.
- jddj 6mo ago
  Getting to that point is likely going to involve a lot of (the business and personal equivalent of) Teslas electing to drive through white semitrailers.
- deleted 6mo ago
  [deleted]
- khafra 6mo ago
  "Did the vehicle just crash" has a short feedback loop, very amenable to RL. "Did this product strategy tank our earnings/reputation/compliance/etc" can have a much longer, harder to RL feedback loop. But maybe not that much longer; METR task length improvement is still straight lines on log graphs.
  - dist-epoch 6mo ago
    The AI has read all the business books, blogs and stories. Unless your CEO is Steve Jobs, it's hard to imagine it being much worse than your average pointy haired boss.
    - rapind 6mo ago
      > The AI has read all the business books, blogs and stories. This seems like a liability as most business books, blogs, and stories are either marketing BS or gloss over luck and timing. > Unless your CEO is Steve Jobs, it's hard to imagine it being much worse than your average pointy haired boss. As someone using AI agents daily, this is actually incredible really easy to imagine. It's actually hard to imagine it NOT being horrible! Maybe that'll change though... if gains don't plateau.
    - nprateem 6mo ago
      But they are shit. Over the last 2 days I've got bored of the predictable cycle of it first getting excited about a new idea then back peddling once I shoot it to pieces. They can't write and think critically at the same time. Then subsequent messages are tainted by their earlier nonsensical statements. Opus 3.7 BTW, not some toy open source model.
- oblio 6mo ago
  > AI driven cars have better risk profiles than humans. From which company? I hope you say "Waymo", because Tesla is lying through its teeth and hiding crash statistics from regulators.
  - slfnflctd 6mo ago
    Let's not forget that Waymo requires an extensive, custom mapping and software/pre-training development process for every new city they operate in, are only in 10 cities total after over 20 years, and are still nowhere near profitability (or even with a clear plan to get there as far as I can tell). I personally believe widely available self-driving cars which don't operate at a loss will continue to elude us until we accept the tradeoffs of dedicated lanes, a standardized vehicle-to-vehicle communication protocol, and roadside sensors. We were lied to.
    - oblio 6mo ago
      For a fraction of the cost of developing self-driving cars we could have self-driving trains/trams/subways and most likely minibuses as part of public transportation networks. And self-driving minibuses would basically provide 95% of the benefits of self-driving buses. They could offer 24/7 frequent service with huge coverage, we already have dedicated bus lanes in many places (and we could scale dedicated bus lanes much faster than dedicated self-driving car lanes), etc. Now, I understand that in many places (especially the US) this is infeasible because public anything = communism.
      - ChrisLTD 6mo ago
        Folks in the US are happy to spend tax dollars on roads, it's just that mass transit spending is considered communism. To be fair to the anti-train crowd, we've been led so far down this disastrous path of car-led sprawl that the hope of even building feasible buses that can reach into the byzantine suburbs is unlikely. So, maybe our best hope is self-driving EVs? At least in our lifetimes.
- philipwhiuk 6mo ago
  Or autonomous weapons?
canarias_mate 6mo ago
[flagged]

by aledevv 6mo ago

dist-epoch 6mo ago

AI driven cars have better risk profiles than humans. Why do you think the same will not also be true for AI steerers/managers/CEO? In a year of two, having a human in the loop, will all of their biases and inconsistencies will be considered risky and irresponsible.

jddj 6mo ago
Getting to that point is likely going to involve a lot of (the business and personal equivalent of) Teslas electing to drive through white semitrailers.
deleted 6mo ago
[deleted]
khafra 6mo ago
"Did the vehicle just crash" has a short feedback loop, very amenable to RL. "Did this product strategy tank our earnings/reputation/compliance/etc" can have a much longer, harder to RL feedback loop. But maybe not that much longer; METR task length improvement is still straight lines on log graphs.
- dist-epoch 6mo ago
  The AI has read all the business books, blogs and stories. Unless your CEO is Steve Jobs, it's hard to imagine it being much worse than your average pointy haired boss.
  - rapind 6mo ago
    > The AI has read all the business books, blogs and stories. This seems like a liability as most business books, blogs, and stories are either marketing BS or gloss over luck and timing. > Unless your CEO is Steve Jobs, it's hard to imagine it being much worse than your average pointy haired boss. As someone using AI agents daily, this is actually incredible really easy to imagine. It's actually hard to imagine it NOT being horrible! Maybe that'll change though... if gains don't plateau.
  - nprateem 6mo ago
    But they are shit. Over the last 2 days I've got bored of the predictable cycle of it first getting excited about a new idea then back peddling once I shoot it to pieces. They can't write and think critically at the same time. Then subsequent messages are tainted by their earlier nonsensical statements. Opus 3.7 BTW, not some toy open source model.
oblio 6mo ago
> AI driven cars have better risk profiles than humans. From which company? I hope you say "Waymo", because Tesla is lying through its teeth and hiding crash statistics from regulators.
- slfnflctd 6mo ago
  Let's not forget that Waymo requires an extensive, custom mapping and software/pre-training development process for every new city they operate in, are only in 10 cities total after over 20 years, and are still nowhere near profitability (or even with a clear plan to get there as far as I can tell). I personally believe widely available self-driving cars which don't operate at a loss will continue to elude us until we accept the tradeoffs of dedicated lanes, a standardized vehicle-to-vehicle communication protocol, and roadside sensors. We were lied to.
  - oblio 6mo ago
    For a fraction of the cost of developing self-driving cars we could have self-driving trains/trams/subways and most likely minibuses as part of public transportation networks. And self-driving minibuses would basically provide 95% of the benefits of self-driving buses. They could offer 24/7 frequent service with huge coverage, we already have dedicated bus lanes in many places (and we could scale dedicated bus lanes much faster than dedicated self-driving car lanes), etc. Now, I understand that in many places (especially the US) this is infeasible because public anything = communism.
    - ChrisLTD 6mo ago
      Folks in the US are happy to spend tax dollars on roads, it's just that mass transit spending is considered communism. To be fair to the anti-train crowd, we've been led so far down this disastrous path of car-led sprawl that the hope of even building feasible buses that can reach into the byzantine suburbs is unlikely. So, maybe our best hope is self-driving EVs? At least in our lifetimes.
philipwhiuk 6mo ago
Or autonomous weapons?

canarias_mate 6mo ago

[flagged]