4 ms·

Even the aspirational use cases you're talking about basically are just "digital secretary." There's a massive problem with that even if the models end up being

Even the aspirational use cases you're talking about basically are just "digital secretary." There's a massive problem with that even if the models end up being capable in the future. The value of a secretary is that you know them, they know you, and you trust them to do things right. There are stakes if they don't. No company can provide that as a service at scale for everyone without it being a disaster. Not because it's not technically possible, but because of the incentives. That much power over the details of so many people's lives is irresistible; there will be persistent temptation to use it. The presence of that possibility makes the secretary impossible to trust.

cjonas 4mo ago
These use cases will just be built as "open source" (openclawd) or even custom one off application in the future. I've been building apps to run the tedious parts of my life recently. Meal planning, personal finance, bills, tax organization... Why would I pay for services that will be enshiftified when I can build a app that does exactly what I want in an afternoon. Yes the code is shit and it wouldn't scale... But it doesn't need to
- losteric 4mo ago
  > Why would I pay for services that will be enshiftified when I can build an app that does exactly what I want in an afternoon. When we talk about “the market”, the customer base, remember it’s a market that typically doesn’t know how to or care to even install an adblocker.
  - deleted 4mo ago
    [deleted]
  - cjonas 4mo ago
    I don't see any mention of "the market" anywhere in this thread. I'm just talking about the ability for a motivated user to solve real problems with these tools. Right now these solutions are available to software developers but over time it will become approachable to more users
    - deleted 4mo ago
      [deleted]
- whywhywhywhy 4mo ago
  > Why would I pay for services that will be enshiftified when I can build a app that does exactly what I want in an afternoon Because the problem now took a whole afternoon to solved and sapped your creative energy instead.
zmmmmm 4mo ago
it's an interesting question if any of the AI companies would be willing to step up and absorb the risk ie: to give the AI agent a "stake". eg: if my booking is wrong, they will cover the cost and compensate me. It would sort of just come down to buying premium travel insurance for everyone that uses it. And insurance for anything else they do. It has to be one of two things - they either believe the risks are worth it (so then there should be a financial model that can absorb the cost of insurance to do it), or in fact, the risks are too great. At some point, if they keep offering the tech on a "use at your own risk" basis, they are implicitly communicating that they themselves think the risks are too great - so YOU shouldn't trust it either.
- teiferer 4mo ago
  > eg: if my booking is wrong, they will cover the cost and compensate me That would be nice, but it's the wrong angle. The reason people like real secretaries is not because somebody is compensated when things go wrong. It's because things don't go wrong. I don't use this thing if I need to fear things go wrong, even if I'd be compensated. Maybe it would provide the right incentives for the companies though.
  - waterhouse 4mo ago
    Surely, if the compensation was high enough, you'd be like, "Sure, I'm happy with that outcome." And then, if the AI company thinks they have a low enough failure rate that the expected cost of paying out the compensation still lets them make a profit, then they could make that promise to all customers. Though a compensation that high sounds like it would invite fraud, where the customer would be glad to have something go "wrong" and get a fat check. Not sure if that's a solvable problem.
    - ml-anon 4mo ago
      they are literally burning billions of dollars and can presumably keep doing this for a while. Taking a trivial amount of additional financial liability to launder their reputation wouldn't meaningfully improve things.
9dev 4mo ago
The vast majority of people never even asked for a personal assistant, because that isn’t something normal people have or do need in the first place. They aren’t so occupied/privileged/posh to need someone to do the trivial tasks of daily life for them. This whole venue of technology is an exercise in ivory tower construction completely disconnected from ordinary people.
- Snoddas 4mo ago
  I think you are wrong. They never asked for one becase they never imagined being able to afford one. The amount of administration organizing a normal household takes I suspect most would be glad to leave to someon/something they trust and that can be held accountable. Today that someone needs to be a person (imo). But who knows, a startup may be plotting accountable digital assistans as we speak.
  - tempfile 4mo ago
    Obligatory: https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/ https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
    - pnut 4mo ago
      Pretty sure unaccountability is a desired feature of management decisions in most organisations. That quote has an unexpressed precondition to the effect of "In order for an organisation to be objectively well run..." or "In order for an organisation to equitably benefit all stakeholders at all levels..." etc
  - djaro 4mo ago
    I think theres some real use cases in the household department. I would love an AI that I just tell my nutrition goals (I want to eat X calories, Y protein, Z fiber, hit all vitamins) and it just generates a full meal plan for me each day. Like, I go to the store and it made the complete shopping list for me. And automatically updates the rest of the day if I tell it I skipped a meal or ate a snack.

cjonas 4mo ago

These use cases will just be built as "open source" (openclawd) or even custom one off application in the future. I've been building apps to run the tedious parts of my life recently. Meal planning, personal finance, bills, tax organization... Why would I pay for services that will be enshiftified when I can build a app that does exactly what I want in an afternoon. Yes the code is shit and it wouldn't scale... But it doesn't need to

losteric 4mo ago
> Why would I pay for services that will be enshiftified when I can build an app that does exactly what I want in an afternoon. When we talk about “the market”, the customer base, remember it’s a market that typically doesn’t know how to or care to even install an adblocker.
- deleted 4mo ago
  [deleted]
- cjonas 4mo ago
  I don't see any mention of "the market" anywhere in this thread. I'm just talking about the ability for a motivated user to solve real problems with these tools. Right now these solutions are available to software developers but over time it will become approachable to more users
  - deleted 4mo ago
    [deleted]
whywhywhywhy 4mo ago
> Why would I pay for services that will be enshiftified when I can build a app that does exactly what I want in an afternoon Because the problem now took a whole afternoon to solved and sapped your creative energy instead.

zmmmmm 4mo ago

it's an interesting question if any of the AI companies would be willing to step up and absorb the risk ie: to give the AI agent a "stake". eg: if my booking is wrong, they will cover the cost and compensate me. It would sort of just come down to buying premium travel insurance for everyone that uses it. And insurance for anything else they do. It has to be one of two things - they either believe the risks are worth it (so then there should be a financial model that can absorb the cost of insurance to do it), or in fact, the risks are too great. At some point, if they keep offering the tech on a "use at your own risk" basis, they are implicitly communicating that they themselves think the risks are too great - so YOU shouldn't trust it either.

teiferer 4mo ago
> eg: if my booking is wrong, they will cover the cost and compensate me That would be nice, but it's the wrong angle. The reason people like real secretaries is not because somebody is compensated when things go wrong. It's because things don't go wrong. I don't use this thing if I need to fear things go wrong, even if I'd be compensated. Maybe it would provide the right incentives for the companies though.
- waterhouse 4mo ago
  Surely, if the compensation was high enough, you'd be like, "Sure, I'm happy with that outcome." And then, if the AI company thinks they have a low enough failure rate that the expected cost of paying out the compensation still lets them make a profit, then they could make that promise to all customers. Though a compensation that high sounds like it would invite fraud, where the customer would be glad to have something go "wrong" and get a fat check. Not sure if that's a solvable problem.
  - ml-anon 4mo ago
    they are literally burning billions of dollars and can presumably keep doing this for a while. Taking a trivial amount of additional financial liability to launder their reputation wouldn't meaningfully improve things.

9dev 4mo ago

The vast majority of people never even asked for a personal assistant, because that isn’t something normal people have or do need in the first place. They aren’t so occupied/privileged/posh to need someone to do the trivial tasks of daily life for them. This whole venue of technology is an exercise in ivory tower construction completely disconnected from ordinary people.

Snoddas 4mo ago
I think you are wrong. They never asked for one becase they never imagined being able to afford one. The amount of administration organizing a normal household takes I suspect most would be glad to leave to someon/something they trust and that can be held accountable. Today that someone needs to be a person (imo). But who knows, a startup may be plotting accountable digital assistans as we speak.
- tempfile 4mo ago
  Obligatory: https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/ https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
  - pnut 4mo ago
    Pretty sure unaccountability is a desired feature of management decisions in most organisations. That quote has an unexpressed precondition to the effect of "In order for an organisation to be objectively well run..." or "In order for an organisation to equitably benefit all stakeholders at all levels..." etc
- djaro 4mo ago
  I think theres some real use cases in the household department. I would love an AI that I just tell my nutrition goals (I want to eat X calories, Y protein, Z fiber, hit all vitamins) and it just generates a full meal plan for me each day. Like, I go to the store and it made the complete shopping list for me. And automatically updates the rest of the day if I tell it I skipped a meal or ate a snack.