3 ms·

The value extortion plan writes itself. How long before someone pitches the idea that the models explicitly almost keep solving your problem to get you to keep

by gtowey 9mo ago

The value extortion plan writes itself. How long before someone pitches the idea that the models explicitly almost keep solving your problem to get you to keep spending? Would you even know?

fragmede 9mo ago
The free market proposition is that competition (especially with Chinese labs and grok) means that Anthropic is welcome to do that. They're even welcome to illegally collude with OpenAi such that ChatGPT is similarly gimped. But switching costs are pretty low. If it turns out I can one shot an issue with Qwen or Deepseek or Kimi thinking, Anthropic loses not just my monthly subscription, but everyone else's I show that too. So no, I think that's some grade A conspiracy theory nonsense you've got there.
- thunderfork 9mo ago
  As a rational consumer, how would you distinguish between some intentional "keep pulling the slot machine" failure rate and the intrinsic failure rate? I feel like saying "the market will fix the incentives" handwaves away the lack of information on internals. After all, look at the market response to Google making their search less reliable - sure, an invested nerd might try Kagi, but Google's still the market leader by a long shot. In a market for lemons, good luck finding a lime.
  - krupan 9mo ago
    FWIW, kagi is better than Google
    - direwolf20 9mo ago
      yes, that was their point. Everyone uses Google anyway.
- coffeefirst 9mo ago
  It’s not that crazy. It could even happen by accident in pursuit of another unrelated goal. And if it did, a decent chunk of the tech industry would call it “revealed preference” because usage went up.
  - hnuser123456 9mo ago
    LLMs became sycophantic and effusive because those responses were rated higher during RLHF, until it became newsworthy how obviously eager-to-please they got, so yes, being highly factually correct and "intelligent" was already not the only priority.
- jrflowers 9mo ago
  This is a good point. For example if you have access to a bunch of slot machines, one of them is guaranteed to hit the jackpot. Since switching from one slot machine to another is easy, it is trivial to go from machine to machine until you hit the big bucks. That is why casinos have such large selections of them (for our benefit).
  - krupan 9mo ago
    "for our benefit" lol! This is the best description of how we are all interacting with LLMs now. It's not working? Fire up more "agents" ala gas town or whatever
    - direwolf20 9mo ago
      gas is the transaction fees in Ethereum. It's a fitting name.
  - robotmaxtron 9mo ago
    last time I was at a casino I checked to see what company built the machines, imagine my surprise that it was (by my observation) a single vendor.
- daxfohl 9mo ago
  To be clear I don't think that's what they're doing intentionally. Especially on a subscription basis, they'd rather me maximize my value per token, or just not use them. Lulling users into using tokens unproductively is the worst possible option. The way agents work right now though just sometimes feels that way; they don't have a good way of saying "You're probably going to have to figure this one out yourself".
- deleted 9mo ago
  [deleted]
- bandrami 9mo ago
  > But switching costs are pretty low Switching costs are currently low. Once you're committed to the workflow the providers will switch to prepaying for a year's worth of tokens.
- zelphirkalt 9mo ago
  And we all know the market always gives us the best quality product ...
sailfast 9mo ago
That’s far-fetched. It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise. High value to user + lower compute costs = better pricing power and better margins overall.
- xienze 9mo ago
  > It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise. Unless you’re paying by the token.
- d0mine 9mo ago
  > far-fetched Remember Google? Once it was far-fetched that they would make the search worse just to show you more ads. Now, it is a reality. With tokens, it is even more direct. The more tokens users spend, the more money for providers.
  - throwthrowuknow 9mo ago
    Only if you are paying per token on the API. If you are paying a fixed monthly fee then they lose money when you need to burn more tokens and they lose customers when you can’t solve your problems within that month and max out your session limits and end up with idle time which you use to check if the other providers have caught up or surpassed your current favourite.
    - layla5alive 9mo ago
      Indeed, unlimited plan seems like the only way that makes sense to not have it be guaranteed to be abused by the provider
  - retsibsi 9mo ago
    > Now, it is a reality. What are the details of this? I'm not playing dumb, and of course I've noticed the decline, but I thought it was a combination of losing the battle with SEO shite and leaning further and further into a 'give the user what you think they want, rather than what they actually asked for' philosophy.
password4321 9mo ago
First time I've seen this idea, I have a tingling feeling it might become reality sooner rather than later.
Fnoord 9mo ago
I was thinking more of deliberate backdoor in code. RCE is an obvious example, but another one could be bias. "I'm sorry ma'am, computer says you are ineligable for a bank account." These ideas aren't new. They were there in 90s already when we still thought about privacy and accountability regarding technology, and dystopian novels already described them long, long ago.
chanux 9mo ago
Is this from a page of dating apps playbook?
- direwolf20 9mo ago
  yes

by gtowey 9mo ago

The value extortion plan writes itself. How long before someone pitches the idea that the models explicitly almost keep solving your problem to get you to keep spending? Would you even know?

fragmede 9mo ago

The free market proposition is that competition (especially with Chinese labs and grok) means that Anthropic is welcome to do that. They're even welcome to illegally collude with OpenAi such that ChatGPT is similarly gimped. But switching costs are pretty low. If it turns out I can one shot an issue with Qwen or Deepseek or Kimi thinking, Anthropic loses not just my monthly subscription, but everyone else's I show that too. So no, I think that's some grade A conspiracy theory nonsense you've got there.

thunderfork 9mo ago
As a rational consumer, how would you distinguish between some intentional "keep pulling the slot machine" failure rate and the intrinsic failure rate? I feel like saying "the market will fix the incentives" handwaves away the lack of information on internals. After all, look at the market response to Google making their search less reliable - sure, an invested nerd might try Kagi, but Google's still the market leader by a long shot. In a market for lemons, good luck finding a lime.
- krupan 9mo ago
  FWIW, kagi is better than Google
  - direwolf20 9mo ago
    yes, that was their point. Everyone uses Google anyway.
coffeefirst 9mo ago
It’s not that crazy. It could even happen by accident in pursuit of another unrelated goal. And if it did, a decent chunk of the tech industry would call it “revealed preference” because usage went up.
- hnuser123456 9mo ago
  LLMs became sycophantic and effusive because those responses were rated higher during RLHF, until it became newsworthy how obviously eager-to-please they got, so yes, being highly factually correct and "intelligent" was already not the only priority.
jrflowers 9mo ago
This is a good point. For example if you have access to a bunch of slot machines, one of them is guaranteed to hit the jackpot. Since switching from one slot machine to another is easy, it is trivial to go from machine to machine until you hit the big bucks. That is why casinos have such large selections of them (for our benefit).
- krupan 9mo ago
  "for our benefit" lol! This is the best description of how we are all interacting with LLMs now. It's not working? Fire up more "agents" ala gas town or whatever
  - direwolf20 9mo ago
    gas is the transaction fees in Ethereum. It's a fitting name.
- robotmaxtron 9mo ago
  last time I was at a casino I checked to see what company built the machines, imagine my surprise that it was (by my observation) a single vendor.
daxfohl 9mo ago
To be clear I don't think that's what they're doing intentionally. Especially on a subscription basis, they'd rather me maximize my value per token, or just not use them. Lulling users into using tokens unproductively is the worst possible option. The way agents work right now though just sometimes feels that way; they don't have a good way of saying "You're probably going to have to figure this one out yourself".
deleted 9mo ago
[deleted]
bandrami 9mo ago
> But switching costs are pretty low Switching costs are currently low. Once you're committed to the workflow the providers will switch to prepaying for a year's worth of tokens.
zelphirkalt 9mo ago
And we all know the market always gives us the best quality product ...

sailfast 9mo ago

That’s far-fetched. It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise. High value to user + lower compute costs = better pricing power and better margins overall.

xienze 9mo ago
> It’s in the interest of the model builders to solve your problem as efficiently as possible token-wise. Unless you’re paying by the token.
d0mine 9mo ago
> far-fetched Remember Google? Once it was far-fetched that they would make the search worse just to show you more ads. Now, it is a reality. With tokens, it is even more direct. The more tokens users spend, the more money for providers.
- throwthrowuknow 9mo ago
  Only if you are paying per token on the API. If you are paying a fixed monthly fee then they lose money when you need to burn more tokens and they lose customers when you can’t solve your problems within that month and max out your session limits and end up with idle time which you use to check if the other providers have caught up or surpassed your current favourite.
  - layla5alive 9mo ago
    Indeed, unlimited plan seems like the only way that makes sense to not have it be guaranteed to be abused by the provider
- retsibsi 9mo ago
  > Now, it is a reality. What are the details of this? I'm not playing dumb, and of course I've noticed the decline, but I thought it was a combination of losing the battle with SEO shite and leaning further and further into a 'give the user what you think they want, rather than what they actually asked for' philosophy.

password4321 9mo ago

First time I've seen this idea, I have a tingling feeling it might become reality sooner rather than later.

Fnoord 9mo ago

I was thinking more of deliberate backdoor in code. RCE is an obvious example, but another one could be bias. "I'm sorry ma'am, computer says you are ineligable for a bank account." These ideas aren't new. They were there in 90s already when we still thought about privacy and accountability regarding technology, and dystopian novels already described them long, long ago.

chanux 9mo ago

Is this from a page of dating apps playbook?

direwolf20 9mo ago
yes