3 ms·

Imagine a coffee shop refusing to have a price list until after the coffee's been made.

kevindamm 2y ago
In many countries a taxi won't tell you how much the ride will cost. The first time I traveled to somewhere that negotiated the cost up front it blew my mind. Frequently, contracts will have room for additional charges if circumstances change even a little, or products will have a market rate (fish, equity, etc.). It might seem absurd but variable cost things are not uncommon.
- Breza 2y ago
  This is what Washington DC did when I moved here. They theoretically had zones, but in reality it was arbitrary. Moving to meters was an amazing development. Similarly, as LLMs become more and more commonplace, the pricing models will need to be more predictable. My LLM expenses are only around $100/month, but it's a bigger impediment to pushing projects to production when I can't tell the boss exactly how it'll be priced.
- MattGaiser 2y ago
  In Oman, every tax fare was basically just of the bills. So 5 rials, 10 rials, or 20 rials. So many different ways people price things around the world.
- _flux 2y ago
  In this case there's nothing that's variable, though, and the competition is able to pull it off precisely. Indeed, they themselves were able to do it before!
  - kevindamm 2y ago
    Perhaps they have a new tokenization method that's non-deterministic? If there are parallel lookaheads, not necessarily an rng, race conditions could make for variable cost. Or an expansion of certain terms into multiple token outputs, but the selection of which terms are expanded is based on a dynamic pool of Named Entities being recognized. Or maybe they just want to hide their process.. there is some secret sauce there even if it's deterministic, and so much depends on getting a good initial embedding, I've seen a tokenizer make or break an otherwise great model. I am merely hypothesizing, it may not be nondeterministic but I'm not going to assume it's not.
- scoot 2y ago
  > In many countries a taxi won't tell you how much the ride will cost. I've only ever seen: fixed price based on destination (typically for fares originating from an airport), negotiated, or metered. A better analog analogy would be metered pricing, but where the cost per mile is a secret.
  - sitkack 2y ago
    Where the cost per mile is published, but their mile ain’t yer mile.
- szundi 2y ago
  These are called the variable variable costs now
itake 2y ago
reminds me of the coffee shop incident in seattle last week with the hammer
amanda99 2y ago
I mean you already pay per output token and (sure, you can limit it), but it's unpredictable given the prompt?

In many countries a taxi won't tell you how much the ride will cost. The first time I traveled to somewhere that negotiated the cost up front it blew my mind. Frequently, contracts will have room for additional charges if circumstances change even a little, or products will have a market rate (fish, equity, etc.). It might seem absurd but variable cost things are not uncommon.

Breza 2y ago
This is what Washington DC did when I moved here. They theoretically had zones, but in reality it was arbitrary. Moving to meters was an amazing development. Similarly, as LLMs become more and more commonplace, the pricing models will need to be more predictable. My LLM expenses are only around $100/month, but it's a bigger impediment to pushing projects to production when I can't tell the boss exactly how it'll be priced.
MattGaiser 2y ago
In Oman, every tax fare was basically just of the bills. So 5 rials, 10 rials, or 20 rials. So many different ways people price things around the world.
_flux 2y ago
In this case there's nothing that's variable, though, and the competition is able to pull it off precisely. Indeed, they themselves were able to do it before!
- kevindamm 2y ago
  Perhaps they have a new tokenization method that's non-deterministic? If there are parallel lookaheads, not necessarily an rng, race conditions could make for variable cost. Or an expansion of certain terms into multiple token outputs, but the selection of which terms are expanded is based on a dynamic pool of Named Entities being recognized. Or maybe they just want to hide their process.. there is some secret sauce there even if it's deterministic, and so much depends on getting a good initial embedding, I've seen a tokenizer make or break an otherwise great model. I am merely hypothesizing, it may not be nondeterministic but I'm not going to assume it's not.
scoot 2y ago
> In many countries a taxi won't tell you how much the ride will cost. I've only ever seen: fixed price based on destination (typically for fares originating from an airport), negotiated, or metered. A better analog analogy would be metered pricing, but where the cost per mile is a secret.
- sitkack 2y ago
  Where the cost per mile is published, but their mile ain’t yer mile.
szundi 2y ago
These are called the variable variable costs now

itake 2y ago

reminds me of the coffee shop incident in seattle last week with the hammer

amanda99 2y ago

I mean you already pay per output token and (sure, you can limit it), but it's unpredictable given the prompt?