3 ms·
The fact that tokens aren't free will limit the number of unserious requests. Email is free, and so there's a lot of spam emails sitting in my inbox, but tokens
by b473a 2mo ago
The fact that tokens aren't free will limit the number of unserious requests. Email is free, and so there's a lot of spam emails sitting in my inbox, but tokens aren't.
On the other hand, op said he had wanted to do this for years but never had the time / ability. We can probably assume any increase in unserious requests will come with an increase in serious requests from people paying for the tokens to get the quotes.
- Gormo 2mo ago> The fact that tokens aren't free will limit the number of unserious requests. Email is free, and so there's a lot of spam emails sitting in my inbox, but tokens aren't. Back when email was new, using it required paying an hourly fee to a proprietary online service. Then, with economies of scale and protocol standardization, it got to the point where the resources needed for email were so minimal that unlimited usage could be baked into flat-rate service offerings, and anyone who cared to could run their own SMTP/POP/IMAP servers on commodity hardware. Using cloud-hosted LLMs is currently still in the "$5/hour CompuServe account" territory, but imagine what things might look like in five years.
- no-name-here 2mo ago> The fact that tokens aren't free will limit the number of unserious requests. LLM costs are already pretty cheap (regardless of whether someone believes they'll continue falling). For a recent example, see a relatively powerful LLM like DeepSeek Flash, where you can get a million tokens for $0.20. And if someone is OK with their task being batched (instead of executing it right now), that can lower prices too. Inference price trends in previous years: https://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends