3 ms·

Or if you want to hedge against the various tail risks of third-party providers raising prices or denying you service or somehow abusing your data...

Aurornis 3mo ago
> hedge against the various tail risks of third-party providers raising prices They could 10X the prices and you’d still be better off. It’s also unlikely that prices go up enough to warrant a $100K local investment to prevent paying a couple bucks per million tokens. > or denying you service I guess you’re not familiar with OpenRouter? There are many providers there. There are providers outside of OpenRouter. There will always be someone to take your business. > or somehow abusing your data... If data security is your concern then you’re better renting a server as needed still. If you cannot tolerate any data leaving, then local models are the only way. You pay a high premium for it!
- gizajob 3mo ago
  People seem to miss that with local models you can have them burning their wee digital brains out 24/7, which is a different class of AI usage than that from online models even at a few dollars per million tokens.
  - CamperBob2 3mo ago
    There's a definite psychological branch point. With a remote provider, no matter how readily you can afford it, your mindset is always going to be, "I should think twice about what I'm doing. I hate to waste tokens." With your own hardware, your mindset is more like, "I should try to get more done. I hate to see this thing just sitting there idle."
incrudible 3mo ago
Raising prices is not a tail risk, anything a local LLM setup can do for you can be done by any cloud provider, with the same capex as yours (or less), there is no moat here, so it is highy price competitive and will remain so. If you want to speculate on hardware shortages, that is a different business altogether and you need no janky garage setup to profit.

Aurornis 3mo ago

> hedge against the various tail risks of third-party providers raising prices They could 10X the prices and you’d still be better off. It’s also unlikely that prices go up enough to warrant a $100K local investment to prevent paying a couple bucks per million tokens. > or denying you service I guess you’re not familiar with OpenRouter? There are many providers there. There are providers outside of OpenRouter. There will always be someone to take your business. > or somehow abusing your data... If data security is your concern then you’re better renting a server as needed still. If you cannot tolerate any data leaving, then local models are the only way. You pay a high premium for it!

gizajob 3mo ago
People seem to miss that with local models you can have them burning their wee digital brains out 24/7, which is a different class of AI usage than that from online models even at a few dollars per million tokens.
- CamperBob2 3mo ago
  There's a definite psychological branch point. With a remote provider, no matter how readily you can afford it, your mindset is always going to be, "I should think twice about what I'm doing. I hate to waste tokens." With your own hardware, your mindset is more like, "I should try to get more done. I hate to see this thing just sitting there idle."

incrudible 3mo ago

Raising prices is not a tail risk, anything a local LLM setup can do for you can be done by any cloud provider, with the same capex as yours (or less), there is no moat here, so it is highy price competitive and will remain so. If you want to speculate on hardware shortages, that is a different business altogether and you need no janky garage setup to profit.