18 ms·
Even if you had a 64GB machine: Are you willing to reserve 90% of your memory to run a LLM? With dirt cheap models like deepseek-v4-flash that will run "forever
by sparkling 2mo ago
Even if you had a 64GB machine: Are you willing to reserve 90% of your memory to run a LLM? With dirt cheap models like deepseek-v4-flash that will run "forever" on $10, the answer for me is clearly: no.
- zoobab 2mo ago"With dirt cheap models like deepseek-v4-flash that will run "forever" on $10, the answer for me is clearly: no." When it's free, you are the product.
- IMTDb 2mo agoDeepseek flash is open weight, this means we can download and run that model without any connection to deepseek, no data/tokens/usage data ever reaches them. They cannot make us their product.
- prplxd_nihilist 2mo agoI see many people saying deepseek and other chinese providers have always been profitable. Also they show their training costs publicly. Can't say for sure since I have not used it personally, but I think they'll for sure outlive the western SOTAs.
- LogicFailsMe 2mo agoOpenAI apparently runs a profitable inference business with 40% gross margin, but their advertising budget is nutso and their real costs are pretraining and research. I suspect Deepseek's comp is not predicated on capturing the lightcone of all future value, some googling insinuates their top pay is $212K US which would support that suspicion. Compare and contrast with the $1.35M and up at OpenAI.
- amrit3128 2mo agoAh yes, I'm sure Trovalds and Stallman are harvesting my data through free software, aren't they? This argument is used by boomers who were fed cold war era propoganda that surely everybody is selfish, and you're always at fault.
- kipchak 2mo agoThink they're talking about things that are free as in beer but not free as in freedom, not FOSS
- Manfrednotfunny 2mo agoI'm waiting for the speed/quality per dollar metric to go down a little bit further and then I will def run it at home. Its not just that you send a sentence to an API endpoint, you always send EVERYTHING to that agent as a context. You want to analyse your spending history? You now send everything to someone. Either no one cares but understands this implication on how easy it is to really capture you or no one really things about it. But i'm a lot more diligent on what I send. I disabled the gemini activity feature for example because google started telling me that my stuff could be reviwed by humans.
- plufz 2mo agoYeah, it does feel a bit silly with my encrypted disks, encrypted backups, unique passwords, advanced router, etc, while I send everything I do in plain text to anthropic.
- zbendefy 2mo agoSimiliarly I wonder why we dont run our own email server despite the sensitive data there.
- Manfrednotfunny 2mo agoI did, it started to become too much work to run it well due to all the spam :| (even with the right signatures and configs, until you learn what a blacklisted ip is and that ips need some time of 'positive history' and what not.....) But at least with your email, you had to trust only one company, as shitty as it is. Separation of concerns was also easy. Now with OpenRouter, you just might by accident, send your whole context to just everyone because OpenRouter just routes to different models and you might just switch around between some free model, the good one etc. And it is always the whole context.
- deleted 2mo ago[deleted]
- halJordan 2mo agoIt's the size of a big vm. There's nothing wrong with reserving that much working space for one item.