4 ms·
What could be the benefit of paying $20 to Ollama to run inferior models instead of paying the same amount of money to e.g. OpenAI for access to sota models?
by jacekm 1y ago
What could be the benefit of paying $20 to Ollama to run inferior models instead of paying the same amount of money to e.g. OpenAI for access to sota models?
- vanillax 1y agonothing lmao. this is just ollama trying to make money.
- ibejoeb 1y agoI run a lot of mundane jobs that work fine with less capable models, so I can see the potential benefit. It all depends on the limits though.
- AndroTux 1y agoPrivacy, I guess. But at this point it’s just believing that they won’t log your data.
- daft_pink 1y agoI feel the primary benefit of this Ollama Turbo is that you can quickly test and run different models in the cloud that you could run locally if you had the correct hardware. This allows you to try out some open models and better assess if you could buy a dgx box or Mac Studio with a lot of unified memory and build out what you want to do locally without actually investing in very expensive hardware. Certain applications require good privacy control and on-prem and local are something certain financial/medical/law developers want. This allows you to build something and test it on non-private data and then drop in real local hardware later in the process.
- dawnerd 1y agoQuickly test… the two models they support? This is just another subscription to quantized models.
- daft_pink 1y agoit looks like the plan is to support way more models though. gotta start somewhere.
- fluidcruft 1y agoMe at home: $20/mo while I wait for a card that can run this or dgx box? Decisions, decisions.
- jerieljan 1y ago> quickly test and run different models in the cloud that you could run locally if you had the correct hardware. I feel like they're competing against Hugging Face or even Colaboratory then if this is the case. And for cases that require strict privacy control, I don't think I'd run it on emergent models or if I really have to, I would prefer doing so on an existing cloud setup already that has the necessary trust / compliance barriers addressed. (does Ollama Turbo even have their Trust center up?) I can see its potential once it gets rolling, since there's a lot of ollama installations out there.
- deleted 1y ago[deleted]
- rapind 1y agoI'm not sure the major models will remain at $20. Regardless, I support any and all efforts to keep the space crowded and competitive.
- michelsedgh 1y agoI think its the data privacy is the main point and probably more usage before you hit limits? But mainly data privacy i guess
- _--__--__ 1y agoGroq seems to do okay with a similar service but I think their pricing is probably better.
- Geezus_42 1y agoYeah, the NAZI sex not will be great for business!
- gabagool 1y agoYou are thinking of Elon Grok, not Groq
- janalsncm 1y agoWhen Grok originally came out I thought it was unlucky on Groq’s part. Now that Grok has certain connotations, it’s even more true.
- owebmaster 1y ago"There's no such thing as bad publicity." PT Barnum
- fredoliveira 1y agoGroq (the inference service) != Grok (xAI's model)
- woadwarrior01 1y agoGroq's moat is speed, using their custom hardware.
- adrr 1y agoRunning models without a filter on it. OpenAI has an overzealous filter and won’t even tell you what you violated. So you have to do a dance with prompts to see if it’s copyright, trademark or whatever. Recently it just refused to answer my questions and said it wasn’t true that a civil servant would get fired for releasing a report per their job duties. Another dance sending it links to stories that it was true so it could answer my question. I want a LLMs without training wheels.