4 ms·
This is the problem with cloud models, you build a "predictable" workflow then they remove it with a new and improved one that is less deterministic and often c
by segmondy 3mo ago
This is the problem with cloud models, you build a "predictable" workflow then they remove it with a new and improved one that is less deterministic and often costs more. If you use a local model discontinuation is no longer a thing to worry about.
- tiew9Vii 3mo agoThis is a huge (obvious) risk I've seen very little talk/concern on. It's not even the new one being "less deterministic and often costs more". Application prompts overfit the model they are using to get the output they want. Switch model, the prompt no longer produces the output you expect and you may not be able to get the output you need anymore. If a model is critical for your application/business you want the ability to run it yourself otherwise you are stuck when it becomes unavailable.
- sidpatil 3mo agoFurther, they may change the behavior of the existing model without providing clear notice. E.g. existing models start to behave worse shortly after new model releases, due to resource reallocation away from the older models toward the newly-released ones.
- jazzyjackson 3mo agoYou can run software you control in the cloud, so the cloud/local distinction isn’t really the problem/solution. Relying on proprietary models that can be discontinued is a problem, using an open weights model that can’t be taken away is a solution, whether or not those weights fit on a MacBook.
- Ferret7446 3mo agoThat's the point, if you're using the weaker models, you should shift to a local model (it doesn't make sense for Google to keep providing it). SaaS is best for the SOTA models.