3 ms·
Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of d
by a2128 2y ago
Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same amount of electricity
- 4ndrewl 2y agoIt's Google. You know the answer ;)
- rl3 2y agoI mean, there’s no need to dry up subsidies when the underlying product can just be deprecated without warning.
- stavros 2y agoAre you asking whether giving away $5/day/user (what OpenAI charges) in compute is sustainable?
- faangguyindia 2y agoGoogle has deep pockets and SOTA hardware for training and interference
- phillipcarter 2y agoI think there's a few things to consider: They make a ton of money on large enterprise package deals through Google Cloud. That includes API access but also support and professional services. Most orgs that pay for this stuff don't really need it, but they buy it anyways, as is consistent with most enterprise sales. That can give Google a significant margin to make up the cost elsewhere. Gemini Flash is probably super cheap to run compared to other models. The cost of inference for many tasks has gone down tremendously over the past 1.5 years, and it's still going down. Every economic incentive aligns with running these models more efficiently.
- rcpt 2y agoAren't API calls essentially swappable now between vendors now? If you wanted to switch from Gemini to Chatgpt you could copy/paste your code into Chatgpt and ask it to switch to their API. Disclaimer I work at Google but not on Gemini
- zitterbewegung 2y agoNot tokens allowed per user. Google has the largest token windows .
- snek_case 2y agoDifferent APIs and models are going to come with different capabilities and restrictions.
- zitterbewegung 2y agoRun test queries on all platforms using something like litellm [1] and langsmith [2] . You may not be able to match large queries but, testing will help you transition to other services. [1] https://github.com/BerriAI/litellm https://github.com/BerriAI/litellm [2] https://langtrace.ai/ https://langtrace.ai/
- QuinnyPig 2y agoFacts. Google did the same thing you describe with Maps a few years ago.
- moffkalast 2y agoIt's not just Google, literally every new service always does this. Prices will always go up once the have enough customers and bean counters start pointing at spreadsheets. Ergo, local is the only option if you don't want to be held for ransom afterwards. As goes for web servers, scraper bots, and whatever, so goes for llms.
- smrtinsert 2y agoIt's "free cloud picture video storage" rush all over again