5 ms·

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

nycdatasci 2y ago
This is great for experimentation, but as others have pointed out recently there are persistent issues with Gemini that prevent use in actual products. The recitation/self-sensoring issue results in random failures: https://github.com/google/generative-ai-docs/issues/257 https://github.com/google/generative-ai-docs/issues/257
- faangguyindia 2y ago
  I had this problem too but 002 solves this I think (not tested exhaustively), but I've not run into any problems since 002 and vertex + block all on all safety is now working fine, earlier I had problems with "block all" in safety settings and api throwing errors. I am using it in https://github.com/zerocorebeta/Option-K https://github.com/zerocorebeta/Option-K (currently it doesn't have lowest safety settings because api wouldn't allow it, but now I am going to push new update with safety disabled) Why? I've another application which is working since yesterday after 002 launch, I've safety settings to none and it will not answer certain questions but since yesterday it answers everything.
- o11c 2y ago
  And yet - if Gemini actually bothers to tell you when it detects verbatim copying of copyrighted content, how often must that occur on other AIs without notice?
a2128 2y ago
Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same amount of electricity
- 4ndrewl 2y ago
  It's Google. You know the answer ;)
  - rl3 2y ago
    I mean, there’s no need to dry up subsidies when the underlying product can just be deprecated without warning.
- stavros 2y ago
  Are you asking whether giving away $5/day/user (what OpenAI charges) in compute is sustainable?
- faangguyindia 2y ago
  Google has deep pockets and SOTA hardware for training and interference
- phillipcarter 2y ago
  I think there's a few things to consider: They make a ton of money on large enterprise package deals through Google Cloud. That includes API access but also support and professional services. Most orgs that pay for this stuff don't really need it, but they buy it anyways, as is consistent with most enterprise sales. That can give Google a significant margin to make up the cost elsewhere. Gemini Flash is probably super cheap to run compared to other models. The cost of inference for many tasks has gone down tremendously over the past 1.5 years, and it's still going down. Every economic incentive aligns with running these models more efficiently.
- rcpt 2y ago
  Aren't API calls essentially swappable now between vendors now? If you wanted to switch from Gemini to Chatgpt you could copy/paste your code into Chatgpt and ask it to switch to their API. Disclaimer I work at Google but not on Gemini
  - zitterbewegung 2y ago
hobofan 2y ago
Not locked to the US, you get 1 billion tokens per month per model with Mistral since their recent announcement: https://mistral.ai/news/september-24-release/ https://mistral.ai/news/september-24-release/ (1 request per second is quite a harsh rate limit, but hey, free is free) I'm pretty excited what all the services adopting free tiers is going to do to the landscape, as that should allow for a lot more experimentation and a lot more hobby projects transitioning into full-time projects, that previously felt a lot more risky/unpredictable with pricing.
Deathmax 2y ago
The free tier API isn't US-only, Google has removed the free tier restriction for UK/EEA countries for a while now, with the added bonus of not training on your data if making a request from the UK/CH/EEA.
deleted 2y ago
[deleted]
airspresso 2y ago
Free of cost != free open model. Free of cost means all your requests are logged for Google to use as training data and whatnot. Llama3.2 on the other hand runs locally, no data is ever sent to a 3rd party, so I can freely use it to summarize all my notes regardless of one of them being from my most recent therapy session and another being my thoughts on how to solve a delicate problem involving politics at work. I don't need to pre-classify all the input to make sure it's safe to share. Same with images, I can use Llama3.2 11B locally to interpret any photo I've taken without having to worry about getting consent from the people in the photo to share it with a 3rd party, or whether the photo is of my passport for some application I had to file or a receipt of something I bought that I don't want Google to train their next vision model OCR on. TL;DR - Google free of cost models are irrelevant when talking about local models.

nycdatasci 2y ago

This is great for experimentation, but as others have pointed out recently there are persistent issues with Gemini that prevent use in actual products. The recitation/self-sensoring issue results in random failures: https://github.com/google/generative-ai-docs/issues/257 https://github.com/google/generative-ai-docs/issues/257

faangguyindia 2y ago
I had this problem too but 002 solves this I think (not tested exhaustively), but I've not run into any problems since 002 and vertex + block all on all safety is now working fine, earlier I had problems with "block all" in safety settings and api throwing errors. I am using it in https://github.com/zerocorebeta/Option-K https://github.com/zerocorebeta/Option-K (currently it doesn't have lowest safety settings because api wouldn't allow it, but now I am going to push new update with safety disabled) Why? I've another application which is working since yesterday after 002 launch, I've safety settings to none and it will not answer certain questions but since yesterday it answers everything.
o11c 2y ago
And yet - if Gemini actually bothers to tell you when it detects verbatim copying of copyrighted content, how often must that occur on other AIs without notice?

a2128 2y ago

Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same amount of electricity

4ndrewl 2y ago
It's Google. You know the answer ;)
- rl3 2y ago
  I mean, there’s no need to dry up subsidies when the underlying product can just be deprecated without warning.
stavros 2y ago
Are you asking whether giving away $5/day/user (what OpenAI charges) in compute is sustainable?
faangguyindia 2y ago
Google has deep pockets and SOTA hardware for training and interference
phillipcarter 2y ago
I think there's a few things to consider: They make a ton of money on large enterprise package deals through Google Cloud. That includes API access but also support and professional services. Most orgs that pay for this stuff don't really need it, but they buy it anyways, as is consistent with most enterprise sales. That can give Google a significant margin to make up the cost elsewhere. Gemini Flash is probably super cheap to run compared to other models. The cost of inference for many tasks has gone down tremendously over the past 1.5 years, and it's still going down. Every economic incentive aligns with running these models more efficiently.
rcpt 2y ago
Aren't API calls essentially swappable now between vendors now? If you wanted to switch from Gemini to Chatgpt you could copy/paste your code into Chatgpt and ask it to switch to their API. Disclaimer I work at Google but not on Gemini
- zitterbewegung 2y ago

hobofan 2y ago

Not locked to the US, you get 1 billion tokens per month per model with Mistral since their recent announcement: https://mistral.ai/news/september-24-release/ https://mistral.ai/news/september-24-release/ (1 request per second is quite a harsh rate limit, but hey, free is free) I'm pretty excited what all the services adopting free tiers is going to do to the landscape, as that should allow for a lot more experimentation and a lot more hobby projects transitioning into full-time projects, that previously felt a lot more risky/unpredictable with pricing.

Deathmax 2y ago

The free tier API isn't US-only, Google has removed the free tier restriction for UK/EEA countries for a while now, with the added bonus of not training on your data if making a request from the UK/CH/EEA.

deleted 2y ago

[deleted]

airspresso 2y ago

Free of cost != free open model. Free of cost means all your requests are logged for Google to use as training data and whatnot. Llama3.2 on the other hand runs locally, no data is ever sent to a 3rd party, so I can freely use it to summarize all my notes regardless of one of them being from my most recent therapy session and another being my thoughts on how to solve a delicate problem involving politics at work. I don't need to pre-classify all the input to make sure it's safe to share. Same with images, I can use Llama3.2 11B locally to interpret any photo I've taken without having to worry about getting consent from the people in the photo to share it with a 3rd party, or whether the photo is of my passport for some application I had to file or a receipt of something I bought that I don't want Google to train their next vision model OCR on. TL;DR - Google free of cost models are irrelevant when talking about local models.