12 ms·
Gemini flash models have the least hype, but in my experience in production have the best bang for the buck and multimodal tooling. Google is silently winning
by arnaudsm 1y ago
Gemini flash models have the least hype, but in my experience in production have the best bang for the buck and multimodal tooling.
Google is silently winning the AI race.
- belter 1y ago> Google is silently winning the AI race. That is what we keep hearing here...The last Gemini I cancelled the account, and can't help notice the new one they are offering for free...
- arnaudsm 1y agoSorry I was talking of B2B APIs for my YC startup. Gemini is still far behind for consumers indeed.
- JeremyNT 1y agoI use Gemini almost exclusively as a normal user. What am I missing out on that they are far behind on? It seems shockingly good and I've watched it get much better up to 2.5 Pro.
- arnaudsm 1y agoMostly brand recognition and the earlier Geminis had more refusals. As a consumer, I also really miss the Advanced voice mode of ChatGPT, which is the most transformative tech in my daily life. It's the only frontier model with true audio-to-audio.
- wavewrangler 1y agoWhat do you mean miss? You don’t have the budget to keep something you truly miss for $20? What am in missing here / I don’t mean to criticize I am just curious is all. I would reword but I have to go
- what_ever 1y agoWhat is true audio-to-audio in this case?
- jorvi 1y ago> and the earlier Geminis had more refusals. Its more so that almost every company is running a classifier on their web chat's output. It isn't actually the model refusing, but rather if the classifier hits a threshold, it'll swap the model's out with "Sorry, let's talk about something else." This is most apparent with DeepSeek. If you use their web chat with V3 and then jailbreak it, you'll get uncensored output but it is then swapped with "Let's talk about something else" halfway through the output. And if you ask the model, it has no idea its previous output got swapped and you can even ask it build on its previous answer. But if you use the API, you can push it pretty far with a simple jailbreak. These classifiers are virtually always ran on a separate track, meaning you cannot jailbreak them. If you use an API, you only have to deal with the inherent training data bias, neutering by tuning and neutering by pre-prompt. The last two are, depending on the model, fairly trivial to overcome. I still think the first big AI company that has the guts to say "our LLM is like a pen and brush, what you write or draw with it is on you" and publishes a completely unneutered model will be the one to take a huge slice of marketshare. If I had to bet on anyone doing that, it would be xAI with Grok. And by not neutering it, the model will perform better in SFW tasks too.
- whistle650 1y agoHave you tried the Gemini Live audio-to-audio in the free Gemini iOS app? I find it feels far more natural than ChatGPT Advanced Voice Mode.
- Jensson 1y ago> and the earlier Geminis had more refusals. You can turn off those, Google lets you decide how much it censors you can completely turn it off. It has separate sliders for sexually explicit, hate, dangerous and harassment. It is by far the best at this, since sometimes you want those refusals/filters.
- int_19h 1y agoThey used to be, but not anymore, not since Gemini Pro 2.5. Their "deep research" offering is the best available on the market right now, IMO - better than both ChatGPT and Claude.
- Layvier 1y agoAbsolutely. So many use cases for it, and it's so cheap/fast/reliable
- danielbln 1y agoI want to use these almost too cheap to meter models like Flash more, what are some interesting use cases for those?
- SparkyMcUnicorn 1y agoAnd stellar OCR performance. Flash 2.0 is cheaper and more accurate than AWS Textract, Google Document AI, etc. Not only in benchmarks[0], but in my own production usage. [0] https://getomni.ai/ocr-benchmark https://getomni.ai/ocr-benchmark
- Fairburn 1y agoSorry, but no. Gemini isn't the fastest horse, yet. And it's use within their ecosystem means it isn't geared to the masses outside of their bubble. They are not leading the race but they are a contender.
- spruce_tips 1y agoi have a high volume task i wrote an eval for and was pleasantly surprised at 2.0 flash's cost to value ratio especially compared to gpt4.1-mini/nano accuracy | input price | output price Gemini Flash 2.0 Lite: 67% | $0.075 | $0.30 Gemini Flash 2.0: 93% | $0.10 | $0.40 GPT-4.1-mini: 93% | $0.40 | $1.60 GPT-4.1-nano: 43% | $0.10 | $0.40 excited to to try out 2.5 flash
- jay_kyburz 1y agoCan I ask a serious question. What task are you writing where its ok to get 7% error rate. I can't get my head around how this can be used.
- spruce_tips 1y agolow stakes text classification but it's something that needs to be done and couldnt be done in reasonable time frames or at reasonable price points by humans
- omneity 1y agoIn my case, I have workloads like this where it’s possible to verify the correctness of the result after inference, so any success rate is better than 0 as it’s possible to identify the “good ones”.
- nonethewiser 1y agoAren’t you basically just saying you are able to measure the error rate? I mean that’s good, but already a given in this scenario where hes reporting the 7% error rate.
- jsnell 1y agoNo. If you're able to verify correctness of individual items of work, you can accept the 93% of verified items as-is and send the remaining 7% to some more expensive slow path. That's very different from just knowing the aggregate error rate.
- BoredPositron 1y agoThe API is free, and it's great for everyday tasks. So yes there is no better bang for the buck.
- drusepth 1y agoWait, the API is free? I thought you had to use their web interface for it to be free. How do you use the API for free?
- mlboss 1y agousing aistudio.google.com
- spruce_tips 1y agocreate an api key and dont set up billing. pretty low rate limits and they use your data
- dcre 1y agoYou can get an API key and they don't bill you. Free tier rate limits for some models (even decent ones like Gemini 2.0 Flash) are quite high. https://ai.google.dev/gemini-api/docs/pricing https://ai.google.dev/gemini-api/docs/pricing https://ai.google.dev/gemini-api/docs/rate-limits#free-tier https://ai.google.dev/gemini-api/docs/rate-limits#free-tier
- NoahZuniga 1y agoThe rate limits I've encountered with free api keys has been way lower than the limits advertised.
- jmacd 1y agoI agree. I found it unusable for anything but casual usage due to the rate limiting. I wonder if I am just missing something?
- 1y ago
- statements 1y agoAbsolutely agree. Granted, it is task dependent. But when it comes to classification and attribute extraction, I've been using 2.0 Flash with huge access across massive datasets. It would not be even viable cost wise with other models.
- sethkim 1y agoHow "huge" are these datasets? Did you build your own tooling to accomplish this?
- xnx 1y agoShhhh. You're going to give away the secret weapon!
- gambiting 1y agoIn my experience they are as dumb as a bag of bricks. The other day I asked "can you edit a picture if I upload one" And it replied "sure, here is a picture of a photo editing prompt:" https://g.co/gemini/share/5e298e7d7613 https://g.co/gemini/share/5e298e7d7613 It's like "baby's first AI". The only good thing about it is that it's free.
- JFingleton 1y agoPrompt engineering is a thing. Learning how to "speak llm" will give you great results. There's loads of online resources that will teach you. Think of it like learning a new API.
- abletonlive 1y agofor now. one would hope that this is a transitory moment in llms and that we can just use intuition in the future.
- asadotzler 1y agoLLM's whole thing is language. They make great translators and perform all kinds of other language tasks well, but somehow they can't interpret my English language prompts unless I go to school to learn how to speak LLM-flavored English? WTF?
- pplante 1y agoI like to think of my interactions with an LLM like I'm explaining a request to a junior engineer or non engineering person. You have to be more verbose to someone who has zero context in order for them to execute a task correctly. The LLM only has the context you provided so they fail hard like a junior engineer would at a complicated task with no experience.
- pplante 1y agoI like to think of my interactions with an LLM like I'm explaining a request to a junior engineer or non engineering person. You have to be more verbose to someone who has zero context in order for them to execute a task correctly. The LLM only has the context you provided so they fail hard like a junior engineer would at a complicated task with no experience.
- rvz 1y agoGoogle always has been winning the AI race as soon as DeepMind was properly put to use to develop their AI models, instead of the ones that built Bard (Google AI team).
- GaggiX 1y agoFlash models are really good even for an end user because how fast and good performance they have.
- ghurtado 1y agoI know it's a single data point, but yesterday I showed it a diagram of my fairly complex micropython program, (including RP2 specific features, DMA and PIO) and it was able to describe in detail not just the structure of the program, but also exactly what it does and how it does it. This is before seeing a single like of code, just going by boxes and arrows. The other AIs I have shown the same diagram to, have all struggled to make sense of it.
- redbell 1y ago> Google is silently winning the AI race Yep, I agree! This convinced me: https://news.ycombinator.com/item?id=43661235 https://news.ycombinator.com/item?id=43661235
- ramesh31 1y ago>”Google is silently winning the AI race.” It’s not surprising. What was surprising honestly was how they were caught off guard by OpenAI. It feels like in 2022 just about all the big players had a GPT-3 level system in the works internally, but SamA and co. knew they had a winning hand at the time, and just showed their cards first.
- wkat4242 1y agoTrue and their first mover advantage still works pretty well. Despite "ChatGPT" being a really uncool name in terms of marketing. People remember it because they were the first to wow them.
- golergka 1y agoIt feels more authentically engineer-coded.
- kaoD 1y agoHow is ChatGPT bad in terms of marketing? It's recognizable and rolls off the tongue in many many many languages. Gemini is what sucks from a marketing perspective. Generic-ass name.
- simonw 1y agoGenerative Pre-trained Transformer is a horrible term to have an acronym for.
- kaoD 1y agoDo you think the mass market thinks GPT is an acronym? It's just a name. Currently synonymous with AI. Ask anyone outside the tech bubble about "Gemini" though. You'll get astrology.
- wkat4242 1y agoTrue I guess they treat it just like SMS. I still think they'd have taken off more if they'd given it a catchy name from the start and made the interface a bit more consumer friendly.
- russellbeattie 1y agoI have to say, I never doubted it would happen. They've been at the forefront of AI and ML for well over a decade. Their scientists were the authors of the "Attention is all you need" paper, among thousands of others. A Google Scholar search produces endless results. There just seemed to be a disconnect between the research and product areas of the company. I think they've got that worked out now. They're getting their ass kicked in court though, which might be making them much less aggressive than they would be otherwise, or at least quieter about it.
- Nihilartikel 1y ago100% agree. I had Gemini flash 2 chew through thousands of points of nasty unstructured client data and it did a 'better than human intern' level conversion into clean structured output for about $30 of API usage. I am sold. 2.5 pro experimental is a different league though for coding. I'm leveraging it for massive refactoring now and it is almost magical.
- jdthedisciple 1y ago> thousands of points of nasty unstructured client data What I always wonder in these kinds of cases is: What makes you confident the AI actually did a good job since presumably you haven't looked at the thousands of client data yourself? For all you know it made up 50% of the result.
- deleted 1y ago[deleted]
- golergka 1y agoMany types of data have very easily checkable aggregates. Think accounting books.
- pamplemoose 1y agoYou take a sample and check
- tominous 1y agoIn my case I had hundreds of invoices in a not-very-consistent PDF format which I had contemporaneously tracked in spreadsheets. After data extraction (pdftotext + OpenAI API), I cross-checked against the spreadsheets, and for any discrepancies I reviewed the original PDFs and old bank statements. The main issue I had was it was surprisingly hard to get the model to consistently strip commas from dollar values, which broke the csv output I asked for. I gave up on prompt engineering it to perfection, and just looped around it with a regex check. Otherwise, accuracy was extremely good and it surfaced a few errors in my spreadsheets over the years.
- no_wizard 1y agoI remember everyone saying its a two horse race between Google and OpenAI, then DeepSeek happened. Never count out the possibility of a dark horse competitor ripping the sod right out from under
- nimchimpsky 1y ago[dead]
- nonethewiser 1y agoHow is deepseak doing though? It seemed like they probably just ingested ChatGPT. https://www.forbes.com/sites/torconstantino/2025/03/03/deepseeks-ai-style-matches-chatgpts-74-percent-of-the-time-new-study/ https://www.forbes.com/sites/torconstantino/2025/03/03/deeps... Still impressive but would really put a cap on expectations for them.
- gs17 1y agoThey supposedly have a new R2 model coming within a month.
- FooBarWidget 1y agoEverybody else also trains on ChatGPT data, have you never heard of public ChatGPT conversation data sets? Yes they trained on ChatGPT data. No it's not "just".
- bhl 1y agoIt's cheap but also lazy. It sometimes generates empty strings or empty arrays for tool calls, and then I just re-route the request to a stronger model for the tool call. I've spent a lot of time on prompts and tool-calls to get Flash models to reason and execute well. When I give the same context to stronger models like 4o or Gemini 2.5 Pro, it's able to get to the same answers in less steps but at higher token cost. Which is to be expected: more guardrails for smaller, weaker models. But then it's a tradeoff; no easy way to pick which models to use. Instead of SQL optimization, it's now model optimization.
- paulcole 1y ago> Google is silently winning the AI race. It’s not clear to me what either the “race” or “winning” is. I use ChatGPT for 99% of my personal and professional use. I’ve just gotten used to the interface and quirks. It’s a good consumer product that I like to pay $20/month for and use. My work doesn’t require much in the way of monthly tokens but I just pay for the OpenAI API and use that. Is that winning? Becoming the de facto “AI” tool for consumers? Or is the race to become what’s used by developers inside of apps and software? The race isn’t to have the best model (I don’t think) because it seems like the 3rd best model is very very good for many people’s uses.