16 ms·
Improved Gemini 2.5 Flash and Flash-Lite
- ahmedfromtunis 1y agoI'm genuinely surprised to see that "thinking" flash-lite is more performant than flash with no "thinking".
- scosman 1y agoUgh. If the model name includes sem_ver version number, increment the version number when making a new release! Anthropic learned this lesson. Google, Deepseek, Kimi, OpenAI and others keep repeating it. This feels like Gemini_2.5_final_FINAL_FINAL_v2.
- rsc 1y agoFWIW, the versions are not semver but they do follow a defined and regular version schema: https://ai.google.dev/gemini-api/docs/models#model-versions https://ai.google.dev/gemini-api/docs/models#model-versions.
- Imustaskforhelp 1y agoI am seeing a lot of demand for something like a semver for AI models. Could thereotically there could be something like a semver that can be autogenerated from that defined and regular version scheme that you shared? Like, Honestly my idea of it is that I could use something like openrouter and then just change the semver without having to worry about these soooo many things as the schema that you shared y'know? A website / tool which can create a semver from this defined scheme and vice versa can be really cool actually :>
- CaptainOfCoit 1y agoI'm not sure if this is a joke or not, but in case it isn't: Semver was mostly created so users of libraries could judge if a new release would break the API interfaces or not, by just looking at the version. So unless the first number changed, you're good to go (in theory, in practice this obviously didn't work as expected). With that in mind, what exactly would semver (or similar) represent for AI models? Setup the proper way, your pipelines should continue working regardless of the model, just that the accuracy or some other metric might change slightly. But there should never be any "breakages" like what semver is supposed to help flag.
- scosman 1y agoModels have changes worthy of semver style major changes. Tokenizer, tool support, tool format, JSON modes, etc. Pipelines absolutely must change when these change. This thread is more about the minor number: not incrementing it when making changes to the internals is painful for dependency tracking. These changes will also break apps (prompts are often tuned to the model).
- qafy 1y ago2.5 isn't the version number, its the model generation. it would only be updated when the underlying model architecture, training, etc are updated. this release is, as the name implies, the same model but likely with hardware optimizations, system prompt, and fine-tuning tweaks applied.
- ComputerGuru 1y agoOk, so if not 2.6 then 2.5.1 :)
- scosman 1y agoIf the weights have changed via training (they have) it’s a new model. This isn’t “hardware optimizations”. It’s additional training/new-weights.
- newfocogi 1y agoNon-AI Summary: Both models have improved intelligence on Artificial Analysis index with lower end-to-end response time. Also 24% to 50% improved output token efficiency (resulting in lower cost). Gemini 2.5 Flash-Lite improvements include better instruction following, reduced verbosity, stronger multimodal & translation capabilities. Gemini 2.5 Flash improvements include better agentic tool use and more token-efficient reasoning. Model strings: gemini-2.5-flash-lite-preview-09-2025 and gemini-2.5-flash-preview-09-2025
- Mistletoe 1y ago2.5 Flash is the first time I've felt AI has become truly useful to me. I was #1 AI hater but now find myself going to the Gemini app instead of Google search. It's just better in every way and no ads. The info it provides is usually always right and it feels like I have the whole generalized and accurate knowledge of the internet at my fingertips in the app. It's more intimate, less distractions. Just me and the Gemini app alone talking about kale's ideal germination temperature, instead of a bunch of mommy bloggers, bots, and SEO spam. Now how long can Google keep this going and cannibalizing how they make money is another question...
- yesco 1y agoIt's also excellent for subjective NLP-type analysis. For example, I use it for "scouting" chapters in my translation pipeline to compile coherent glossaries that I can feed into prompts for per-chapter translation. This involves having it identify all potential keywords and distinct entities, determine their approximate gender (important for languages with ambiguous gender pronouns), and then perform a line-by-line analysis of each chapter. For each line, it identifies the speaking entity, determines whose POV the line represents, and identifies the subject entity. While I didn't need or expect perfection, Gemini Flash 2.5 was the only model I tested that could not only follow all these instructions, but follow them well. The cheap price was a bonus. I was thoroughly impressed, it's now my go-to for any JSON-formatted analysis reports.
- indigodaddy 1y agoGoogle AI mode is excellent as well, which I guess is just Gemini 2.5 Flash I'd imagine as well?
- OGEnthusiast 1y agoI'm not even sure how to evaluate what a "better" LLM is, when I've tried running the exact same model (Qwen3) and prompt and gotten vastly different responses on Qwen Chat vs OpenRouter vs running the model locally.
- 1899-12-30 1y agoThat's a difference in the system prompt, not the model itself.
- OGEnthusiast 1y agoTrue yeah, good point.
- daemonologist 1y agoThere several reasons responses from the same model might vary: - "temperature" - intentional random sampling from the most likely next tokens to improve "creativity" and help avoid repetition - quantization - running models with lower numeric precision (saves on both memory and compute, without impacting accuracy too much) - differences in/existence of a system prompt, especially when using something end-user-oriented like Qwen Chat - not-quite-deterministic GPU acceleration Benchmarks are usually run at temperature zero (always take the most likely next token), with the full-precision weights, and no additions to the benchmark prompt except necessary formatting and stuff like end-of-turn tokens. They also usually are multiple-choice or otherwise expect very short responses, which leaves less room for run-to-run variance. Of course a benchmark still can't tell you everything - real-world performance can be very different.
- OGEnthusiast 1y agoThanks, this is a good checklist.
- magicalhippo 1y agoAFAIK the batch your query lands in can also matter[1]. Though I imagine this should be a smaller effect than different quantization levels say. [1]: https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
- Liwink 1y agoGemini 2.5 Flash is an impressive model for its price. However, I don't understand why Gemini 2.0 Flash is still popular. From OpenRouter last week: * xAI: Grok Code Fast 1: 1.15T * Anthropic: Claude Sonnet 4: 586B * Google: Gemini 2.5 Flash: 325B * Sonoma Sky Alpha: 227B * Google: Gemini 2.0 Flash: 187B * DeepSeek: DeepSeek V3.1 (free): 180B * xAI: Grok 4 Fast (free): 158B * OpenAI: GPT-4.1 Mini: 157B * DeepSeek: DeepSeek V3 0324: 142B
- crazysim 1y agoMaybe the same reason why they kept the name for the 2.5 Flash update. People are lazy at pointing to the latest name.
- koakuma-chan 1y agoWhy is Grok so popular
- coder543 1y agoI think it has been free in some editor plugins, which is probably a significant factor. I would rather use a model that is good than a model that is free, but different people have different priorities.
- YetAnotherNick 1y agoNon free has double usage than free. Free one uses your data for training.
- Imustaskforhelp 1y agoI mean, I can kinda roll through a lot of iterations with this model without worrying about any AI limits. Y'know with all these latest models, the lines are kinda blurry actually. The definition of "good" is being foggy. So it might as well be free as the definition of money is clear as crystal. I also used it for some time to test on something really really niche like building telegram bot in cloudflare workers and grok-4-fast was kinda decent on that for the most part actually. So that's nice.
- tardyp 1y agoLLM Model versioning really makes me perplex those days...
- _ea1k 1y agoYeah, why is it that working with AI makes people completely forget what version numbers mean? gemini-2.5-flash-preview-09-2025 - what are they thinking? I thought about joking that they had AI name it for them, but when I asked Gemini, it said that this name was confusing, redundant, and leads to unnecessarily high cognitive load. Maybe Googlers should learn from their own models.
- iamdelirium 1y agoBecause the number is model generation.
- ImPrajyoth 1y agoI’ve been tinkering with the last version for code gen. This update might finally put it on par with Claude for latency. Anyone tried benchmarking the new preview yet?
- aeon_ai 1y agoI think a Model-specific SemVer needs to be created to be clearer as to what degree of change has taken place, in the age of model weights. Something that distinguishes between a completely new pre-training process/architecture, and standard RLHF cycles/optimizations.
- brap 1y agoAm I the only one who is starting to feel the Gemini Flash models are better than Pro? Flash is super fast, gets straight to the point. Pro takes ages to even respond, then starts yapping endlessly, usually confuses itself in the process and ends up with a wrong answer.
- selimthegrim 1y agoI tried to put Pro deep research on an actual research task and it didn’t even return anything just kept on working.
- gnulinux 1y agoThis is not my experience. In my experience Gemini 2.5 Pro is the best model in every use-case I tried. There are a few very hard (graduate level) logic or math problems that Claude 4.1 Opus edged-out over Gemini 2.5 Pro, but in general if you have no idea which model will perform best on a difficult question, imho Gemini 2.5 Pro is a safer bet especially since it's significantly cheaper. Gemini 2.5 Flash is really good but imho not nearly as good as Pro in (1) research math (2) creative/artistic writing (3) open ended programming debugging. On the other hand, I do prefer using Claude 4 Sonnet on very open-ended agentic programming tasks because it seems to have a better integration with VSCode Copilot. Gemini 2.5 Pro bugs out much more often where Claude works fine almost every time.
- dvkramer 1y agoYeah that's how I feel too. Flash is less verbose and every LLM nowadays seems to be designed by some low-taste people who reward the model for falsely hedging (i.e. "The 2024 Corolla Cross usually has an X gallon gas tank") on stuff that isn't at all variable or questionable. This false hedging is way more of an issue than hallucinations in my experience and the "smarter" 2.5 Pro is not any better at avoiding this issue than Flash Also 2.5 Pro is often incapable of searching and will hallucinate instead. I don't know why. It will claim it searched and then return some made up results instead. 2.5 Flash is much more consistently capable of searching
- ashwindharne 1y agoGoogle seems to be the main foundation model provider that's really focusing on the latency/TPS/cost dimensions. Anthropic/OpenAI are really making strides in model intelligence, but underneath some critical threshold of performance, the really long thinking times make workflows feel a lot worse in collaboration-style tools, vs a much snappier but slightly less intelligent model. It's a delicate balance, because these Gemini models sometimes feel downright lobotomized compared to claude or gpt-5.
- jjani 1y agoCan't agree with that. Gemini doesn't lead just on price/performance - ironically it's the best "normie" model most of the time, despite it's lack of popularity with them until very recent. It's bad at agentic stuff, especially coding. Incomparably so compared to Claude and now GPT-5. But if it's just about asking it random stuff, and especially going on for very long in the same conversation - which non-tech users have a tendency to do - Gemini wins. It's still the best at long context, noticing things said long ago. Earlier this week I was doing some debugging. For debugging especially I like to run sonnet/gpt5/2.5-pro in parallel with the same prompt/convo. Gemini was the only one that, 4 or so messages in, pointed out something very relevant in the middle of the logs in the very first message. GPT and Sonnet both failed to notice, leading them to give wrong sample code. I would've wasted more time if I hadn't used Gemini. It's also still the best at a good number of low-resource languages. It doesn't glaze too much (Sonnet, ChatGPT) without being overly stubborn (raw GPT-5 API). It's by far the best at OCR and image recognition, which a lot of average users use quite a bit. Google's ridiculously bad at marketing and AI UX, but they'll get there. They're already much more than just a "bang for the buck" player. FWIW I use all 3 above mentioned on a daily basis for a wide variety of tasks, often side-by-side in parallel to compare performance.
- dpoloncsak 1y agoDoes it still try to 'unplug' itself if it gets something wrong, or did they RL that out yet?
- 1y ago
- ChildOfChaos 1y agoHopefully this isn't instead of the rumoured Gemini 3 pro this week.
- Imustaskforhelp 1y agoI think that the Gemini 3 pro might be next month I am not sure. can I get the sources of your rumour please? (Yes I know that I can search it but I would honestly prefer it if you could share it, thanks in advance!)
- ChildOfChaos 1y agoBens bites was suggesting we might be Gemini 3 pro and Claude 4.5 this week. To be honest, I hadn't heard that elsewhere, but I haven't been following it massively this week.
- fnordsensei 1y agoNext week is next month.
- Imustaskforhelp 1y agoI swear I forgot :sob: I AM LAUGHING SO HARD RIGHT NOWWWWW LMAOOOO I wish to upvote this twice lol
- minimaxir 1y agoGemini 2.5 Flash has been the LLM I've used the most recently for a variety of domains, especially image inputs and structured outputs which beat both OpenAI and Anthropic in my opinion.
- Fiahil 1y agoQuestion to the one that tested it : Does it still timeout a lot with unreliable response time (1-5 sec) ?
- zitterbewegung 1y agoOkay this is a nitpick but why wouldn't you increment a part of the version number to signify that there is an improvement? These releases are confusing.
- bl4ckneon 1y agoI would assume that it will supersede the model that they currently have. So eventually 2.5 flash will be the new and improved 2.5 Flash rather than 2.6. Same way that openai updated their 4-o models and the like, which didn't turn out so well when it started glazing everyone and they had to revert it (maybe that was just chat and not api)
- zitterbewegung 1y agoEven if it was just chat and or API I have used the API and I know that they have at minimum added the retraining date and time that they could just affix to the Gemini 2.5 Flash and Flash-Lite because when I use the API I have to verify that the upgrade of the backend system didn't break anything and pinning versions I assume is pretty common.
- TIPSIO 1y agoThis is also my beef... Anthropic kind of did the same thing [1] except it back-fired recently with the cries of "nerfing". We buy these tokens, which are very hard to do in limited tiers, they expire after only a year, and we don't even know how often the responses are changing in the background. Even a 1% improvement or reduction I would want disclosed. Really scary foundation AI companies are building on IMO. Transparency and access is important. [1] https://status.claude.com/incidents/h26lykctfnsz https://status.claude.com/incidents/h26lykctfnsz
- Aeolun 1y agoAre your tokens at any risk of lasting longer than a year? When I buy them it’s generally because I expect to use them reasonably soonish.
- deleted 1y ago
- davidmckayv 1y agoThis really captures something I've been experiencing with Gemini lately. The models are genuinely capable when they work properly, but there's this persistent truncation issue that makes them unreliable in practice. I've been running into it consistently, responses that just stop mid-sentence, not because of token limits or content filters, but what appears to be a bug in how the model signals completion. It's been documented on their GitHub and dev forums for months as a P2 issue. The frustrating part is that when you compare a complete Gemini response to Claude or GPT-4, the quality is often quite good. But reliability matters more than peak performance. I'd rather work with a model that consistently delivers complete (if slightly less brilliant) responses than one that gives me half-thoughts I have to constantly prompt to continue. It's a shame because Google clearly has the underlying tech. But until they fix these basic conversation flow issues, Gemini will keep feeling broken compared to the competition, regardless of how it performs on benchmarks. https://github.com/googleapis/js-genai/issues/707 https://github.com/googleapis/js-genai/issues/707 https://discuss.ai.google.dev/t/gemini-2-5-pro-incomplete-responses/94600 https://discuss.ai.google.dev/t/gemini-2-5-pro-incomplete-re...
- dorianmariecom 1y agochatgpt also has lots of reliability issues
- diego_sandoval 1y agoIf anyone from OpenAI is reading this, I have two complaints: 1. Using the "Projects" thing (Folder organization) makes my browser tab (on Firefox) become unusably slow after a while. I'm basically forced to use the default chats organization, even though I would like to organize my chats in folders. 2. After editing a message that you already sent,you get to select between the different branches of the chat (1/2, and so on), which is cool, but when ChatGPT fails to generate a response in this "branched conversation" context, it will continue failing forever. When your conversation is a single thread and a ChatGPT message fails with an error, re trying usually works and the chat continues normally.
- 1y ago
- bogtog 1y ago> Today, we are releasing updated versions of Gemini 2.5 Flash and 2.5 Flash-Lite, available on Google AI Studio and Vertex AI, aimed at continuing to deliver better quality while also improving the efficiency. Typo in the first sentence? "... improving the efficiency." Gemini 2.5 Pro says this is perfectly good phrasing, whereas ChatGPT and Claude recognize that it's awkward or just incorrect. Hmm...
- gpm 1y ago"Improving the efficiency" sounds fine to me (a native English speaker), what's wrong with it in your opinion?
- bre1010 1y agoYou would just say "improving efficiency". Whereas theirs is like: "Improving the efficiency [... of what?]"
- codazoda 1y agoYou left out words at the front that are important. “deliver better quality while also improving the efficiency.” Reads fine to me. An editor would likely drop “the”.
- latentnumber 1y ago"the" is redundant is probably what GP means.
- burkaman 1y agoUsually you would say "improving the efficiency of x and y". In this case at the end of the sentence it should be "improving the models' efficiency" or just "improving efficiency". I don't think it's "wrong" and it's obviously clear what they mean, but I agree that the phrasing is a little awkward.
- mwest217 1y agoChatGPT and Claude are mistaken if they think it is incorrect. The parallelism in verb tenses is between "continuing to deliver" and "improving the efficiency". It's a bit wordy, but definitely not wrong.
- simonw 1y agoI added support to these models to my llm-gemini plugin, so you can run them like this (using uvx so no need to install anything first): export LLM_GEMINI_KEY='...' uvx --isolated --with llm-gemini llm -m gemini-flash-lite-latest 'An epic poem about frogs at war with ducks' Release notes: https://github.com/simonw/llm-gemini/releases/tag/0.26 https://github.com/simonw/llm-gemini/releases/tag/0.26 Pelicans: https://github.com/simonw/llm-gemini/issues/104#issuecomment-3335485366 https://github.com/simonw/llm-gemini/issues/104#issuecomment...
- canadiantim 1y agoWho wins in the end? the frogs? the ducks? or the pelicans?
- zamalek 1y agoI wonder if [good examples of] SVGs of pelicans on bikes are "being introduced" into training sets. Some of the engineers who work on this stuff are the kind to hang out here.
- simonw 1y agoIt's possible, but honestly I've never seen a decent vector illustration of a pelican on a bicycle myself so they'd have to work pretty hard to find one!
- dimal 1y agoThey could just ask a designer to do a few bespoke illustrations, then generate synthetic data from that, right? Have an image model generate a set of variations, then convert them to SVG. But looking at these images, Google clearly hasn’t done that yet.
- modeless 1y agoWhy are model providers allergic to version number increments?
- recursive 1y agoBecause they want to retain the ability to do silent changes. They can't let people get used to stable version == stable result.
- dcchambers 1y agoWhy do all of these model providers have such issues naming/versioning them? Why even use a version number (2.5) if you aren't going to change it when you update the model? This industry desperately needs a Steve Jobs to bring some sanity to the marketing.
- jama211 1y agoSeems llm progress really is plateauing. I guess that was to be expected.
- throwuxiytayq 1y agoAnd this existing model’s update is evidence how? What were your expectations of this update? I actually even agree that the progress is plateauing, but your comment is a non-sequitur.
- jama211 1y agoI’ll admit it is a bit of a non-sequitur. Just feels like the news I see on HN about LLMs is less groundbreaking every day and more becoming normal/boring
- throwuxiytayq 1y agoThis article is about Google improving something. Sounds pretty out of the ordinary to me
- jama211 1y agoHahaha fair
- Workaccount2 1y agoThis is a performance update to a previous generation model. It's not a new model.
- jama211 1y agoI’m fully aware.
- rpdillon 1y agoNot really. A lot of new amazing Qwen models just dropped.
- deleted 1y ago[deleted]
- simianwords 1y agoWhich model does gemini.goolge.com use when I choose 2.5 flash here?
- saltserv 1y ago[dead]
- fzimmermann89 1y agoThe switch by Artificial Analysis from per-token-cost to per-benchmark-cost shows some effect! Its nice that labs are now trying to optimize what I actually have to pay to get an answer - It always annoys me to have to pay for all the senseless rambling of the less-capable reasoning models.
- svantana 1y agoDid they? I'm looking at the Artificial Analysis leaderboard site now and I only see price as USD/1M tokens.
- agluszak 1y agoWhy isn't it called Gemini 2.6 then?
- thrownawayohman 1y agoWow checking cool
- pier25 1y agoThe most annoying thing about Gemini is that it can't stop suggesting youtube videos. Even when you ask it to stop doing that, multiple times in the same conversation, it will just keep doing it.
- lelanthran 1y agoMight be be builtin to the model because it is impossible to remove completely... And I say this because, I added about 50 prompts in the settings to prevent video recommendations and to remove any links to videos. but I still get text saying "the linked video explains this more" even though there is no linked video. This is not a bad way to monetise the free tier. Non of the other token providers found any way to monetise the free tier but Gemini is doing it on almost every prompt.
- niekiepriekie 1y agoThis! I feel he suddenly started doing this even though I've told him to stop. And he knows, every time he tells me he's so sorry. It feels like Google is already monetizing Gemini for their ad market.
- phartenfeller 1y agoIt's weird that the just keep the version number. Why not release it as 2.6 or something else. Now it is confusing, do my existing workflows automatically use the updated version and if yes do I need to monitor them for unwanted changed behavior etc.
- barbazoo 1y agoIf you want stable models I think you could get that through Azure.
- stephen_cagle 1y agoI still can't understand how functioning adults believe that releasing their work in two separate places is a good idea (Ai Studio and Vertex AI).
- deleted 1y ago[deleted]
- herpderperator 1y agoSerious question: If it's an improved 2.5 model, why don't they call it version 2.6? Seems annoying to have to remember if you're using the old 2.5 or the new 2.5. Kind of like when Apple released the third-gen iPad many years ago and simply called it the "new iPad" without a number.
- skerit 1y agoThat's why people called the second version of Sonnet v3.5 simply v3.6, and Anthropic acknowledged that by naming the next version v3.7
- Aeolun 1y agoOnly Anthropic has a slightly understandable version scheme.
- qafy 1y ago2.5 is not the version number, it's the generation of the underlying model architecture. Think of it like the trim level on a Mazda 3 hatchback. Mazda already has the Mazda 3 Sport in their lineup, then later they release the Mazda 3 Turbo which is much faster. When they release this new version of the vehicle its not called the Mazda 4... that would be an entirely different vehicle based on a new platform and powertrain etc (if it existed). The new vehicle is just a new trim level / visual refresh of the existing Mazda 3. That's why Google names it like this, but I agree its dumb. Semver would be easier.
- pests 1y agoGonna steal this to help explain to non tech friends when it comes up again.
- someguyiguess 1y agoI’d say it’s more like naming your Operating System off of the kernel version number.
- alwillis 1y ago
- artur_makly 1y agoGrok 4-Fast still looks much better in terms of price: https://x.com/ArtificialAnlys/status/1971273380335845683 https://x.com/ArtificialAnlys/status/1971273380335845683 going to stick to that for bit and see.. Gemini 2.5 Flash Preview $0.30 $2.50 Grok 4 Fast $0.20 $0.50
- Hobadee 1y agoAm I using a different Gemini from everyone else? We have Google Workspace at my job, so Gemini is baked in. It is HORRENDOUS when compared to other models. I hear a bunch of other people talking about how great Gemini is, but I've never seen it. The responses are usually either incorrect, way too long, (essays when I wanted summaries) or just...not...good. I will ask the exact same question to both Gemini and ChatGPT (free) and GPT will give a great answer while the Gemini answer is trash. Am I missing something?
- do_anh_tu 1y agoMaybe you are using it wrong.
- Twirrim 1y agoI've been finding it leaps and bounds above other models but I'm only using it via aistudio. I haven't tried any IDE integration or similar, so can't talk to that. I do still have to tell it to stop it with the effusive praise (I guess that also helps reduce context windows)
- mastercheif 1y agoI agree. I think it comes down OpenAI's superior post-training. ChatGPT is better at: A) Interpreting what I'm asking it for me needing to provide additional explicit context. B) Formatting answers in a way that are easily digestible.
- ls612 1y agoI use Gemini almost exclusively for coding and 2.5 Pro is extremely good at it. It has revised hundreds of lines of academic code for me at a time and the results run correctly with only minor revision. I will also say whatever they use for the AI search summary is good enough for me like 50% of the time I google something, but those are generally the simpler 50% of queries.
- BlueGh0st 1y agoI have the same sentiment. I've never really had success using Gemini outside of translation. Although, even with that, Gemini would often refuse and I had to remind it that it does actually know other languages. My most recent trials output single commas as responses to basic questions or it simply refuses the task on ethical grounds such as generating a photo of a backpack wearing a hoodie for some reason (it claimed harmful stereotypes and instead generated an ape). Refusing to do perfectly ethical tasks is probably the most consist problem I've had.
- DoctorOetker 1y agoI would really like to see the 270M but which also knows phonetic alphabetic pronounciation in sentences. Perhaps IPA? I would like to try a small computer->human "upload" experiment, basic multilingual understanding without pronounciation knowledge would be very sad. I intend to make a sort of computer reflexive game, I want to compare different upload strategies (with/without analog or classic error correcting codes, empirical spaced repetition constants, a ML predictor of which parameters I'm forgetting / losing resolution on.
- maxdo 1y agoI tried to switch today from gpt-4.1 , one of the few models with decent response time and ok quality. It’s not on par unfortunately
- guybedo 1y agoi just switched my project to this new flash-lite version. Here's a summary of this discussion with the new version: https://extraakt.com/extraakts/the-great-llm-versioning-debate-why-can-t-we-have-clear-semver https://extraakt.com/extraakts/the-great-llm-versioning-deba...
- user3939382 1y agoGemini is also the name of a protocol which, I appreciate most disagree, but I find is actual much more important than Google’s AI.
- rasz 1y agoThrew few short python scripts at 2.5. Got stupid messages like "OMG Significant Flaw!!1 all of your functions have non-obvious dependency on this global variable declared in main, nothing will work if you dont execute main first!!1" I mean sure, technically correct, the best kind of LLM correct. It kept finding those fatal flaws and starting to explain them to then slowly finish with "oh yes this works as intended".
- boomer_joe 1y agoGemini 2.5 Pro feels heavily lobotomized for me lately, failing at very simple tasks with a frequency far above what I was used to seeing back when it first released. The personality seems to be getting worse too - I'm getting very tired of those dumbed analogies it loves to spew. Would like to know whether Flash exhibits these issues as well.
- grej 1y agoI love the gemini models and think Google has done a great job on them, but no model series I use seems to get context rot more in long conversations. Which seems strange given the longer context.
- sreekanth850 1y agoCode with gemini code assist and sanity check with sonnet is my current way.
- strangescript 1y agoFlash-Lite is a seriously good model. I have had zero structured calls fail with it as its cranking out obscene tok/s. If you can run with something that isn't quite bleeding edge smart, this model is gold.
- rafaelero 1y agoThis new Gemini Flash 2.5 is cutting the response in the middle. Did anyone experience that?
- Moosdijk 1y agoMy experience with Gemini is the sole reason I am convinced that there's an AI hype going on. It consistently hallucinates key information which has led me to spend countless hours tracking down which information the output was based on, only to find that it dreamt up the facts that it gave to me. The way I have come to perceive AI is that it's mostly good at reassuring/reaffirming people's beliefs and ideas than an actual source of truth. That would not be an issue if it was actually marketed as such, but seeing the "guided learning" function fail time and again makes me think we should be a lot more critical of what we're being told by tech enthusiasts/companies about AI.
- ikgn 1y agoI have a small test suite for the voice AI math tutor we built, about 50 tests, mostly about correctly following the system instructions. The newly released Flash 2.5 is much worse than current stable version. Gemini 2.5 pro will fail 2—3 tests. Flash 2.5 stable, which we use in production, fails about 10, and the new one fails 20. Every test runs 3 times and the model has to be right every time. Will look into it more, I haven‘t yet looked into actual output. This is not about solving math, the system follows given solution paths.
- rldjbpin 1y agohaving developed a large-batch workflow for a client using gemini models, this is a welcome improvement. however, no news on the DSQ [1] issues is a bummer. at least for us, the bottleneck is the amount of retries/waiting needed to max out how many requests we can make in parallel. [1] https://cloud.google.com/vertex-ai/generative-ai/docs/dynamic-shared-quota https://cloud.google.com/vertex-ai/generative-ai/docs/dynami...
- whinvik 1y agoHaving done some tests, its clearly better at instruction following and JSON output now. However its hampered by max output tokens. Gemini is at 65 K while GPT 5 mini is at 128K. Both of them have similar costs as well so as such apart from the 1M context limit GPT 5 mini is better in every way.
- dgemm 1y agoI just wish the Gemini app would stop inserting and auto playing a YouTube video into nearly every response when I'm on a mobile connection. There appears to be no way to stop it.
- sinuhe69 1y agoMaybe disallow autoplay on your Youtube account can help. Gemini insert YT video in my answers as well, but they don't auto play.
- lysecret 1y agoOk wow these models are great and fast! Tested it for pdf extraction tasks.