12 ms·
Gemini 2.5 Deep Think
- amatic 1y agoAt the moment, Deep Think is only available with the ULTRA subscription ($250 per month).
- stingraycharles 1y agoIt’s not available through the API?
- siva7 1y agoIs it available in EU? Someone can confirm?
- highfrequency 1y agoApproach is analogous to Grok 4 Heavy: use multiple "reasoning" agents in parallel and then compare answers before coming back with a single response, taking ~30 minutes. Great results, though it would be more fair for the benchmark comparisons to be against Grok 4 Heavy rather than Grok 4 (the fast, single-agent model).
- stingraycharles 1y agoYeah the general “discovery” is that using the same reasoning compute effort, but spreading them over multiple different agents generally leads to better results. It solves the “longer thinking leads to worse results” problem by approaching multiple paths of thinking in parallel, but just not think as long.
- Xenoamorphous 1y ago> Yeah the general “discovery” is that using the same reasoning compute effort, but spreading them over multiple different agents generally leads to better results. Isn’t the compute effort N times as expensive, where N is the number of agents? Unless you meant in terms of time (and even then, I guess it’d be the slowest of the N agents).
- NitpickLawyer 1y agoNot exactly N times, no. In a traditional transformer arch token 1 is cheaper to generate than token 1000 is cheaper than token 10k and so on. So having 10x 1000 tokens would be cheaper to run concurrently than 10.000 in one session. You also run into context issues and quality degradation the longer you go. (this is assuming gemini uses a traditional arch, and not something special regarding attention)
- stingraycharles 1y agoThe idea is that instead of assigning 10,000 thinking tokens to one chain of thought, assigning 1,000 thinking tokens to 10 chains of thought and composing those independent outputs into a single output yields better results. The fact that it can be done in parallel is just a bonus.
- lynx97 1y agoI am surprised such a simple approach has taken so long to be actually used. My first image description cli attempt did basically that: Use n to get several answers and another pass to summarize.
- cinntaile 1y agoIt's very resource intensive so maybe they had to wait until processes got more efficient? I can also imagine they would want to try and solve it in a... better way before doing this.
- simianwords 1y agoI agree but I think its hard to get a sufficient increase in performance that would justify 3-4x increase in cost.
- datadrivenangel 1y agoIt's an expensive approach, and depends on assessment being easy, which is often not the case.
- golol 1y agoPeople have played with (multi-) agentic frameworks for LLMs from the very beginning but it seems like only now with powerful reasoning models it is really making a difference.
- NitpickLawyer 1y agoI have a similar thing built around a year ago w/ autogen. The difference now is models can really be steered towards "part" of the overall goal, and they actually follow that. Before this, even the best "math" models were RLd to death to only solve problems. If you wanted it to explore "method_a" of solving a problem you'd be SoL. The model would start like "ok, the user wants me to explore method_a, so here's the solution: blablabla doing whatever it wanted, unrelated to method_a. Similar things for gathering multiple sources. Only recently can models actually pick the best thing out of many instances, and work effectively at large context lengths. The previous tries with 1M context lengths were at best gimmicks, IMO. Gemini 2.5 seems the first model that can actually do useful stuff after 100-200k tokens.
- mvieira38 1y agoIs o3-pro the same as these?
- satvikpendem 1y agoNo, it doesn't take 30 minutes
- Workaccount2 1y agoGrok-4 heavy benchmarks used tools, which trivializes a lot of problems.
- littlestymaar 1y agoThat this kind of approach works is good news for local LLM enthusiasts, as it makes Cloud LLM using this more expensive while local LLM can do so for free up to a point (because LLM inference is limited by memory bandwidth not compute, you can run multiple queries in parallel on your graphic card at the same speed as the single one. Until you become compute-bound of course).
- mettamage 1y agoWait, how does this work? If you load in one LLM of 40 GB, then to load in four more LLMs of 40 GB still takes up an extra 160 GB of memory right?
- masterjack 1y agoIt will typically be the same 40 GB model loaded in, but called with many different inputs simultaneously
- zozbot234 1y ago> because LLM inference is limited by memory bandwidth not compute, you can run multiple queries in parallel on your graphic card at the same speed as the single one I don't think this is correct, especially given MoE. You can save some memory bandwidth by reusing model parameters, but that's about it. It's not giving you the same speed as a single query.
- littlestymaar 1y agoAbsolutely, it's not going to work that well for MoE, though today most local models (except Qwen3-30B-A3B) are dense ones. But even for MoE it will still work: sure the second parallel agent running is going to divide the token rate by almost two, but the reduction is exponentially decreasing and the 30th will almost be free. So if you have enough VRAM to run Qwen3-32B, you can run Qwen3-30B-3A at the same speed as the 32B version but you'll be running a hundred of instances.
- dweekly 1y agoDumb (?) question but how is Google's approach here different than Mixture of Experts? Where instead of training different experts to have different model weights you just count on temperature to provide diversity of thought. How much benefit is there in getting the diversity of thought in different runs of the same model versus running a consortium of different model weights and architectures? Is there a paper contrasting results given fixed computation between spending that compute on multiple runs of the same model vs different models?
- joaogui1 1y agoMixture of Experts isn't using multiple models with different specialties, it's more like a sparsity technique, where you massively increase the number of parameters and use only a subset of the weights in each forward pass.
- HarHarVeryFunny 1y agoMOE is just a way to add more parameters/capacity to a model without making it less efficient to run, since it's done in a way that not all parameters are used for each token passing through the model. The name MOE is a bit misleading since the "experts" are just alternate paths though part of the model, not having any distinct expertise in the way the name might suggest. Just running the model multiple times on the same input and selecting the best response (according to some judgement) seems a bit of a haphazard way of getting much diversity of response, if that is really all it is doing. There are multiple alternate approaches to sampling different responses from the model that come to mind, such as: 1) "Tree of thoughts" - generate a partial response (e.g. one token, or one reasoning step), then generate branching continuations of each of those, etc, etc. Compute would go up exponentially according to number of chained steps, unless heavy pruning is done similar to how it is done for MCTS. 2) Separate response planning/brainstorming from response generation by first using a "tree of thoughts" like process just to generate some shallow (e.g. depth < 3) alternate approaches, then use each of those approaches as additional context to generate one or more actual responses (to then evaluate and choose from). Hopefully this would result in some high level variety of response without the cost of of just generating a bunch of responses and hoping that they are usefully diverse.
- bitshiftfaced 1y agoWhat makes you sure of that? From the article, > Deep Think pushes the frontier of thinking capabilities by using parallel thinking techniques. This approach lets Gemini generate many ideas at once and consider them simultaneously, even revising or combining different ideas over time, before arriving at the best answer. This doesn't exclude the possibility of using multiple agents in parallel, but to me it doesn't necessarily mean that this is what's happening, either.
- highfrequency 1y agoWhat could “parallel thinking techniques” entail if not “using multiple agents in parallel”?
- JyB 1y agoHow can it not be exactly what’s happening?
- Melatonic 1y agoSurprised no one has released an app yet that pits all the major models against each other for a final answer.
- 7373737373 1y ago139.99€/month for something you can't even test first, lol
- lynx97 1y agoWait... So if someone cool enough, they could actually give us a DeepThought model? Please, let that happen. Vendor-DeepThought-42B maybe?
- xnx 1y ago> could actually give us a DeepThought model Yes, but the response time is terrible. 7.5 million years
- stocksinsmocks 1y agoQwen-DeepInThot-69b-Instruct is what I’m looking forward to.
- simianwords 1y agoGrok 4 heavy, o3 pro and Gemini Deep Think all are equivalent. I wonder how they compare?
- deleted 1y ago[deleted]
- askl 1y ago[flagged]
- baggachipz 1y agoYou can't go anywhere without having Gemini shoved in your face. I had an immediate visceral reaction to this.
- serialNumber 1y agoI’m never the one to defend AI, but what do you mean? Is it the “AI overview” that pops up on Google? Other than that, I would say Gemini is definitely less in your face than ChatGPT for example
- baggachipz 1y agoMy company uses google workspace and every google doc, spreadsheet, calendar, online meeting and search puts nonstop callouts and messages about using Gemini. It's gotten so bad that I'm about to try building a browser extension to block that bullshit. It clutters the UI and nags. If I wanted that crap, I'd turn it on.
- serialNumber 1y agoI see - I feel the same way about Copilot (as my company uses the Microsoft ecosystem).
- breppp 1y agoYes it smells like google+, but at least gemini is actually a good product
- int_19h 1y agoI also find those "please please please try me!" popups annoying, but at least Google Workspace is one product where deep AI integration actually makes sense. I like the ability to quickly get summaries or edit documents by telling it what needs to be done in generic terms.
- gkbrk 1y agoSounds like you don't like a product choice that your company made. Have you considered talking to your company?
- reddalo 1y ago[flagged]
- resoluteteeth 1y ago> If you’re a Google AI Ultra subscriber, you can use Deep Think in the Gemini app today with a fixed set of prompts a day by toggling “Deep Think” in the prompt bar when selecting 2.5 Pro in the model drop down. If fixed set means fixed number it would be nice to know how many. Otherwise i would like to know what fixed set means here.
- Workaccount2 1y agoYou get 10 requests per day it seems. Apparently the model will think for 30+ minutes on a given prompt. So it seems it's more for research or dense multi-faceted problems than for general coding or writing fan fic.
- barapa 1y agoAvailable via API?
- asaddhamani 1y agoI find it interesting, how OpenAI came out with a $200 plan, Anthropic did $100 and $200, then Gemini ups it to $250, and now Grok is at $300. OpenAI is the only one that says "practically unlimited" and I have never hit any limit on my ChatGPT Pro plan. I hit limits on Claude Max (both plans) several times. Why are these companies not upfront about what the limits are?
- thimabi 1y ago> Why are these companies not upfront about what the limits are? Most likely because they reserve the right to dynamically alter the limits in response to market demands or infrastructure changes. See, for instance, the Ghibli craze that dominated ChatGPT a few months ago. At the time OpenAI had no choice but to severely limit image generation quotas, yet today there are fewer constraints.
- jstummbillig 1y agoBecause if you are transparent about the limits, more people will start to game the limits, which leads to lower limits for everyone – which is a worse outcome for almost everyone. tldr: We can't have nice things, because we are assholes.
- holowoodman 1y agoBecause they want to have their cake and eat it too. A fair pricing model would be token-based, so that a user can see for each query how much they cost, and only pay for what they actually used. But AI companies want a steady stream of income, and they want users to pay as much as possible, while using as little as possible. Therefore they ask for a monthly or even yearly price with an unknown number of tokens included, such that you will always pay more then with token-based payments.
- hnav 1y agoI don't think it's that, I think they just want people to onboard onto these things before understanding what the actual cost might be once they're not subsidized by megacorps anymore. Something similar to loss-leading endeavors like Uber and Lyft in the 2010s, I suspect that that showing the actual cost of inference would raise questions about the cost effectiveness of these things for a lot of applications. Internally, Google's data query surface tell you cost in terms of SWE-time (e.g. this query cost 1 SWE hour) since the incentives are different.
- cpursley 1y agoI can’t even convince Gemini CLI while planning things to not go off and make a bunch of random changes on its own, even after being very clear not to do so, intercepting to tell it to stop doing that, then it just continues on fucking everything up.
- WhitneyLand 1y agoAgents muddy the waters. Claude Code gets the most out of Anthropic’s models, that’s why people love it. Conversely, Gemini CLI makes Gemini Pro 2.5 less capable than the model itself actual is. It’s such a stark difference I’ve given up using Gemini CLI even with it being free, but still use it for situations amenable to a prompt interface on a regular basis. It’s a very strong model.
- deleted 1y ago[deleted]
- panarky 1y agoThat's my experience too, when I give Gemini CLI a big, general task and just let it run. But if I give it structure so it can write its own context, it is truly astonishing. I'll describe my big, general task and tell it to first read the codebase and then write a detailed requirements document, and not to change any code. Then I'll tell it to read the codebase and the detailed requirements document it just wrote, and then write a detailed technical spec with API endpoints, params, pseudocode for tricky logic, etc. Then I'll tell it to read the codebase, and the requirements document it just wrote, and the tech spec it just wrote, and decomp the whole development effort into weekly, daily and hourly tasks to assign to developers and save that in a dev plan document. Only then is it ready to write code. And I tell it to read the code base, requirements, tech spec and dev plan, all of which it authored, and implement Phase 1 of the dev plan. It's not all mechanical and deterministic, or I could just script the whole process. Just like with a team of junior devs, I still need to review each document it writes, tweak things I don't like, or give it a better prompt to reflect my priorities that I forgot to tell it the first time, and have it redo a document from scratch. But it produces 90% or more of its own context. It ingests all that context that it mostly authored, and then just chugs along for a long time, rarely going off the rails anymore.
- TechDebtDevin 1y ago[flagged]
- simonw 1y agoThis isn't the exact same model that achieved gold in the IMO a few weeks ago but is a close relative: https://x.com/OfficialLoganK/status/1951262261512659430 https://x.com/OfficialLoganK/status/1951262261512659430 It's not yet available via an API.
- foundry27 1y agoI started doing some experimentation with this new Deep Think agent, and after five prompts I reached my daily usage limit. For $250 USD/mo that’s what you’ll be getting folks. It’s just bizarrely uncompetitive with o3-pro and Grok 4 Heavy. Anecdotally (from my experience) this was the one feature that enthusiasts in the AI community were interested in to justify the exorbitant price of Google’s Ultra subscription. I find it astonishing that the same company providing free usage of their top models to everybody via AI Studio is nickel-and-diming their actual customers like that. Performance-wise. So far, I couldn’t even tell. I provided it with a challenging organizational problem that my business was facing, with the relevant context, and it proposed a lucid and well-thought-out solution that was consistent with our internal discussions on the matter. But o3 came to an equally effective conclusion for a fraction of the cost, even if it was less “cohesive” of a report. I guess I’ll have to wait until tomorrow to learn more.
- andsoitis 1y agoit turns out that AI at this level is very expensive to run (capex, energy). my bet is that AI itself won't figure out how to overcome these constraints and reach escape velocity.
- crowcroft 1y agoMainframes are the only viable way to build computers. Micro processors will never figure out how to get small and fast enough for personal computers to reach escape velocity.
- andsoitis 1y agoWhy do you think the analogy hold?
- suddenlybananas 1y agoWell it's an analogy, that's a water-tight argument.
- 1y ago
- TheAceOfHearts 1y agoI would be interested in reading about how people who are paying for access to Google's top AI plan are intending to use this. Do you have any examples of immediate use-cases that might benefit? Is Google using this tool internally? One would expect them to give some examples of how it's helping internal teams accelerate or solve more challenging problems, if they were eating their own dogfood.
- meta_ai_x 1y agoI'm guessing most Ultra users are there for Veo 3, where it has monetary benefits if your 3s video go viral on TikTok/Reels/Shorts
- irthomasthomas 1y agoYou can spin up a version of this at home using simonw's LLM cli with the llm-consortium plugin. Bonus 1: Use any combination of models. Mix n match models from any lab. Bonus 2: Serve your custom consortium on a local API from a single command using the llm-model-gateway plugin and use it in your apps and coding assistants. https://x.com/karpathy/status/1870692546969735361 https://x.com/karpathy/status/1870692546969735361 > uv tool install llm llm install llm-consortium llm consortium save gthink-n5 -m gemini-pro -n 5 --arbiter gemini-flash --confidence-threshold 99 --max-iterations 4 llm serve --host 0.0.0.0 curl http://0.0.0.0:8000/v1/chat/completions \ -X POST \ -H "Content-Type: application/json" \ -d '{ "model": "gthink-n5", "messages": [{"role": "user", "content": "find a polynomial algorithm for graph-isomorphism"}] }' You can also build a consortium of consortiums like so: llm consortium save gem-squared -m gthink-n5 -n 2 --arbiter gem-flash Or even make the arbiter a consortium: llm consortium save gem-cubed -m gthink-n5 -n 2 --arbiter gthink-n5 --max-iteration 2 or go openweights only: llm consortium save open-council -m qwen3:2 -m kimi-k2:2 -m glm-4.5:2 -m mistral:2 --arbiter minimax-m1 --min-iterations 2 --confidence-threshold 95 https://GitHub.com/irthomasthomas/llm-consortium https://GitHub.com/irthomasthomas/llm-consortium
- barapa 1y agoI am not seeing this llm serve command
- irthomasthomas 1y agoit's a separate plugin rn. llm install llm-model-gateway
- mt_ 1y agoIs the European Union a consortium of consortiums?
- SubiculumCode 1y ago1. Why do you say this is a version of Gemini deep think? It seems like there could be multiple ways to build a multiagent model to explore a space. 2. The covariance between models leads to correlated errors, lowering the individual effectiveness of each contributing model. It would seem to me that you'd want to find a set of model architectures/prompt_congigs that minimizes covariance while maintaining individual accuracy, on a benchmark set of problems that have multiple provable solutions (i.e. not one path to a solution that is objectively correct).
- nickandbro 1y agoLadies and Gentlemen, Here's Gemini Deep Think when prompted with: "Create a svg of a pelican riding on a bicycle" https://www.svgviewer.dev/s/5R5iTexQ https://www.svgviewer.dev/s/5R5iTexQ Beat Simon Willison to it :)
- baggachipz 1y agoTruly worth the price. We live in the future.
- jjcm 1y agoHonestly the first one where I would have guessed "this is a pelican riding a bicycle" if presented with just the image and 0 other context. This and the voxel tower are fairly impressive - we're seeing some semblance of visual / spatial understanding with this model.
- criemen 1y agoHow much would it cost at list-price API pricing?
- arnaudsm 1y agoMeme benchmarks like this and Strawberry are funny but very easy to game, I bet they're all over training sets nowadays.
- simonw 1y agoIf you train a model on the SVGs of pelicans on a bicycle that are out there already you're going to get a VERY weird looking pelican on a bicycle: https://simonwillison.net/tags/pelican-riding-a-bicycle/ https://simonwillison.net/tags/pelican-riding-a-bicycle/
- nickandbro 1y agoThanks for the mention on your blog. You’re the original GPT pelican artist
- 1y ago
- TechDebtDevin 1y ago[flagged]
- dang 1y agoYour comments were being flagged by users, for what should be obvious reasons.
- judge123 1y agoI'm wondering if 'slow AI' like this is a temporary bridge, or a whole new category we need to get used to. Is the future really about having these specialized 'deep thinkers' alongside our fast, everyday models? Or is this just a clunky V1 until the main models get this powerful on their own in seconds?
- onlyrealcuzzo 1y agoIt's not unreasonable to think that with improvements on the software side - a Saturn-like model based on diffusion could be this powerful within a decade - with 1s responses. I'd highly doubt in 10 years, people are waiting 30m for answers of this quality - either due to the software side, the hardware side, and/or scaling. It's possible in 10 years, the cost you pay is still comparable, but I doubt the time will be 30m. It's also possible that there's still top-tier models like this that use absurd amounts of resources (by today's standards) and take 30m - but they'd likely be at a much higher quality than today's.
- regularfry 1y agoThe pressure in the other direction is tool use. The more a model wants to call out to a series of tools, the more the delay will be, just because it the serial process isn't part of the model.
- twobitshifter 1y agowe’re optimizing for quality over performance right now, at some point the pendulum will swing the other way, but there might be problems that require deep thinking just like we have a need for supercomputers to run jobs for days today.
- nusl 1y agoBeen using Gemini for a few months, somehow it's gotten much, much worse in that time. Hallucinations are very common, and it will argue with you when you point it out. So, don't have much confidence.
- panarky 1y agoIn my experience with chat, Flash has gotten much, much better. It's my go-to model even though I'm paying for Pro. Pro is frustrating because it too often won't search to find current information, and just gives stale results from before its training cutoff. Flash doesn't do this much anymore. For coding I use Pro in Gemini CLI. It is amazing at coding, but I'm actually using it more to write design docs, decomp multi-week assignments down to daily and hourly tasks, and then feed those docs back to Gemini CLI to have it work through each task sequentially. With a little structure like this, it can basically write its own context.
- declan_roberts 1y agoI like flash because when it's wrong it's wrong very quickly. You can either change the prompt or just solve the problem yourself. It works well for people who can spot the answer as being "wrong"
- okdood64 1y ago> Flash has gotten much, much better. It's my go-to model even though I'm paying for Pro. Same I think also Pro got worse...
- vicnov 1y agointeresting out of all "thinking models," I struggle with Gemini the most for coding. Just can't make it perform. I feel like they silently nerfed it over the last months.
- nusl 1y agoIt does feel worse. I've swapped to Claude and it's massively better for my tasks. Jules just released so I'll see if that's useful.
- AtlanticThird 1y agoUpgraded and quickly hit my limit. And find that they have limits, I just wish that they were more transparent. Even if it's just a vague statement about limited usage. I assumed it would be similar to regular Gemini 2.5 on the pro plan but it's not
- unsupp0rted 1y agoInteresting they compared Gemini 2.5 Deep Think on code to all the top models EXCEPT Claude, the best top model at code.
- greybox 1y ago[flagged]
- cheema33 1y ago> Can AI suck my balls? Not quite yet. But if you live long enough an AI robot will grant you these and other similar wishes.
- childintime 1y agoThis comes at a time where my experience with Gemini is lacking, it seems to get worse. It's not picking up on my intention, sometimes replies in the wrong language, etc. Either that or I am just transparent that it's a tool and its feelings are hurt. I've had to call it a moron several times, and it was funny when it started reprimanding me for my foul language once. But it was wrong. This behavior seems new. I could never trust it to not do random edits everywhere in a document, so nowadays I use it to check Claude, which can be trusted with a document.
- declan_roberts 1y agoI had a good experience with gemini-cli (which I think uses pro initially?). It's not very good, but it's very fast. So when it's wrong it's wrong very quickly and you can either solve it yourself or pivot your prompt. For a professional software engineer this actually works out OK.
- kelvinjps10 1y agoThese AI names are getting ridiculous
- jawerty 1y agoIf anyone wants a free to use deep research agent check out what i'm working on: https://projectrex.onrender.com/ https://projectrex.onrender.com/
- 44za12 1y agoIs it something like tree of thought reasoning?
- jawerty 1y agoIf anyone wants a free to use research agent try rex I've been building it for a couple months: https://projectrex.onrender.com/ https://projectrex.onrender.com/
- tdhz77 1y agoI think at this point its fair to say that I've switched modals more than Leonardo Dicaprio.
- mclau157 1y agoAsking AI to create 3D scenes like the example in the page seems like asking someone to hammer something with a screwdriver, we would need an AI compatible 3D software that either has easier to use voxels built in so it can create similar to pixel art, or easier math defined curves that can be meshed, either way AI just does not currently have the right tools to generate 3D scenes
- mhh__ 1y agoOne of the most alluring things about LLMs though is that it is like having a screwdriver that can just about work like a hammer, and draft an email to your landlord and so on
- depluber 1y agoIs this a joke? I am paying $250/month because I want to use Deep Think and I literally just burned through my daily quota in 5 PROMPTS. That is insane!
- srameshc 1y agoI've been using Gemini 2.5 Pro for a few months now and have found my experience to be very positive. I primarily use it for coding and through the API, and I feel it's consistently improving. I haven't yet tried Deep Think, though.
- km3k 1y agoThey missed an oppportunity to name it Deep Thought.
- rasmul 1y agowould have made possible a class of jokes: - i wonder how many iterations we need with it before succeeding - i ran it 5 minutes and the deep thought model started to hallucinate, i hope not because oxygen deprivation ...
- motbus3 1y agoI wonder how long it will take for someone starts o investigating the relation between SEO and Google AI crawler
- Invictus0 1y agoI used to be enthusiastic about Gemini-2.5-Pro but now I can't even get it to do decent file-diff summaries for a PR commit.
- qnleigh 1y agoCan someone with an ultra subscription ask it how many rs are in the word strawberry? And report back how long it takes to answer?