22 ms·
Gemini 2.0: our new AI model for the agentic era
- sachou 2y agoWhat is the main difference compare to before?
- transcriptase 2y ago“Hey google turn on kitchen lights” “Sure, playing don’t fear the reaper on bathroom speaker” Ok
- famouswaffles 2y agoThe Gemini 2 models support native audio and image generation but the latter won't be generally available till January. Really excited for that as well as 4o's image generation (whenever that comes out). Steerability has lagged behind aesthetics in image generation for a while now and it's be great to see a big advance in that. Also a whole lot of computer vision tasks (via LLMs) could be unlocked with this. Think Inpainting, Style Transfer, Text Editing in the wild, Segmentation, Edge detection etc They have a demo: https://www.youtube.com/watch?v=7RqFLp0TqV0 https://www.youtube.com/watch?v=7RqFLp0TqV0
- jncfhnb 2y agoThese are not computer vision tasks…
- Jabrov 2y agoWhat are they, then…?
- 85392_school 2y agoThe first two are tasks which involve making images. They could be called image generation or image editing.
- newfocogi 2y agoMaybe some of these tasks are arguably not aligned with the traditional applications of CV, but Segmentation and Edge detection are definitely computer vision in every definition I've come across - before and after NNs took over.
- kthartic 2y agoI asked Gemini 2.0 Flash (with my voice) whether it natively understands audio or is converting my voice to text. It replied: "That's an insightful question. My understanding of your speech involves a pipeline first. Your voice is converted to text and then I process the text to understand what you're saying. So I don't understand your voice directly but rather through a text representation of it." Unsure if this is a hallucination, but is disappointing if true. Edit: Looking at the video you linked, they say "native audio output", so I assume this means the input isn't native? :(
- famouswaffles 2y agoNative audio output won't be in general availability until early next year. If you're using Gemini in aistudio(not sure about the real-time API but everything else) then it has native audio input
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- jerpint 2y agoWe’re definitely going to need better benchmarks for agentic tasks, and not just code reasoning. Things that are needlessly painful that humans go through all the time
- AuthConnectFail 2y agoit's insane on lmarena for a size, livebench should have it soon too I guess
- maeil 2y agoThe size isn't stated, not necessarily a given that it's as small as 1.5-Flash.
- bradhilton 2y agoBeats Gemini 1.5 Pro at all but two of the listed benchmarks. Google DeepMind is starting to get their bearings in the LLM era. These are the minds behind AlphaGo/Zero/Fold. They control their own hardware destiny with TPUs. Bullish.
- p1esk 2y agoAre these benchmarks still meaningful?
- maeil 2y agoNo, and they haven't been for at least half a year. Utterly optimized for by the providers. Nowadays if a model would be SotA for general use but not #1 on any of these benchmarks, I doubt they'd even release it.
- CamperBob2 2y agoI've started keeping an eye out for original brainteasers, just for that reason. GCHQ's Christmas puzzle just came out [1], and o1-pro got 6 out of 7 of them right. It took about 20 minutes in total. I wasn't going to bother trying those because I was pretty sure it wouldn't get any of them, but decided to give it an easy one (#4) and was impressed at the CoT. Meanwhile, Google's newest 2.0 Flash model went 0 for 7. 1: https://metro.co.uk/2024/12/11/gchq-christmas-puzzle-2024-released-can-solve-22161098 https://metro.co.uk/2024/12/11/gchq-christmas-puzzle-2024-re...
- nrvn 2y agoDid it get the 8 right? The linked article provides the wrong answer btw.
- CamperBob2 2y agoI didn't see a straightforward way to submit the final problem, because I used different contexts for each of the 7 subproblems. Given the right prompt, though, I'm sure it could handle the 'find the corresponding letter from the landmarks to form an anagram' part. That's easier than most of the other problems. You're saying the ultimate answer isn't 'PROTECTING THE UNITED KINGDOM'?
- echelon 2y agoGemini in search is answering so many of my search questions wrong. If I ask natural language yes/no questions, Gemini sometimes tells me outright lies with confidence. It also presents information as authoritative - locations, science facts, corporate ownership, geography - even when it's pure hallucination. Right at the top of Google search. edit: I can't find the most obnoxious offending queries, but here was one I performed today: "how many islands does georgia have?". Compare that with "how many islands does georgia have? Skidaway Island". This is an extremely mild case, but I've seen some wildly wrong results, where Google has claimed companies were founded in the wrong states, that towns were located in the wrong states, etc.
- sib301 2y agoThis has happened to me zero times. :shrug:
- airstrike 2y agoDoesn't match my experience. It also feels like it's getting better over time.
- nilayj 2y agocan you provide some example queries that Gemini in search gets wrong?
- echelon 2y agoI just found another one: > A depsipeptide is a cyclic peptide where one or more amide groups are replaced by ester groups. Depsipeptides are not necessarily cyclic, and I'd probably use "bond" instead of "group". https://imgur.com/a/YslvJO2 https://imgur.com/a/YslvJO2 These errors are happening all the time.
- nice__two 2y agoGemini 1.5 indeed is a lot of hit-and-miss. Also, the politically correct and medical info filtering is limiting its usefulness a lot, IMHO. I also miss that it’s not yet really as context aware as ChatGPTo4. Even just asking a follow-up question, confuses Gemini 1.5. Hope Gemini 2.0 will improve that!
- zb3 2y agoIs this the gemini-exp model on LMArena?
- jasonjmcghee 2y agoBoth are available on aistudio so I don't think so. In my own testing "exp 1206" is significantly better than Gemini 2. Feels like haiku 3.5 vs sonnet 3.5 kind of thing.
- warkdarrior 2y agoYes, LMArena shows Gemini-2.0-Flash-Exp ranking 3rd right now, after Gemini-Exp-1206 and ChatGPT-4o-latest_(2024-11-20), and ahead of o1-preview and o1-mini: https://lmarena.ai/?leaderboard https://lmarena.ai/?leaderboard
- zb3 2y agoThere's also the "gremlin" model (not reachable directly) and it seems to be pretty smart.. maybe that's the deep research mode? EDIT: probably not deep research.. is it Google testing their equivalent of o1? who knows..
- usaar333 2y agoIt looks like gemini-exp-1121 slightly upgraded. 1206 is something else.
- crowcroft 2y agoBig companies can be slow to pivot, and Google has been famously bad at getting people aligned and driving in one direction. But, once they do get moving in the right direction the can achieve things that smaller companies can't. Google has an insane amount of talent in this space, and seems to be getting the right results from that now. Remains to be seen how well they will be able to productize and market, but hard to deny that their LLM models aren't really, really good though.
- manishsharan 2y ago>> hard to deny that their LLM models aren't really, really good though. The context window of Gemini 1.5 pro is incredibly large and it retains the memory of things in the middle of the window well. It is quite a game changer for RAG applications.
- KaoruAoiShiho 2y agoIt looks like long context degraded from 1.5 to 2.0 according to the 2.0 launch benchmarks.
- verdverm 2y agoThose benchmarks are for the Flash 2.0 model, showing it improved over the 1.5 Pro model minus for that one benchmark
- caeril 2y agoBear in mind that a "1 million token" context window isn't actually that. You're being sold a sparse attention model, which is guaranteed to drop critical context. Google TPUs aren't running inference on a TERABYTE of fp8 query-key inputs, let alone TWO of fp16. Google's marketing wins again, I guess.
- pelorat 2y agoWell, compared to github copilot (paid), I think Gemini Free is actually better at writing non-archaic code.
- jncfhnb 2y agoAm I alone in thinking the word “agentic” is dumb as shit? Most of these things seem to just be a system prompt and a tool that get invoked as part of a pipeline. They’re hardly “agents”. They’re modules.
- thomassmith65 2y agoIt's easier for consultants and sales people to sell to enterprise if the terminology is familiar but mysterious. Bad 1. installed Antivirus software 2. added screen-size CSS rules 3. copied 'Assets' harddrive to DropBox 4. edited homepage to include Bitcoin wallet address link 5. upgraded to ChatGPT Pro "Good" 1. Cyber-security defenses 2. Responsive Design implementation 3. Cloud Storage 4. Blockchain Technology gateway 5. Agentic enhancements
- endorphine 2y agoGeorge Carlin's "Euphimisms" comes to mind.
- kthartic 2y agoThese are great examples. Also an excellent technique for relaying project updates to non-technical managers
- xnx 2y agoControlling a browser in Project Mariner seems very agentic: https://youtu.be/Fs0t6SdODd8?t=86 https://youtu.be/Fs0t6SdODd8?t=86
- uludag 2y agoDefinitely not alone. With all the this money at stake, coining dumb terms like this might make you a pretty penny. It's like a meme that can be milked for monetization.
- Agentus 2y agoThe beauty of LLMs isn’t just these coding objects speak human vernacular but they can be concatenated with human vernacular prompts and that itself can be used as an input, command or output sensibly without necessarily causing error even if a series of inputs combinations weren't preprogrammed. I have an A.I. textbook that has agent terminology that was written preLLm days. agents are just autonomous ish code that loops on itself with some extra functionality. LLMs in their elegance can more easily out the box selfloop just on the basis concatenating language prompts, sensibly. They are almost agent ready out the box by this very elegant quality(the textbook agentic diagram is just a conceptual self perpetuation loop), except… Except they fail at a lot or get stuck at hiccups. But, here is a novel thought. What if an LLM becomes more agentic (ie more able to sustain autonomous chain prompts that do actions without a terminal failure) and less copilotee not by more complex controlling wrapper self perpetuation code, but by means of training the core llm itself to more fluidly function in agentic scenarios. a better agentically performing llm that isnt mislabeled with a bad buzzword might not reveal itself in its wrapper control code but through it just performing better in an typical agentic loop or environment conditions with whatever initiating prompt, control wrapper code, or pipeline that initiates its self perpetuation cycle.
- brokensegue 2y agoAny word on price? I can't find it at https://ai.google.dev/pricing https://ai.google.dev/pricing
- Oras 2y ago£18/month https://gemini.google/advanced/?Btc=web&Atc=owned&ztc=gemini_lp https://gemini.google/advanced/?Btc=web&Atc=owned&ztc=gemini... then sign in with Google account and you'll see it
- brokensegue 2y agoOh but I only care about api pricing
- xnx 2y agoI think it is free for 1500 requests/day. See the model dropdown on https://aistudio.google.com/prompts/new_chat https://aistudio.google.com/prompts/new_chat
- gman83 2y agoI've been using Gemini Flash for free through the API using Cline for VS Code. I switch between Claude and Gemini Flash, using Claude for more complicated tasks. Hope that the 2.0 model comes closer to Claude for coding.
- SV_BubbleTime 2y agoOr… just continue using Claude?
- 85392_school 2y agoI think they try to conserve costs by only using Claude when needed.
- IAmGraydon 2y ago
- airstrike 2y agoOT: I’m not entirely sure why, but "agentic" sets my teeth on edge. I don't mind the concept, but the word itself has that hollow, buzzwordy flavor I associate with overblown LinkedIn jargon, particularly as it is not actually in the dictionary...unlike perfectly serviceable entries such as "versatile", "multifaceted" or "autonomous"
- deleted 2y ago[deleted]
- geodel 2y agoHuh, all three words you mentioned as replacement are equally buzzwordy and I see them a lot in CVs while screen candidates for job interview.
- airstrike 2y agoAt least all three of them are actually in the dictionary
- hombre_fatal 2y agoThat's not necessarily a good thing because they are overloaded while novel jargon is specific. We use new words so often that we take it for granted. You've passively picked up dozens of new words over the last 5 or 10 years without questioning them.
- raincole 2y agoVersatile implies it can to more kinds of tasks (than it's predecessor or competitor). Agentic implies it requires less human intervention. I don't think these are necessary buzzwords if the product really does what they imply.
- lolinder 2y agoThey agree—they're saying that at least those buzzwords are in the dictionary, not that they'd be a good replacement for "agentic".
- geodel 2y agoJust searched for GVP vs SVP and got: "GVP stands for Good Pharmacovigilance Practice, which is a set of guidelines for monitoring the safety of drugs. SVP stands for Senior Vice President, which is a role in a company that focuses on a specific area of operations." Seems lot of pharma regulation in my telecom company.
- PaulWaldman 2y agoAnecdotally, using the Gemini App with "Gemini Advanced 2.0 Flash Experimental", the response quality is ignorantly improved and faster at some basic Python and C# generation.
- xnx 2y ago> ignorantly improved autocorrect of "significantly improved"?
- oldpersonintx 2y agothe demos are amazing I need to rewire my brain for the power of these tools this plus the quantum stuff...Google is on a win streak
- SubiculumCode 2y agoConsidering so many of us would like more vRAM than NVIDIA is giving us for home compute, is there any future where these Trillium TPUs become commodity hardware?
- geodel 2y agoSo many of us are probably in thousands they need to be 3 order magnitude higher before Google can even think of it.
- kajecounterhack 2y agoPower concerns aside, individual chips in a TPU pod don't actually have a ton of vRAM; they rely on fast interconnects between a lot of chips to aggregate vRAM and then rely on pipeline / tensor parallelism. It doesn't make sense to try to sell the hardware -- it's operationally expensive. By keeping it in house Google only has to support the OS/hardware in their datacenter and they can and do commercialize through hosted services. Why do you want the hardware vs just using it in the cloud? If you're training huge models you probably don't also keep all your data on prem, but on GCS or S3 right? It'd be more efficient to use training resources close to your data. I guess inference on huge models? Still isn't just using a hosted API simpler / what everyone is doing now?
- deleted 2y ago[deleted]
- katamari-damacy 2y agoIs it better than GPT4o? Does it have an API?
- jerrygenser 2y agoAPI is accessible via Vertex AI on Google Cloud in preview. I think it's also available in the consumer Gemini Chat.
- nmfisher 2y agoAlso FYI for anyone else, I can't actually see the gemini-2.0-flash-exp list in the model directory for Vertex AI in GCP Console, but passing in google/gemini-2.0-flash-exp to the vertexai Python SDK does actually work. (you'll probably also need to increase your quotas right away, the default is only 10 requests per minute).
- kgwgk 2y agohttps://ai.google.dev/gemini-api/docs/models/gemini-v2 https://ai.google.dev/gemini-api/docs/models/gemini-v2
- chrsw 2y agoInstead of throwing up tables of benchmarks just let me try to do stuff and see if it's useful.
- topicseed 2y agoIs it on AI studio already?
- siliconc0w 2y agoWhat's everyone's favorite LLM leaderboard? Gemini 2 seems to be edging out 4o on chatbot arena(https://lmarena.ai/?leaderboard https://lmarena.ai/?leaderboard)
- SV_BubbleTime 2y agoAI benchmarks and leaderboards are complete nonsense though. Find something you like, use it, be ready to look again in a month or two.
- falcor84 2y agoWith the accelerating progress, the "be ready to look again" is becoming a full time job that we need to be able to delegate in some way, and I haven't found anything better than benchmarks, leaderboards and reviews. EDIT: Typo
- siliconc0w 2y agoFWIW I've found the 'coding' 'category' of the leaderboard to be reasonably accurate. Claude was the best, o1-mini then was typically stronger, now the Gemini Exp 1206 is at the top. I find myself just paying a la carte via the API rather than paying the $20/mo so I can switch between the models.
- hombre_fatal 2y agopoe.com has a decent model where you buy credits and spend them talking to any LLM which makes it nice to swap between them even during the same conversation instead of paying for multiple subscriptions. Though gpt-4o could say "David Mayer" on poe.com but not on chat.openai.com which makes me wonder if they sometimes cheat and sneak in different models.
- manishsharan 2y agoLeaderboards are not that useful for measuring real-life effectiveness of the models atleast in my day-today usage. I am currently struggling to diagnose an ipv6 mis-configuration in my enormous aws cloudformation yaml code. I gave the same input to Claude Opus, Gemini and ChatGPT ( o1 and 4o). 4o was the worst. verbose and waste of my time. Claude completely went off-tangent and began recommending fixes for ipv4 while I specifically asked for ipv6 issues o1 made a suggestion which I tried out and it fixed it. It literally found a needle in the haystack. The solution is working well now. Gemini made a suggestion which almost got it right but it was not a full solution. I must clarify diagnosing network issues on AWS VPC is not my expertise and I use the LLMs to supplement my knowledge.
- dangoodmanUT 2y agoJules looks like it's going after Devin
- m3kw9 2y agoClaude MCP does the same thing. It’s the setup that is hard. It will do push pull create branch automatically from a single prompt. 500$ a month for Devin could be worth it if you want it taken care off plus use the models for a team, but a single person can set it up
- moralestapia 2y ago>2,000 words of bs >General availability will follow in January, along with more model sizes. >Benchmarks against their own models which always underperformed >No pricing visible anywhere Completely inept leadership at play.
- thisoneworks 2y ago"gemini for video games" - here we go again with the AI does the interesting stuff for you rather than the boring stuff
- EternalFury 2y agoThink of Google as of a tanker ship. It takes a while to change course, but it has great momentum. Sundar just needs to make sure the course is right.
- griomnib 2y agoAnd where is the ship headed if they are no longer supporting the open web? Publishers are being squeezed and going under, or replacing humans with hallucinated genai slop. It’s like we’re taking the private equity model of extracting value and killing something off to the entire web. I’m not sure where this is headed, but I don’t think Sundar has any strategy here other than playing catch up. Demis’ goal is pretty transparently positioning himself to take over.
- EternalFury 2y agoThe Web is dead. It’s pretty clear future web pages, if we call them that, will be assembled on-the-fly by AI based your user profile and declared goals, interests and requests.
- CSMastermind 2y agoThat's almost word for word what people said about Windows Phone when I was at Microsoft.
- atorodius 2y agoWas the Windows Phone ever at the frontier tho?
- scarmig 2y agoWindows Phone was superior to everything else on the market at the time. But phones are an ecosystem, and MS was a latecomer.
- blinding-streak 2y ago
- gotaran 2y agoGoogle beat OpenAI at their own game.
- wonderfuly 2y agoChat now: https://app.chathub.gg/chat/cloud-gemini-2.0-flash https://app.chathub.gg/chat/cloud-gemini-2.0-flash
- m3kw9 2y agoCan these guy lead for once? They are always responding to what OpenAI is doing.
- losvedir 2y agoThis naming is confusing... Anyway, I'm glad that this Google release is actually available right away! I pay for Gemini Advanced and I see "Gemini Flash 2.0" as an option in the model selector. I've been going through Advent of Code this year, and testing each problem with each model (GPT-4o, o1, o1 Pro, Claude Sonnet, Opus, Gemini Pro 1.5). Gemini has done decent, but is probably the weakest of the bunch. It failed (unexpectedly to me) on Day 10, but when I tried Flash 2.0 it got it! So at least in that one benchmark, the new Flash 2.0 edged out Pro 1.5. I look forward to seeing how it handles upcoming problems! I should say: Gemini Flash didn't quite get it out of the box. It actually had a syntax error in the for loop, which caused it to fail to compile, which is an unusual failure mode for these models. Maybe it was a different version of Java or something (I'm also trying to learn Java with AoC this year...). But when I gave Flash 2.0 the compilation error, it did fix it. For the more Java proficient, can someone explain why it may have provided this code: for (int[] current = queue.remove(0)) { which was a compilation error for me? The corrected code it gave me afterwards was just for (int[] current : queue) { and with that one change the class ran and gave the right solution.
- ianmcgowan 2y agoA tangent, but is there a clear best choice amongst those models for AOC type questions?
- notamy 2y ago> For the more Java proficient, can someone explain why it may have provided this code: To me that reads like it was trying to accomplish something like int[] current; while((current = queue.pop()) != null) {
- srameshc 2y agoI use a Claude and Gemini a lot for coding and I realized there is no good or best model. Every model has it's upside and downside. I was trying to get authentication working according to the newer guidelines of Manifest V3 for browser extensions and every model is terrible. It is one use case where there is not much information or right documentation so every model makesup stuff. But this is my experience and I don't speak for everyone.
- nuz 2y agoI guess this means we'll have an openai release soon
- nightski 2y agoAnyone else annoyed how the ML/AI community just adopted the word "reasoning" when it seems like it is being used very out of context when looking at what the model actually does?
- ramoz 2y agoThese models take an instruction, along with any contextual information, and are trained to produce valid output. That production of output is a form of reasoning via _some_ type of logical processing. No? Maybe better to say computational reasoning. That’s a mouthful.
- nightski 2y agoStatic computation is not reasoning (these models are not building up an argument from premises, they are merely finding statistically likely completions). Computational thinking/reasoning would be breaking down a problem into an algorithmic steps. The model is doing neither. I wouldn't confuse the fact that it can break it into steps if you ask it, because again that is just regurgitation. It's not going through that process without your prompt. That is not part of its process to arrive at an answer.
- ramoz 2y agoThe point is emergent capabilities in LLMs go beyond statistical extrapolation, as they demonstrate reasoning by combining learned patterns. When asked, “If Alice has 3 apples and gives 2 to Bob, how many does she have left?”, the model doesn’t just retrieve a memorized answer—it infers the logical steps (subtracting 2 from 3) to generate the correct result, showcasing reasoning built on the interplay of its scale and architecture rather than explicit data recall.
- thelastbender12 2y agoI kinda agree with you but I can also see why it isn't that far from "reasoning" in the sense humans do it. To wit, if I am doing a high school geometry proof, I come up with a sequence of steps. If the proof is correct, each step follows logically from the one before it. However, when I go from step 2 to step 3, there are multiple options for step-3 I could have chose. Is it so different from a "most-likely-prediction" an LLM makes? I suppose the difference is humans can filter out logically-incorrect steps, or prune chains-of-steps that won't lead to the actual theorem quicker. But an LLM predictor coupled with a verifier doesn't feel that different from it.
- summerlight 2y agoIt is interesting to see that they keep focusing on the cheapest model instead of the frontier model. Probably because of their primary (internal?) customer's need?
- discobot 2y agothe problem is that last generation of the largest models failed to overcome smaller models on the benchmarks, see lack of new claude opus or gpt-5. The problem is probably in the benchmarks, but anyway.
- coder543 2y agoIt's cheaper and faster to train a small model, which is better for a research team to iterate on, right? If Google decides that a particular small model is really good, why wouldn't they go ahead and release it while they work on scaling up that work to train the larger versions of the model?
- summerlight 2y agoI have no knowledge of Google specific cases, but in many teams smaller models are trained upon bigger frontier models through distillation. So the frontier models come first then smaller models later.
- coder543 2y agoTraining a "frontier model" without testing the architecture is very risky. Meta trained the smaller Llama 3 models first, and then trained the 405B model on the same architecture once it had been validated on the smaller ones. Later, they went back and used that 405B model to improve the smaller models for the Llama 3.1 release. Mistral started with a number of small models before scaling up to larger models. I feel like this is a fairly common pattern. If Google had a bigger version of Gemini 2.0 ready to go, I feel confident they would have mentioned it, and it would be difficult to distill it down to a small model if it wasn't ready to go.
- int_19h 2y ago
- xyst 2y agoSide note on Gemini: I pay for Google Workspace simply to enable e-mail capability for a custom domain. I never used the web interface to access email until recently. To my surprise, all of the AI shit is enabled by default. So it’s very likely Gemini has been training on private data without my explicit consent. Of course G words it as “personalizing” the experience for me but it’s such a load of shit. I’m tired of these companies stealing our data and never getting rightly compensated.
- hnuser123456 2y agoGmail is hosting your email. Being able to change the domain doesn't change that they're hosting it on their terms. I think there are other email providers that have more privacy-focused policies.
- dandiep 2y agoGemini multimodal live docs here: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/multimodal-live https://cloud.google.com/vertex-ai/generative-ai/docs/model-... A little thin... Also no pricing is live yet. OpenAI's audio inputs/outputs are too expensive to really put in production, so hopefully Gemini will be cheaper. (Not to mention, OAI's doesn't follow instructions very well.)
- kwindla 2y agoThe Multimodal Live API is free while the model/API is in preview. My guess is that they will be pretty aggressive with pricing when it's in GA, given the 1.5 Flash multimodal pricing. If you're interested in this stuff, here's a full chat app for the new Gemini 2 API's with text, audio, image, camera video and screen video. This shows how to use both the WebSocket API and to route through WebRTC infrastructure. https://github.com/pipecat-ai/gemini-multimodal-live-demo https://github.com/pipecat-ai/gemini-multimodal-live-demo
- dandiep 2y agoThanks, this is great!
- spencerchubb 2y agoI am eager to learn the pricing as well. It works sooo well but the pricing will make or break whether it's viable for apps
- AJRF 2y agoI think they are really overloading that word "Agent". I know there isn't a standard definition - but I think Google are stretching the meaning of that way thinner than most C Suite level execs talk about agents at. I think DeepMind could make progress if they focused on the agent definition of multi-step reasoning + action through a web browser, and deliver a ton of value, outside of lumping in the seldom used "Look at the world through a camera" or "Multi modal Robots" thing. If Google cracked robots, past plays show that the market for those aren't big enough to interest Google. Like VR, you just can't get a billion people to be interested in robots - so even if they make progress, it won't survive under Google. The "Look at the world through a camera" thing is a footnote in an Android release. Agentic computer use _is_ a product a billion people would use, and it's adjacent to the business interests of Google Search.
- jstummbillig 2y agoReminder that implied models are not actual models. Models have failed to materialize repeatedly and vanished without further mention. I assume no one is trying to be misleading but, at this point, maybe overly optimistic.
- nycdatasci 2y agoGemini 2.0 Flash is available here: https://aistudio.google.com/prompts/new_chat https://aistudio.google.com/prompts/new_chat Based on initial interactions, it's extremely verbose. It seems to be focused on explaining its reasoning, but even after just a few interactions I have seen some surprising hallucinations. For example, to assess current understanding of AI, I mentioned "Why hasn't Anthropic released Claude 3.5 Opus yet?" Gemini responded with text that included "Why haven't they released Claude 3.5 Sonnet First? That's an interesting point." There's clearly some reflection/attempted reasoning happening, but it doesn't feel competitive with o1 or the new Claude 3.5 Sonnet that was trained on 3.5 Opus output.
- snthpy 2y agoJust speculating here but the verbose reasoning might be to make the "agentic" part work better as per the headline, ie pass context along to other agents. Would be good to get concise responses for human interactions though.
- IanCal 2y agoI'm having it lie a lot too. It wrote some pretty decent attempt at code in a new framework given instructions, which is great, but it made a key error about That's fine, but it couldn't spot it, then it told me that I had put that syntax in the instructions and quoted me - but that wasn't true. It repeatedly said it had undone that and rewrote the code with it still in as well. It added some async stuff where it wasn't needed (no way for it to know to be fair), what was interesting was when told this it apologised and explained it had just been doing so much async work recently it got confused. Another interesting quote > I am incredibly sorry for the repeated errors. I have made so many mistakes, and I am very frustrated with myself. I believe this is now correct, and adheres to the instructions.
- simonw 2y agoI released a new llm-gemini plugin with support for the Gemini 2.0 Flash model, here's how to use that in the terminal: llm install -U llm-gemini llm -m gemini-2.0-flash-exp 'prompt goes here' LLM installation: https://llm.datasette.io/en/stable/setup.html https://llm.datasette.io/en/stable/setup.html Worth noting that the Gemini models have the ability to write and then execute Python code. I tried that like this: llm -m gemini-2.0-flash-exp -o code_execution 1 \ 'write and execute python to generate a 80x40 ascii art fractal' Here's the result: https://gist.github.com/simonw/0d8225d62e8d87ce843fde471d1432cc https://gist.github.com/simonw/0d8225d62e8d87ce843fde471d143... It can't make outbound network calls though, so this fails: llm -m gemini-2.0-flash-exp -o code_execution 1 \ 'write python code to retrieve https://simonwillison.net/ and use a regex to extract the title, run that code' Amusingly Gemini itself doesn't know that it can't make network calls, so it tries several different approaches before giving up: https://gist.github.com/simonw/2ccfdc68290b5ced24e5e0909563c62a https://gist.github.com/simonw/2ccfdc68290b5ced24e5e0909563c... The new model seems very good at vision: llm -m gemini-2.0-flash-exp describe -a https://static.simonwillison.net/static/2024/pelicans.jpg I got back a solid description, see here: https://gist.github.com/simonw/32172b6f8bcf8e55e489f10979f8f085 https://gist.github.com/simonw/32172b6f8bcf8e55e489f10979f8f...
- rafram 2y ago> Some pelicans have white on their heads, suggesting that some of them are older birds. Interesting theory!
- smackay 2y agoBrown Pelican (Pelecanus occidentalis) heads are white in the breeding season. Birds start breeding aged three to five. So technically the statement is correct but I wonder if Gemini didn't get its pelicans and cormorants in a muddle. The mainland European Great Cormorant (Phalacrocorax carbo sinensis) has a head that gets progressively whiter as birds age.
- pcwelder 2y agoCode execution is okay, but soon runs into the problem of missing packages that it can't install. Practically, sandboxing hasn't been super important for me. Running claude with mcp based shell access has been working fine for me, as long as you instruct it to use venv, temporary directory, etc.
- melvinmelih 2y agoNo mention of Perplexity yet in the comments but it's obvious to me that they're targeting Perplexity Pro directly with their new Deep Research feature (https://blog.google/products/gemini/google-gemini-deep-research/ https://blog.google/products/gemini/google-gemini-deep-resea...). I still wonder why Perplexity is worth $7 billion when the 800-pound gorilla is pounding on their door (albeit slowly).
- yandie 2y agoJust tried the deep search. It's a much much slower experience than perplexity at the moment - taking sweet many minutes to return result. Maybe it's more extensive but I use perplexity for quick information summary a lot and this is a very different UX. Haven't used it enough to evaluate the quality, however.
- BoorishBears 2y agoBefore dropping it for a different project that got some traction, "Slow Perplexity" was something I was pretty set on building. Perplexity is a much less versatile product than it has to be in the chase of speed: you can only chew through so many tokens, do so much CoT, etc. in a given amount of time. They optimized for virality (it's just as fast as Google but gives me more info!) but I suspect it kills the stickiness for a huge number of users since you end up with some "embarrassing misses": stuff that should have been a slam dunk, goes off the rails due to not enough search, or the wrong context being surfaced from the page, etc. and the user just doesn't see value in it anymore.
- ianbutler 2y agoUnfortunately the 10rpm quota for this experimental model isn't enough to run an actual Agentic experience on. That's my main issue with google there's several models we want to try with our agent but quota is limited and we have to jump through hoops to see if we can get it raised.
- aantix 2y agoTheir offering is just so... bad. Even the new model. All the data in the world, yet they trail behind. They have all of these extensions that they use to prop up the results in the web UI. I was asking for a list of related YouTube videos - the UI returns them. Ask the API the same prompt, it returns a bunch of made up YouTube titles and descriptions. How could I ever rely on this product?
- fuddle 2y agoI'd be interested to see Gemini 2.0's performance on SWE-Bench.
- mherrmann 2y agoTheir Mariner tool for controlling the browser sounds scary and exciting. At the moment, it's an extension, which means JavaScript. Some web sites block automation that happens this way, and developers resort to tools such as Selenium. These use the Chrome DevTools API to automate the browser. It's better, but can still be distinguished from normal use with very technical details. I wonder if Google, who still own Chrome, will give extensions better APIs for automation that can not be distinguished from normal use.
- horyd 2y agoYou would think so, and probably starting the productivity suite (Docs, Sheets, etc.). If Mariner could seamlessly integrate with those, then times are about to get real interesting (if they aren't already!)
- tpoacher 2y agoI know this isn't really a useful comment, but, I'm still sour about the name they chose. They MUST have known about the Gemini protocol. I'm tempted to think it was intentional, even. It's like Microsoft creating an AI tool and calling it Peertube. "Hurr durr they couldn't possibly be confused; one is a decentralised video platform and the other is an AI tool hurr durr. And ours is already more popular if you 'bing' it hurr durr."
- jvolkman 2y ago> It's like Microsoft creating an AI tool and calling it Peertube. How is it like that? Gemini is a much more common word than Peertube. https://en.wikipedia.org/wiki/Gemini https://en.wikipedia.org/wiki/Gemini
- esafak 2y agoI'd never heard of this protocol, and I try to keep up so I can't blame Google.
- petesergeant 2y agoSpeed looks good vis-a-vis 4o-mini, and quality looks good so far against my eval set. If it's cheaper than 4o-mini too (which, it probably will be?) then OpenAI have a real problem, because switching between them is a value in a config file.
- smallerfish 2y agoWas this written by an LLM? It's pretty bad copy. Maybe they laid off their copywriting team...? > "Now millions of developers are building with Gemini. And it’s helping us reimagine all of our products — including all 7 of them with 2 billion users — and to create new ones" and > "We’re getting 2.0 into the hands of developers and trusted testers today. And we’re working quickly to get it into our products, leading with Gemini and Search. Starting today our Gemini 2.0 Flash experimental model will be available to all Gemini users."
- utopcell 2y agoSorry, what's wrong with these phrases?
- echelon 2y agoIt reads like a transcribed speech. You can picture this being read from a teleprompter at a conference keynote. Short sentence fact. And aspirational tagline - pause for some metrics - and more. And. Today. And. And. Today.
- krona 2y ago> all of our products — including all 7 of them All the products including all the products?
- deleted 2y ago[deleted]
- iamdelirium 2y agoWhy did you specifically ignore the remainder of the sentence? "...all of our products — including all 7 of them with 2 billion users..." It tells people that 7 of their products have 2b users.
- fluoridation 2y agoThat's not really any better, since "all of our products" already includes the subset that has at least 2B users. "I brought all my shoes, including all my red shoes."
- ryandvm 2y agoI am sure Google has the resources to compete in this space. What I'm less sure about is whether Google can monetize AI in a way that doesn't cannibalize their advertising income. Who the hell wants an AI that has the personality of a car salesman?
- ipsum2 2y agoTested out Gemini-2 Flash, I had such high hopes that a better base model would help. It still hallucinates like crazy compared to GPT-4o.
- gbickford 2y agoSmall models don't "know" as much so they hallucinate more. They are better suited for generations that are based in a ground truth, like in a RAG setup. A better comparison might be Flash 2.0 vs 4o-mini. Even then, the models aren't meant to have vast world knowledge, so benchmarking them on that isn't a great indicator of how they would be used in real-world cases.
- ipsum2 2y agoYes, it's not an apples to apples comparison. My point is the position it's at on the lmarena leaderboard is misplaced due to the hallucination issues.
- CSMastermind 2y ago> We're also launching a new feature called Deep Research, which uses advanced reasoning and long context capabilities to act as a research assistant, exploring complex topics and compiling reports on your behalf. It's available in Gemini Advanced today. Anyone seeing this? I don't have an option in my dropdown.
- atorodius 2y agoRolling out the next few days accorsing to Jeff
- fudged71 2y agoNot seeing it yet on web or mobile (in Canada)
- jcims 2y agoI just used it (a day later). It's only available on the 1.5 model right now. If anything it found a bunch of cool articles on the subject matter that I could use on my own. The actual results from its research in the domain I was using were much better than the standard chat bot response that 1.5 or 2.0 gave.
- echohack5 2y agoIs this what it feels like to become one of the gray bearded engineers? This sounds like a bunch of intentionally confusing marketing drivel. When capitalism has pilfered everything from the pockets of working people so people are constantly stressed over healthcare and groceries, and there's little left to further the pockets of plutocrats, the only marketing that makes sense is to appeal to other companies in order to raid their coffers by tricking their Directors to buy a nonsensical product. Is that what they mean by "agentic era"? Cause that's what it sounds like to me. Also smells alot like press release driven development where the point is to put a feather in the cap of whatever poor Google engineer is chasing their next promotion.
- weatherlite 2y ago> Is that what they mean by "agentic era"? Cause that's what it sounds like to me. What are you basing your opinion on? I have no idea how well these LLM agents will perform but its definitely a thing. OpenAI is working on them , Claude and certainly Google.
- cush 2y agoYeah it’s a lot of marketing fluff but these tools are genuinely useful and there’s no wonder why Google is working hard to prevent them from destroying their search-dependent bottom line. Marketing aside, agents are just LLMs that can reach out of their regular chat bubbles and use tools. Seems like just the next logical evolution
- serjester 2y agoBuried in the announcement is the real gem — they’re releasing a new SDK that actually looks like it follows modern best practices. Could be a game-changer for usability. They’ve had OpenAI-compatible endpoints for a while, but it’s never been clear how serious they were about supporting them long-term. Nice to see another option showing up. For reference, their main repo (not kidding) recommends setting up a Kubernetes cluster and a GCP bucket to submit batch requests. [1]https://github.com/googleapis/python-genai https://github.com/googleapis/python-genai
- pkkkzip 2y agoits interesting that just as the LLM hype appears to be simmering down, DeepMind is making big strides. I'm more excited by this than any of OpenAI's announcements.
- deleted 2y ago[deleted]
- redrix 2y agoOh wow, it supports directly specifying a Pydantic model as an output schema that it will adhere to for structured JSON output. That’s fantastic! https://github.com/googleapis/python-genai?tab=readme-ov-file#json-response-schema https://github.com/googleapis/python-genai?tab=readme-ov-fil...
- jcheng 2y agoFYI all the LLM Python SDKs that support structured output can use Pydantic for the schema—at least all the ones I can think of.
- mark_l_watson 2y agoI looked carefully at the SDK earlier today - it does look very nice, but it is also a work in progress.
- ComputerGuru 2y agoI've been using gemini-exp-1206 and I notice a lot of similarities to the new gemini-2.0-flash-exp: they're not that much actually smarter but they go out of their way to convince you they are with overly verbose "reasoning" and explanations. The reasoning and explanations aren't necessarily wrong per se, but put them aside and focus on the actual logical reasoning steps and conclusions to your prompts and it's still very much a dumb model. The models do just fine on "work" but are terrible for "thinking". The verbosity of the explanations (and the sheer amount of praise the models like to give the prompter - I've never had my rear end kissed so much!) should lead one to beware any subjective reviews of their performance rather than objective reviews focusing solely on correct/incorrect.
- epolanski 2y agoI'm not gonna lie I like Google's models. Flash combines speed and cost and is extremely good to build apps on. People really take that whole benchmarking thing more seriously than necessary.
- Animats 2y ago"Over the last year, we have been investing in developing more agentic models, meaning they can understand more about the world around you, think multiple steps ahead, and take action on your behalf, with your supervision." "With your supervision". Thus avoiding Google being held responsible. That's like Teslas Fake Self Driving, where the user must have their hands on the wheel at all times.
- sorenjan 2y agoPublished the day after one of the authors, Demis Hassabis, received his Nobel prize in Stockholm.
- beepbooptheory 2y agoWe are moving through eras faster than years these days.
- deleted 2y ago[deleted]
- strongpigeon 2y agoDid anyone get to play with the native image generation part? In my experience, Imagen 3 was much better than the competition so I'm curious to hear people's take on this one.
- strongpigeon 2y agoHrm, when I tried to get it to generate an image it said it was using Imagen 3. Not sure what “native” image generation means then.
- famouswaffles 2y agonative audio and image generation won't be in general availability till early next year
- fpgaminer 2y agoI work with LLMs and MLLMs all day (as part of my work on JoyCaption, an open source VLM). Specifically, I spend a lot of time interacting with multiple models at the same time, so I get the chance to very frequently compare models head-to-head on real tasks. I'll give Flash 2 a try soon, but I gotta say that Google has been doing a great job catching up with Gemini. Both Gemini 1.5 Pro 002 and Flash 1.5 can trade blows with 4o, and are easily ahead of the vast majority of other major models (Mistral Large, Qwen, Llama, etc). Claude is usually better, but has a major flaw (to be discussed later). So, here's my current rankings. I base my rankings on my work, not on benchmarks. I think benchmarks are important and they'll get better in time, but most benchmarks for LLMs and MLLMs are quite bad. 1) 4o and its ilk are far and away the best in terms of accuracy, both for textual tasks as well as vision related tasks. Absolutely nothing comes even close to 4o for vision related tasks. The biggest failing of 4o is that it has the worst instruction following of commercial LLMs, and that instruction following gets _even_ worse when an image is involved. A prime example is when I ask 4o to help edit some text, to change certain words, verbage, etc. No matter how I prompt it, it will often completely re-write the input text to its own style of speaking. It's a really weird failing. It's like their RLHF tuning is hyper focused on keeping it aligned with the "character" of 4o to the point that it injects that character into all its outputs no matter what the user or system instructions state. o1 is a MASSIVE improvement in this regard, and is also really good at inferring things so I don't have to explicitly instruct it on every little detail. I haven't found o1-pro overly useful yet. o1 is basically my daily driver outside of work, even for mundane questions, because it's just better across the board and the speed penalty is negligible. One particularly example of o1 being better I encountered yesterday. I had it re-wording an image description, and thought it had introduced a detail that wasn't in the original description. Well, I was wrong and had accidentally skimmed over that detail in the original. It _told_ me I was wrong, and didn't update the description! Freaky, but really incredible. 4o never corrects me when I give it an explicit instruction. 4o is fairly easy to jailbreak. They've been turning the screws for awhile so it isn't as easy as day 1, but even o1-pro can be jailbroken. 2) Gemini 1.5 Pro 002 (specifically 002) is second best in my books. I'd guesstimate it at being about 80% as good as 4o on most tasks, including vision. But it's _significantly_ better at instruction following. Its RLHF is a lot lighter than ChatGPT models, so it's easier to get these models to fall back to pretraining, which is really helpful for my work specifically. But in general the Gemini models have come a long way. The ability to turn off model censorship is quite nice, though it does still refuse at times. The Flash variation is interesting; often times on-par with Pro with Pro edging out maybe 30% of the time. I don't frequently use Flash, but it's an impressive model for its size. (Side note: The Gemma models are ... not good. Google's other public models, like so400m and OWLv2 are great, so it's a shame their open LLMs forays are falling behind). Google also has the best AI playground. Jailbreaking Gemini is a piece of cake. 3) Claude is third on my list. It has the _best_ instruction following of all the models, even slightly better than o1. Though it often requires multi-turn to get it to fully follow instructions, which is annoying. Its overall prowess as an LLM is somewhere between 4o and Gemini. Vision is about the same as Gemini, except for knowledge based queries which Gemini tends to be quite bad at (who is this person? Where is this? What brand of guitar? etc). But Claude's biggest flaw is the insane "safety" training it underwent, which makes it practically useless. I get false triggers _all_ the time from Claude. And that's to say nothing of how unethical their "ethics" system is to begin with. And what's funny is that Claude is an order of magnitude _smarter_ when its reasoning about its safety training. It's the only real semblance of reason I've seen from LLMs ... all just to deny my requests. I've put Claude three out of respect for the _technical_ achievements of the product, but I think the developers need to take a long look in the mirror and ask why they think it's okay to for _them_ to decide what people with disabilities are and are not aloud to have access to. 4) Llama 3. What a solid model. It's the best open LLM, hands down. Nowhere near the commercial models above, but for a model that's completely free to use locally? That's invaluable. Their vision variation is ... not worth using. But I think it'll get better with time. The 8B variation far outperforms its weight class. 70B is a respectable model, with better instruction following than 4o. The ability to finetune these models to a task with so little data is a huge plus. I've made task specific models with 200-400 examples. 5) Mistral Large (I forget the specific version for their latest release). I love Mistral as the "under-dog". Their models aren't bad, and behave _very_ differently from all other models out there, which I appreciate. But Mistral never puts any effort into polishing their models; they always come out of the oven half-baked. Which means they frequently glitch out, have very inconsistent behavior, etc. Accuracy and quality is hard to assess because of this inconsistency. On its best days it's up near Gemini, which is quite incredible considering the models are also released publicly. So theoretically you could finetune them to your task and get a commercial grade model to run locally. But rarely see anyone do that with Mistral, I think partly because of their weird license. Overall, I like seeing them in the race and hope they get better, but I wouldn't use it for anything serious. Mistral is lightly censored, but fairly easy to jailbreak. 6) Qwen 2 (or 2.5 or whatever the current version is these days). It's an okay model. I've heard a lot of praises for it, but in all my uses thus far its always been really inconsistent, glitchy, and weak. I've used it both locally and through APIs. I guess in _theory_ it's a good model, based on benchmarks. And it's open, which I appreciate. But I've not found any practical use for it. I even tried finetuning with Qwen 2VL 72B, and my tiny 8B JoyCaption model beat it handily. That's about the sum of it. AFAIK that's all the major commercial and open models (my focus is mainly on MLLMs). OpenAI are still leading the pack in my experience. I'm glad to see good competition coming from Google finally. I hope Mistral can polish their models and be a real contender. There are a couple smaller contenders out there like Pixmo/etc from allenai. Allen AI has hands down the _best_ public VQA dataset I've seen, so huge props to them there. Pixmo is ... okayish. I tried Amazon's models a little but didn't see anything useful. NOTE: I refuse to use Grok models for the obvious reasons, so fucks to be them.
- nopcode 2y ago> What can you tell me about this sculpture? > It's located in London. Mind blowing.
- jacooper 2y agoThe best thing about gemini models is the huge context windows, you can just throw big documents and find stuff real fast, rather than struggling with cut off in perplexity or Claude.
- bluelightning2k 2y agoDoes anyone know how to sign up to the speech output wait list or tester program? I have a decent spent with GCP over the years if that helps at all. Really want DemoTime videos to use those voices. (I like how truly incredible best in the world tts is like a footnote in this larger announcement.)
- jgalt212 2y agoCan cloudflare turnstile (and others) detect these agents as bots?
- relatedtitle 2y agoCF Turnstile is just proof of work, not a CAPTCHA. It works on automated browsers from my experience.
- jamesponddotco 2y agoAt least when it comes to Go code, I'm pretty impressed by the results so far. It's also pretty good at following directions, which is a problem I have with open source models, and seems to use or handle Claude's XML output very well. Overall, especially seeing as I haven't paid a dime to use the API yet, I'm pretty impressed.
- mfonda 2y agoDoes anyone have any insights into how Google selects source material for AI overviews? I run an educational site with lots of excellent information, but it seems to have been passed over entirely for AI overviews. With these becoming an increasingly large part of search--and from the sound of it, now more so with Gemini 2.0--this has me a little worried. Anyone else run into similar issues or have any tips?
- nuancebydefault 2y agoI'm quite impressed with the flash demo's reasoning capabilities. Did the 20 questions game with it, it found the computer mouse i had in my head. At first it was confused about our roles and said something weird, it thought that it had to guess its own word. Afterwards I had a meta conversation about that weirdness and it gave impressive insights: " Why My "I Guess What I Have in Mind" Statement Doesn't Make Sense The Problem of Agency: You're spot-on when you say it doesn't make sense for me to state "I guess what I (the computer) have in mind." AI does not have a "mind" in the same way that humans do. My internal state is a complex pattern of numerical data, not conscious thoughts or intentions in the way you have things in mind. My phrasing gave the illusion of having a "thing in mind," but in reality, I don't have this capacity. Misinterpretation of the Task: My statement "I guess what I have in mind" misinterprets the task. You were clearly signaling that I needed to ask questions to uncover what you were thinking of. My failure to grasp this created a weird scenario where it seemed like I was trying to determine my own data set!"
- paradite 2y agoHonestly this post makes Google sound like the new IBM. Very corporate. "Hear from our CEO first, and then our other CEO in charge of this domain and CTO will tell you the actual news." I haven't seen other tech companies write like that.
- ofermend 2y agoGemini-2.0-Flash does extremely well on the Hallucination Evaluation Leaderboard, at 1.3% hallucination rate https://github.com/vectara/hallucination-leaderboard https://github.com/vectara/hallucination-leaderboard
- refulgentis 2y agoFascinating, thanks for calling that out: I found 1.0 promising in practice, but with hallucination problems. Then I saw it had gotten 57% of questions wrong on open book true/false and I wrote it off completely - no reason to switch to it for speed and cost if it's just a random generator. That's a great outcome.
- jug 2y agoSpeaking of which, I wonder how they'd do on SimpleQA. OpenAI is an outlier there in the negative sense vs Anthropic. This benchmark also deals with hallucination and "inappropriate certainty".
- stared 2y agoHow does it compare to OpenAI and Anthropic models, on the same benchmarks?
- anothernewdude 2y agoAgents are the worst idea. I think AI will start to progress again when we get better models that drop this whole chat idea, and just focus on completions. Building the tools on top should be the work that is given over to the masses. It's sad that instead the most powerful models have been hamstrung.
- tkgally 2y agoI tried accessing Gemini 2.0 Flash through Google AI Studio in the Safari browser on my iPhone, and to my surprise it worked. After I gave it access to my microphone and camera, I was able to have a pretty smooth conversation with it about what it saw through the camera. I pointed the camera at things in my room and asked what they were, and it identified them accurately. It was also able to read text in both English and Japanese. It correctly named a note I played on a piano when I showed it the keyboard with my finger playing the note, but it couldn’t identify notes by sound alone. The latency was low, though the conversation got cut off a few times.
- computergert 2y agoIt's funny how even the latest AI models struggle with simple questions. "What's the first name of Freddy LaStrange"? >> "I do not have enough information about that person to help with your request. I am a large language model, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions, but my knowledge about this person is limited. Is there anything else I can do to help you with this request?" (Of course, we can't be 100% sure that his first name is Freddy. But I would expect that to be part of the answer then)
- afro88 2y agoIs o1 not one of the latest models? https://chatgpt.com/share/675ab91c-c158-8004-9dfc-ea176ba38796 https://chatgpt.com/share/675ab91c-c158-8004-9dfc-ea176ba387... A better way to say it is: "funny how Google can't keep up with the reasoning effectiveness of OpenAI's latest models"
- IanCal 2y agoThese kinds of things just strike me as trick questions, why would anyone ask that? Also > Freddy LaStrange is a fictional character and his first name is Freddy. and > Freddy is a nickname for Frederick. So, the first name of Freddy LaStrange is Frederick.
- r33b33 2y agoCan it work with large codebase? How? I have an Xcode project with python server for online game. Is this sufficient to understand code fully and implement edits correctly?
- modeless 2y agohttps://aistudio.google.com/live https://aistudio.google.com/live is by far the coolest thing here. You can just go there and share your screen or camera and have a running live voice conversation with Gemini about anything you're looking at. As much as you want, for free. I just tried having it teach me how to use Blender. It seems like it could actually be super helpful for beginners, as it has decent knowledge of the toolbars and keyboard shortcuts and can give you advice based on what it sees you doing on your screen. It also watched me play Indiana Jones and the Great Circle, and it successfully identified some of the characters and told me some information about them. You can enable "Grounding" in the sidebar to let it use Google Search even in voice mode. The video streaming and integrated search make it far more useful than ChatGPT Advanced Voice mode is currently.
- selvan 2y agoGet started documentation on Multimodal Live API : https://ai.google.dev/api/multimodal-live https://ai.google.dev/api/multimodal-live
- Brotkrumen 2y agoSounds interesting, but voice input isn't working for me there. I guess I'm too niche with my Mac and Firefox setup.
- mentalgear 2y agoActually plenty of tech people are using mac & firefox
- portaouflop 2y agoIrony detectors malfunctioning perhaps?
- baq 2y ago'irony' meant 'something made of metal' last time I checked
- weatherlite 2y agoGoogle is on fire lately, stock went up considerably in the last few days and rightly so; besides the usual cash cows (search, youtube) they made huge progress with GCP, Gemini and quite possibly Waymo. They have a fantastic momentum imo.
- ducttapecrown 2y agoAnd quantum computing! They found a way to make error decrease with number of qubits.
- lngnmn2 2y ago[dead]
- mike80it 2y agoGemini 1120, 1206, and Gemini 2.0 flash have better coding results than ChatGPT o1 and Claude Sonnet 3.5. They did it: from now on Google will keep a leadership position. They have too much data (Search, Maps, Youtube, Chrome, Android, Gmail, etc.), and they have their own servers (it's free!) and now the Willow QPU. To me, it is evident how the future will look. I'll buy some more Alphabet stocks
- eichi 2y agoGemini needs to improve quality of the software.
- irthomasthomas 2y agoI wrote about a possibly emergent behaviour in Gemini 2 that was only seen previously in sonnet-3.5 https://x.com/xundecidability/status/1867044846839431614 https://x.com/xundecidability/status/1867044846839431614
- greenchair 2y agoshouldn't there be a benchmark category called 'strawberry' by now? or maybe these AIs are still too dumb to handle it which is why it is left off?
- MetroWind 2y agoYes, "agentic". Very creative word.
- NoZZz 2y agoUgh. You're investing in this, but it disenfranchises the people that are investing in you. Well. Unless you'd like to admit that your income is pure fantasy, which it is.
- s3p 2y agoThis might be a dumb thought, but the naming of this model naturally suggests we have a Pro model coming out soon. Similar to Claude 3.5 Sonnet, where they never mentioned the Opus model afterward, does anyone think this is all we will see from Google in terms of the 2.0 model family? Does anyone think there will be some 2.0 Pro model that will blow this out of the water? The naming makes it seem so.