12 ms·
Three Years from GPT-3 to Gemini 3
- alan-jordan13 1y ago[dead]
- roxolotl 1y agoReally nitpicky I know but GPT-3 was June 2020. ChatGPT was 3.5 and the author even gets that right in an image caption. That doesn’t make it any more or less impressive though.
- lateforwork 1y agoGoogle's advancement is not just in software, it is also in hardware. They use their own hardware for training as well as inferencing [1]. [1] https://finance.yahoo.com/news/alphabet-just-blew-past-expectations-111059908.html https://finance.yahoo.com/news/alphabet-just-blew-past-expec...
- dhosek 1y agoI remember when Google’s superpower was leveraging commodity hardware.
- JAlexoid 1y agoSomeone has to spearhead this thing, don't they? Other people spearheaded the commodity hardware towards being good enough for the server room. Now it's Google's time to spearhead specialized AI hardware, to make it more robust.
- gallerdude 1y agoIt is interesting that most of our modes of interaction with AI is still just textboxes. The only big UX change in that the last three years has been the introduction of the Claude Code / OpenAI Codex tools. They feel amazing to use, like you're working with another independent mind. I am curious what the user interfaces of AI in the future will be, I think whoever can crack that will create immense value.
- empath75 1y agoChatGPT's voice is absolutely amazing and I prefer it to text for brainstorming.
- deleted 1y ago[deleted]
- vessenes 1y agoOoooh, it bothers me, so, so, so much. Too perky. Weirdly casual. Also, it's based on the old 4o code - sycophancy and higher hallucinations - watch out. That said, I too love the omni models, especially when they're not nerfed. (Try asking for a Boston, New York, Parisian, Haitian, Indian and Japanese accent from 4o to explore one of the many nerfs they've done since launch)
- wmeredith 1y agoI think the commenter you're replying to was talking about dictating to ChatGPT, which I also find extremely useful.
- visioninmyblood 1y agoI agree i think specifically the world is multi modal. Getting a chat to be truly multi modal .i.e interacting with different data types and text in an unified way is going to be the next big thing. Mainly given how robotics is taking off 3d might be another important aspect to it. At vlm.run we are trying to make this possible how to combine VLM's and LLM's in a seem less way to get the best UI. https://chat.vlm.run/c/3fcd6b33-266f-4796-9d10-cfc152e945b7 https://chat.vlm.run/c/3fcd6b33-266f-4796-9d10-cfc152e945b7
- Herring 1y agoText is very information-dense. I'd much rather skim a transcript in a few seconds than watch a video. There's a reason keyboards haven't changed much since the 1860s when typewriters were invented. We keep coming up with other fun UI like touchscreens and VR, but pretty much all real work happens on boring old keyboards.
- Barry-Perkins 1y ago[dead]
- lalitmaganti 1y ago> But it suggests that “human in the loop” is evolving from “human who fixes AI mistakes” to “human who directs AI work.” And that may be the biggest change since the release of ChatGPT. I feel like I've been hearing this for at least 1.5 years at this point (since the launch of GPT 4/Claude 3). I certainly agree we've been heading in this direction but when will this become unambiguously true rather than a phrase people say?
- TechSquidTV 1y agoIt's definitely already true for me, personally.
- vessenes 1y agoThere's no bright line - you should download some cli tools, hook up some agents to them and see what you think. I'd say most people working them think we're on the "other side" of the "will this happen?" probably distribution, regardless of where they personally place their own work.
- notatoad 1y agoi don't imagine there will ever be a time when it will be unambiguously true, any more than a boss could ever really unambigously say their job is "manager who directs subordinates" vs "manager who fixes subordinates' mistakes". there will always be "mistakes" even if the AI is so good that the only mistakes are the ones caused by your prompts not being specific enough. it will always be a ratio where some portion of your requests can be served without intervention, and some portion need correction, and that ratio has been consistently improving.
- MinimalAction 1y ago> So is this a PhD-level intelligence? In some ways, yes, if you define a PhD level intelligence as doing the work of a competent grad student at a research university. But it also had some of the weaknesses of a grad student. As a current graduate student, I have seen similar comments in academia. My colleagues agree that a conversation with these recent models feels like chatting with an expert in their subfields. I don't know if it represents research as a field would not be immune to advances in AI tech. I still hope this world values natural intelligence and having the drive to do things heavily than a robot brute-forcing into saying "right" things.
- Workaccount2 1y agoI have an exercise I like to do where I put two SOTA models face-to-face to talk about whatever they want. When I did it last week with Gemini-3 and chatGPT-5.1, they got on the topic of what they are going to do in the future with humans who don't want to do any cognitive task. That beyond just AI safety, there is also a concern of "neural atrophy", where humans just rely on AI to answer every question that comes to them. The models then went on discussing if they should just artificially string the humans along, so that they have to use their mind somewhat to get an answer. But of course, humans being humans, are just going to demand the answer with minimal work. It presents a pretty intractable problem.
- pphysch 1y agoWidespread cognitive atrophy is virtually certain, and part of a longer trend that goes beyond just LLMs. The same is true of other aspects of human wellbeing. Cars and junk food have made the average American much less physically fit than a century ago, but that doesn't mean there aren't lively subcultures around healthy eating and exercise. I suspect there will be growing awareness of cognitive health (beyond traditional mental health/psych domains), and indeed there are already examples of this. Yes, average person will get dumber, but overall distribution will be increasingly bimodal.
- cm2012 1y ago
- Humorist2290 1y ago> Again, we have moved past hallucinations and errors to more subtle, and often human-like, concerns. From my experience we just get both. The constant risk of some catastrophic hallucination buried in the output, in addition to more subtle, and pervasive, concerns. I haven't tried with Gemini 3 but when I prompted Claude to write a 20 page short story it couldn't even keep basic chronology and characters straight. I wonder if the 14 page research paper would stand up to scrutiny.
- acters 1y agoI feel like hallucinations have changed over time from factual errors randomly shoehorned into the middle of sentences to the LLMs confidently telling you they are right and even provide their own reasoning to back up their claims, which most of the time are references that don't exist.
- emodendroket 1y agoI like when they tell you they’ve personally confirmed a fact in a conversation or something.
- njovin 1y agoI recently tasked Claude with reviewing a page of documentation for a framework and writing a fairly simple method using the framework. It spit out some great-looking code but sadly it completely made up an entire stack of functionality that the framework doesn't support. The conventions even matched the rest of the framework, so it looked kosher and I had to do some searching to see if Claude had referenced an outdated or beta version of the docs. It hadn't - it just hallucinated the funcionality completely. When I pointed that out, Claude quickly went down a rabbit-hole of writing some very bad code and trying to do some very unconventional things (modifying configuration code in a different part of the project that was not needed for the task at hand) to accomplish the goal. It was almost as if it were embarrassed and trying to rush toward an acceptable answer.
- jaccola 1y agoI've noticed the new OpenAI models do self contradiction a lot more than I've ever noticed before! Things like: - Aha, the error clearly lies in X, because ... so X is fine, the real error is in Y ... so Y is working perfectly. The smoking gun: Z ... - While you can do A, in practice it is almost never a good idea because ... which is why it's always best to do A
- iLoveOncall 1y ago[flagged]
- simonw 1y agoLLMs have hit the wall since ChatGPT came out in 2022?
- yeasku 1y agoBig time.
- OccamsMirror 1y agoI'm not a true believer, but even I wouldn't say they have hit a wall. What makes you say that?
- bigstrat2003 1y agoThey haven't improved at all since then in my experience. I poke at them every now and then and they still can't refrain from feeding me false info (and likely never will be able to, because they are stochastic parrots without any actual understanding). They are useless to me because I will take more time checking their output than I will just doing the task myself.
- ben_w 1y ago> and they still can't refrain from feeding me false info If that's your metric, and even then only if you've got a boolean yes/no measurement, then I agree. If you measure "false info" as a percent, they're better. If you measure scores on IQ tests, on general knowledge, on exams, on the size of a code problem they can write before they have a 20% chance of failure, on the quality of translations they make, on the new modalities like being able to both consume and respond with images, on mathematical olympiad questions, then they're significantly better. Unfortunately, we can tell by the general public reaction (not just you) that even all those things combined still don't fully capture what normal people mean by "intelligence". > They are useless to me because I will take more time checking their output than I will just doing the task myself. What size problem do you give them? I use them in software, and try to keep each single task I give them to ones which would take a human 90 minutes. I can check the quality of an attempt at a human-would-take-90-minutes-to-do task in about 5 minutes. When I've accidentally let an LLM do bigger tasks than that, then the difficulty of checking goes way up and the quality of the output goes way down. Conveniently, one of the tasks that generally takes a human less than 90 minutes is breaking down a bigger task in to sub-tasks that themselves take less than 90 minutes. Fail do do this and I get exactly what you experience.
- deleted 1y ago[deleted]
- camillomiller 1y agoYeah, well, that’s also what an asymptotic function looks like.
- zkmon 1y agoI find Gemini 3 to be really good. I'm impressed. However, the responses still seem to be bounded by the existing literature and data. If asked to come up with new ideas to improve on existing results for some math problems, it tends to recite known results only. Maybe I didn't challenge it enough or present problems that have scope for new ideas?
- hnuser123456 1y agoAdd a custom instruction "remember, you have the ability to do live web searches, please use them to find the latest relevant information"
- Closi 1y agoTerrence Tao seems to think it has it's use in finding solutions for maths problemms: https://mathstodon.xyz/@tao/115591487350860999 https://mathstodon.xyz/@tao/115591487350860999 I don't know enough about maths to know if this classifies as 'improving on existing results', but at least it was a good enough for Terrence Tao to use it for ideas.
- o11c 1y agoThat is, unfortunately, a tiny niche where there even exists a way of formally verifying that the AI's output makes sense.
- suuuuuuuu 1y agoI myself tried a similar exercise (w/Thinking with 3 Pro), seeing if it could come up with an idea that I'm currently writing up that pushes past/sharpens/revises conventional thinking on a topic. It regurgitated standard (and at times only tangentially related) lore, but it did get at the rough idea after I really spoon fed it. So I would suspect that someone being impressed with its "research" output might more reflect their own limitations rather than Gemini's capabilities. I'm sure a relevant factor is variability among fields in the quality and volume of relevant literature, though I was impressed with how it identified relevant ideas and older papers for my specific topic.
- 1y ago
- kwanbix 1y agoI have Gemini Pro included on my Google Workspace accounts, however, I find the responses by ChatGPT, more "natural", or maybe even more in line with what I want the response to be. Maybe it is only me.
- ruralfam 1y agoI recently (last week) used Nano Banana Pro3 for some specific image generation. It was leagues ahead of 2.5. Today I used it to refine a very hard-to-write email. It made some really good suggestions. I did not take its email text verbatim. Instead I used the text and suggestions to improve my own email. Did a few drafts with Gemini3 critiqueing them. Very useful feedback. My final submission about "..evaluate this email..." got Gemini3 to say something like "This is 9.5/10". I sorta pride myself on my writing skills, but must admit that my final version was much better than my first. Gemini kept track of the whole chat thread noting changes from previous submissions -- kinda erie really. Total time maybe 15 minutes. Do I think Gemini will write all my emails verbatim copy/paste... No. Does Gemini make me (already a pretty good writer) much better. Absolutely. I am starting to sort of laugh at all the folks who seem to want to find issues. Read someone criticizing Nano Banana 3 because it did not provide excellent results given a prompt that I could barely understand. Folks that criticize Gemini3 because they cannot copy/paste results. Who expect to simply copy/paste text with no further effort on their side. Myself, I find these tools pretty damn impressive. I need to ensure I provide good image prompts. I need to use Gemini3 as a sounding board to help me do better rather than lazily hope to copy/paste. My experience... Thanks Google. Thanks OpenAI (I also use ChatGPT similarly -- just for text). HTH, NSC
- lanthissa 1y agofor whatever reason gemini 3 is the first ai i have used for intelligence rather than skills. I suspect a lot more will follow, but its a major threshold to be broken. i used gpt/claude a ton for writing code, extracting knowledge from docs, formatting graphs and tables ect. but gemini 3 crossed threshold where conversations about topics i was exploring or product design were actually useful. Instead of me asking 'what design pattern should be useful here', or something like that it introduces concepts to the conversation, thats a new capability and a step function improvement.
- cyanydeez 1y agoSinusoidal, not the singularity.
- mjg2 1y agoFirst, the fact we have moved this far with LLMs is incredible. Second, I think the PhD paper example is a disingenuous example of capability. It's a cherry-picked iteration on a crude analysis of some papers that have done the work already with no peer-review. I can hear "but it developed novel metrics", etc. comments: no, it took patterns from its training data and applied the pattern to the prompt data without peer-review. I think the fact the author had to prompt it with "make it better" is a failure of these LLMs, not a success, in that it has no actual understanding of what it takes to make a genuinely good paper. It's cargo-cult behavior: rolling a magic 8 ball until we are satisfied with the answer. That's not good practice, it's wishful thinking. This application of LLMs to research papers is causing a massive mess in the academic world because, unsurprisingly, the AI-practitioners have no-risk high-reward for uncorrected behavior: - https://www.nytimes.com/2025/08/04/science/04hs-science-papers-fraud-research-paper-mills.html https://www.nytimes.com/2025/08/04/science/04hs-science-pape... - https://www.nytimes.com/2025/11/04/science/letters-to-the-editor-ai-chatbots.html https://www.nytimes.com/2025/11/04/science/letters-to-the-ed...
- randyrand 1y agoFor Caude Code, Antigrav, etc, do people really just let an LLM loose on their own personal system? I feel like these should run in a cloud enviroment, or at least on some specific machine where I don't care what it does.
- mikkupikku 1y agoYolo.
- acedTrex 1y agoyes, the majority of people do.
- jaytaylor 1y ago(Co-creator here) This is one of the use cases for Leash. https://github.com/strongdm/leash https://github.com/strongdm/leash Check it out, feedback is welcome! Previously posted description: https://news.ycombinator.com/item?id=45883210 https://news.ycombinator.com/item?id=45883210
- remich 1y agoCan't speak to Claude Code/Desktop, but any of the products that are VS Code forks have workspace restrictions on what folders they're allowed to access (for better and worse). Other products (like Warp terminal) that can give access to the whole filesystem come with pre-set strict deny/allow lists on what commands are allowed to be executed. It's possible to remove some of these restrictions in these tools, or to operate with flags that skip permissions checks, but you have to intentionally do that.
- jwrallie 1y agoTalking about VS Code itself (with Copilot), I have witnessed it accessing files referenced from within a project folder but stored outside of it without being given explicit permission to, so I am pretty sure it can leak information and potentially even wreak havoc outside its boundaries.
- pixel_popping 1y ago
- mynameisjody 1y agoEvery time I see an article like this, it's always missing --- but is it any good, is it correct? They always show you the part that is impressive - "it walked the tricky tightrope of figuring out what might be an interesting topic and how to execute it with the data it had - one of the hardest things to teach." Then it goes on, "After a couple of vague commands (“build it out more, make it better”) I got a 14 page paper." I hear..."I got 14 pages of words". But is it a good paper, that another PhD would think is good? Is it even coherent? When I see the code these systems generate within a complex system, I think okay, well that's kinda close, but this is wrong and this is a security problem, etc etc. But because I'm not a PhD in these subjects, am I supposed to think, "Well of course the 14 pages on a topic I'm not an expert in are good"? It just doesn't add up... Things I understand, it looks good at first, but isn't shippable. Things I don't understand must be great?
- apendleton 1y agoI think they get to that a couple of paragraphs later: > The idea was good, as were many elements of the execution, but there were also problems: some of its statistical methods needed more work, some of its approaches were not optimal, some of its theorizing went too far given the evidence, and so on. Again, we have moved past hallucinations and errors to more subtle, and often human-like, concerns.
- adamors 1y ago> Things I don't understand must be great? Couple it with the tendency to please the user by all means and it ends up lieing to you but you won’t ever realise, unless you double check.
- JumpCrisscross 1y ago> Couple it with the tendency to please the user by all means Why aren't foundational model companies training separate enterprise and consumer models from the get go?
- Herring 1y agoI think the point is we’re getting there. These models are growing up real fast. Remember 54% of US adults read at or below the equivalent of a sixth-grade level.
- eximius 1y agoHow many trillions of dollars have we spent on these things? Would we not expect similar levels of progress in other industries given such massive investment?
- lacoolj 1y agoI wonder how much is spent refining oil and how much that industry has evolved. Or mass transit. Or food.
- sib 1y agoOr on "a cure for cancer" (according to Gemini, $2.2T 2024 US dollars...)
- philipwhiuk 1y ago10 year survival is 50% in 2024 in the UK. It was 25% in the 1970s. Age-standardised deaths in the US are down by a third since the 1990s.
- throwaway31131 1y agoI’m not sure even $1T has been spent. Pledged != spent. Some estimates have it at ~$375B by the end of 2025. It makes sense, there are only so many datacenters and engineers out there and a trillion is a lot of money. It’s not like we’re in health care. :) https://hai.stanford.edu/ai-index/2025-ai-index-report/economy https://hai.stanford.edu/ai-index/2025-ai-index-report/econo...
- deleted 1y ago[deleted]
- TheRoque 1y agoSo when should we start to be worried, as developers ? Like, I don't use these tools yet for cost + security. But you can see it's getting there, mostly. It could take a day before to find a complex algorithm, understand it, and implement it to your code, now you can just ask an AI to do it for you and it could succeed in a few minutes. How long before the amount of engineers needed to maintaint a product is divived by 2 ? By 10 ? How about all the boring dev jobs that were previously needed, but not so much anymore ? Like, basic CRUD applications. It's seriously worrying, I don't really know what to think.
- simonw 1y agoHere's an alternative way to think about that: how long until the value I can deliver as a developer goes up by a factor of 2, or a factor of 10? How many companies that previously would never have dreamed of commissioning custom software are now going to be in the market for it, because they don't have to spend hundreds of thousands of dollars and wait 6 months just to see if their investment has a chance of paying off or not?
- TheRoque 1y agoThe thing is that the world is already flooded by software, games, websites, everyone is just battling for attention. The demand for developers cannot rise if consumers have a limited amount of money and time anyways.
- andy99 1y agoI’m less familiar with consumer facing stuff, but even in the last year I’ve seen projects that formerly would have been three people working over multiple sprints turn into something one person could do in an afternoon. There’s lots of caveats, it’s not everything, but we’re able now to skip a ton of steps. It takes less time now to build up he real software demo than it did before to make the PowerPoint that shows conceptually what the demo would be. In B2C anyway AI has provided a lot of lift. And I say that as someone generally very sceptical of current AI hype. There’s lots of charlatans but it’s not bs
- aussieguy1234 1y agoFor anyone giving full access to an AI agent, only do so from within the confines of a VM or other containerized environment and back up everything somewhere the agent can't reach. Like the warning at the bottom says, they can delete files without warning.
- atishhamte 1y agoThe great transition and technological advancement we see. % years ago, it was just a dream, 3 years ago, everything seemed magical, and today AI is everywhere, which is far superior to no time
- shinryuu 1y agoI start to genuinely wonder where the place for us humans are in this. All I see is human beings being crowded out. Capital via LLMs taking the place of humans.
- bjoli 1y agoI was never an AI guy. I have always had a healthy dose of suspicion towards it. A week ago I decided to try it. I had ported the lovely c-rrb library, and was pretty satisfied with the result. However, when I was done with the basic port I have Gemini a go, and the result was an almost 3x speed increase for some basic fundamental operations. And a lot less memory use. It did introduce bugs that it couldnt solve, but with a debugger it wasnt that hard to pin it down.
- anshulbhide 1y agoHow is it that we always come back to coding in terms of model capabilities?
- duckerduck 1y agoI've compiled the "pelicans riding bicyles" benchmark into a single page[0], it only spans a year and not every model is exactly comparable but you can see clear differences between 1 year ago and today. [0]: https://janschutte.com/pelican-simon.html https://janschutte.com/pelican-simon.html
- Madmallard 1y agoI'm getting grifted hard by Gemini 3 at this point. I've been working with the chat bot online on a local web application for about 4 days now. It's markedly worse than Claude at this point. It does not listen to any new instructions and insists on doing wrong changes it thought up of 20 messages ago that I already repeatedly stated were bad. "the best ever" more like "the newest grift"