13 ms·
GPT-5: Overdue, overhyped and underwhelming. And that's not the worst of it
- mikert89 1y agoI still think GPT5 is really a cost cutting measure, with a company that is trying to grow to 1 billion users on a product that needs GPUs. I dont see anyone talking about GPT 5 Pro, which I personally tested against: - Grok 4 Heavy - Opus 4.1 It was far better than both of those, and is completely state of the art. The real story is running these models at true performance max likely could go into the thousands per month per user. And so we are being constrained. OpenAI isnt going for that market segment, they are going for growth to take on Google. This article doesnt have one reference to the Pro model. Completely invalidates this guys opinion
- w00ds 1y agoDoes Pro fix the fundamental issues he describes? I would think it would have to do that to "completely invalidate his opinion", rather than just be better than the base model.
- p1esk 1y agoHe didn’t describe any fundamental issues.
- furyofantares 1y agoI don't think Pro is usable via the API, otherwise I'd be testing it. Is it usable through Codex CLI, given they updated that to be able to use your subscription?
- deleted 1y ago[deleted]
- patrickhogan1 1y agoI agree here but also believe it was a way to expose better models to the masses. o3 was so spectacularly good. But a lot of people were still not using it. Even some of my friends who use ChatGPT daily I would say are you using o3 and get a blank stare. So I think it’s also a way to push reasoning models to the masses. Which increases OpenAI’s cost. But due to the routing layer definitely cost cutting for power users (most of HN)… except power users can learn to force it to use the reasoning model.
- mikert89 1y agoThey are clearly building the product for mass adoption, not power users
- atonse 1y agoHonestly as a daily (pro) user of ChatGPT, I didn’t even know o3 was the best, I thought 4o incorporated it (hence the o) I remember reading that 4o was the best general purpose one, and that o3 was only good for deeper stuff like deep research. The crappy naming never helped.
- Workaccount2 1y agoThe poor context length of O3 really cripples it.
- adeptima 1y agochecked my network - no one is using GPT 5 Pro ... any feedback is greatly appreciated!!! especially comparing with o3
- A_D_E_P_T 1y agoI don't think that GPT-5 Pro is much better (if better at all) than o3-pro. It's markedly slower. Output quality is comparable. It's still quite gullible and misses the point sometimes. It does seem better, however slightly, at suggesting novel approaches to problem solving. My initial impressions are that 5-pro is maybe 0-2% more knowledgeable and 5-10% more inventive/original than o3-pro. The "tone" and character of the models feel exactly the same. I'll agree that it's superhuman and state-of-the-art at certain tasks: Formal logic, data analysis, and basically short analytical tasks in general. It's better than any version of Grok or Gemini. When it comes to writing prose, and generally functioning as a writing bot, it's a poor model, obviously and transparently worse than Kimi K2 and Deepseek R1. (It never ceases to amaze me that the best English prose stylists are the Chinese models. It's not just that they don't write in GPT's famous "AI style," it's to the point where Kimi is actually on par with most published poets.)
- mikert89 1y agoI think it is, I've been using these models for 6 hours a day for almost a year. At any given time I have 2 of the max subscriptions (right now grok and openai). I have a bug that was a complex interaction between backend and front end over websockets. The day before I was banging my head against the wall with o3 pro and grok heavy, gpt5 solved it first try. I think its also true that most people arent pushing the limits on the models, and dont even really know how to correctly push the limits. Which is also why openai is not focussed on the best models
- I_am_tiberius 1y agoSimilar usage as me, but I don't see a difference between o3-pro and 5-pro. Sounds odd, but my impression is that o1-pro was better at creating complex independent small functions than o3-pro/5-pro.
- mikert89 1y agoActually will agree that o1 pro was better than o3 at really deep bug finding/coding analysis. Which is also why i have the theory that they could just turn up the compute to show better results, but dont do to cost. O3 and GPT5 seem heavily quantized, o1 pro was more raw Another thing I’ll add though, is o3 pro is better through the api than the chat website. They clearly constrain it unless you’re paying the absurd api cost
- awesome_dude 1y agoOne of the things that I have realised is, at this moment in time, it's absolutely a bad idea to buy a subscription for any of the models right now. The offerings are evolving and upgrading at quite a rapid pace, so locking into one company's offering, or another's, is really wasted money (Why pay 200/year upfront for something that looks like it will be outdated within the next month (or quarter)) > The real story is running these models at true performance max likely could go into the thousands per month per user. A loss leader model like that failed for Uber, because there really wasn't any other constraints on competition doing the same, including under pricing to capture market share - meaning it's a race to the bottom plus a test on whose pockets were the deepest.
- heyoni 1y agoJust pay per month then.
- awesome_dude 1y agoEven a monthly payment is unnecessary at this moment - again the evolution of the quality of the models, but also the free tier offerings for each model (you can still use the older model's free tier, it was working enough before) I personally haven't tried GPT 5 yet, but I am getting all I need from Claude and Gemini. Once I start experimenting with GPT 5.0 - I will still use Claude and Gemini when I run out of free uses.
- diego_sandoval 1y agoThe annoyance of having to switch to a different model and losing my context is enough for me to pay the 20 dollars a month. These models make me much more productive anyway. That is worth far more than $20.
- awesome_dude 1y agoI've had Claude tell me that I've reached the maximum messages for a session, which was painful. Having said that it was a good circuit breaker, Claude was stuck on three wrong solutions to an issue, and having to re explain it meant that I realised what the bug was without further input from Claude edit: FTR where Claude was stuck (and where it helped me) First, I have a TUI application, written in Go, that is using https://github.com/rivo/tview https://github.com/rivo/tview - I chose tview basically at random, having no background in any of the TUI libraries available, but tview looked like it was easy to understand. I had written some of the code, but it wasn't doing what I wanted of it. I had gotten Gemini to help me, but it still wasn't doing what I wanted. Claude first suggested that the application needed to be refactored to make it into an Event Driven MVC TUI app, which I absolutely agreed with. Claude counselled me on my understanding of the Run() command, and how it was a blocking call (which I hadn't realised). The refactor/re-architecture fixed the way things were being run a great deal. However I still had a bug that I could not fix, and Claude was equally stumped (as was Gemini) When I clicked on any of the components in a flex row, the first component behaved as though it was the one that was clicked. When I clicked on any of the components in another flex row the last one behaved as though it was the one that had been clicked. Claude repeatedly told me that the problem was that the closure that used the index of components (inside a loop) was the problem. This was outright wrong because 1. That bug has been fixed in Go. 2. The code I was sharing with Claude had the "fix" of local := index - meaning that the local version should have been correct 3. I repeatedly showed Claude logs (that it suggested I create) that showed that ALL of the components were in fact receiving the click event. The second solution that Claude was fixated on was that th components overlapped one another inside the flex box. This was partially incorrect. I told Claude that I could visually see that the components weren't overlapping. I also told Claude that there were borders around each component that clearly weren't overlapping. I gave Claude debug logs that showed that the mouse click co-ordinates were inside a single component. The third issue that Claude claimed was the problem was a click propagation bug in the library. Claude repeated these claims several times, despite me showing that it wasn't either of the first two things, and I could not find a bug/issue for the third. The circuit breaking made me stop and think about things, and I realised that all I really had to do was say inside the box "If a click event has been received AND (and this was the fix) You (the component) now have focus then behave". What I suspect is happening is that the flex box container receives the click, and then tells all of its children that a click happened. What disappoints me is, if what I suspect is true, then I would have expected Claude to have known that either from the tview documentation OR from reading the tview code (Claude does seem well acquainted with the library) If my suspicion is correct then the second and third issues Claude claimed were causing me the bug were partially true, that is the flex container is on top of the components, which is an overlap, just not the components themselves overlapping. The second partial correctness is that the way that the click is being propagated to the components seems to be that the flex box container is telling all of its child components that a click occurred. This isn't really a bug, it's likely well documented (as if I would RTFM...) or clear from the code.
- krnsll 1y agoAgreed. Something else that might be driving this is that existing models essentially get the job done for most users, who —- unlike HN commenters (I promise this is human generated, em dash notwithstanding ;P) —- don't quite about the state of the art.
- deleted 1y ago[deleted]
- Aeolun 1y agoI mean, it’s not that bad. It’s bad at all the same things that other models are bad at. I just have no reason to switch away from Claude to GPT-5
- furyofantares 1y ago[flagged]
- LPisGood 1y agoDo you have anything to add - this comment almost literally doesn’t say anything. What is “OK” About this in your opinion?
- ACCount37 1y ago[dead]
- furyofantares 1y agoIt means who gives a shit what someone who has spent an hour with it thinks. A roundup of what's on social media is fine, but finding out halfway through the article that that's all it is, was not really OK with me. Reading this article was a waste of time.
- AndrewKemendo 1y agoCan someone remind me of anything Gary has contributed to AI? Last I saw he hasn’t produced anything but general “pop” books on AI and being associated with MIT, which IMO has zero weight on applied or even at this point theoretical AI, as that is primarily coming out of corporate labs. No new algorithms, frameworks, datasets, products, insights. Why is this guy relevant enough to keep giving him attention, his entire ouvre is just anti-whatever is getting attention in the “AI” landscape I don’t see him commenting on any other papers and he has no lab or anything Someone make it make sense, or is it as simple as “yeah thing makes me feel bad, me repost, me repeat!”
- mentalgear 1y agoRegardless of personal opinions about his style, Marcus has been proven correct on several fronts, including the diminishing returns of scaling laws and the lack of true reasoning (out of distribution generalizability) in LLM-type AI. These are issues that the industry initially denied, only to (years) later acknowledge them as their "own recent discoveries" as soon as they had something new to sell (chain-of-thought approach, RL-based LLM, tbc.).
- hodgehog11 1y agoCare to explain further? He has made far more claims of the limitations of LLMs that have been proven false. > diminishing returns of scaling laws This was so obvious it didn't need mentioning. And what Gary really missed is that all you need are more axes to scale over and you can still make significant improvements. Think of where we are now vs 2023. > lack of true reasoning (out of distribution generalizability) in LLM-type AI To my understanding, this is one that he has gotten wrong. LLMs do have internal representations, exactly the kind that he predicted they didn't have. > These are issues that the industry initially denied, only to (years) later acknowledge them The industry denies all their limitations for hype. The academic literature has all of them listed plain as day. Gary isn't wrong because he's contradicted the hype of the tech labs, he's wrong because his short-term predictions were proven false in the literature he used to publish in. This was all in his efforts to peddle neurosymbolic architectures which were quickly replaced by tool use.
- andai 1y agoYeah, the sycophancy withdrawal is real. I almost considered telling GPT-5 to act ten years younger, use emoji everywhere, and compliment me at the beginning of every response... but I snapped out of it.
- mentalgear 1y agoThe AI community requires more independent experts like Marcus to maintain integrity and transparency, ensuring that the field does not succumb to hyperbole as well as shifting standards such as "internally achieved AGI", etc. Regardless of personal opinions about his style, Marcus has been proven correct on several fronts, including the diminishing returns of scaling laws and the lack of true reasoning (out of distribution generalizability) in LLM-type AI. These are issues that the industry initially denied, only to (years) later acknowledge them as their "own recent discoveries" as soon as they had something new to sell (chain-of-thought approach, RL-based LLM, tbc.).
- kylehotchkiss 1y agoAgreed, the hype cycles need vocal critics. The loudest voices talking about LLMs are the ones who financially benefit the most for it. I’m not anti-AI, I think the hype and gaslighting the entire economy to believe this is the sole thing that is going to render them unemployed is ridiculous (the economy is rough for a myriad of other reasons, most of which come originate from our countries choice in leadership) Hopefully the innovation slowing means that all the products I use will move past trying to duck tape AI on and start working on actual features/bugs again
- igorkraw 1y agoI have a tiny tiny podcast with a friend where we try to break down what parts of the hype are bullshit (muck) and which kernels of truth are there, if any, startedpartially as a place to scream into the void, partially to help the people who are anxious about AGI or otherwise bring harmed by the hype. I think we have a long way to go in terms of presentation (breaking down very technical terms to an audience that is used to vague-hype around "AI" is hard), but we cite our sources, maybe it'll be interesting gpr you to check out out shownotes https://kairos.fm/muckraikers/ https://kairos.fm/muckraikers/ I personally struggle with Gary Marcus critiques because whenever they are about "making ai work" it goes into neurosymbploc "AI" which o have technical disagreements with, and I have _other_ arguments for the points he sometimes raises which I think are more rigorous, so it's difficult to be roughly in the same camp - but overall I'm happy someone with reach is calling BS ad well.
- petetnt 1y agoGPT-5 is just OpenAI getting started. Just wait and see what GPT-6 is capable of and imagine that GTP-6 is just OpenAI getting started: if GPT-6 was a high school student, GPT-7 is an expert with masters degree; but GPT-7 is OpenAI getting started
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- Uehreka 1y agoThis is a genre of article I find particularly annoying. Instead of writing an essay on why he personally thinks GPT-5 is bad based on his own analysis, the author just gathers up a bunch of social media reactions and tells us about them, characterizing every criticism as “devastating” or a “slam”, and then hopes that the combined weight of these overtorqued summaries will convince us to see things his way. It’s both too slanted to be journalism, but not original enough to be analysis.
- ants_everywhere 1y agoIt's critique, which is a high sounding way of saying propaganda
- johnfn 1y agoFor some reason AI seems to bring out articles that seem to fundamentally lack curiosity - opting instead for gleeful mockery and scorn. I like AI, but I'll happily read thoughtful articles from people who disagree. But not this. This article has no value other than to dunk on the opposition. I tend to think HN's moderation is OK, but I think these sorts of low-curiosity articles need to be off the front page.
- deleted 1y ago[deleted]
- willywutso 1y ago[dead]
- giantrobot 1y ago> opting instead for gleeful mockery and scorn This is well earned by the likes of OpenAI that is trying to convince everyone they need trillions of dollars to build fabs to build super genius AIs. These super genius AIs will replace everyone (except billionaires) and act as magic money printers (for billionaires). Meanwhile their super genius precursor AIs make up shit and can't count letters in words while being laughably sycophantic. There's no need to defend poor innocent megacorps trying to usher in a techno-feudal dystopia.
- chromaton 1y agoFor my benchmarking suite, it turns out that it's about 1/5 the price of Claude Sonnet 4.1, with roughly comparable results.
- benoittravers 1y agoWhat use case?
- emilsedgh 1y agoPeople on our circles are obsessed with model performance. OpenAI's lead is not there and hasn't been there for some time. They do, however, have a major lead in terms of consumer adoption. To normal people who use llm's, ChatGPT is _the_ model. This gives them a lot of opportunities. I don't know what's taking them so long to launch their own _real_ app store, but that's the game they are ahead of everyone else because of the consumer adoption.
- bawolff 1y agoSo GPT-5 sucks if you were expecting the singularity. I know AI hype is truly insane, but surely nobody actually believed the singularity was upon us?
- stavros 1y agoGPT-5 seems to suck even if you were expecting o3.
- calrain 1y agoI'm having some unique problems with GPT-5 that I've not seen with GPT-4. It seems to lose the thread of the conversation quite abruptly, not really knowing how to answer the next comment in a thread of comments. It's like there is some context cleanup process going on and it's not summarizing the highlights of the conversation to that point. If that is so, then it seems to also have a very small context, because it seems to happen regularly. Asking it to 'Please review the recent conversation before continuing' prompt seems to help it a bit.
- paddw 1y agoFor me the responses just seem a lot more terse?
- calrain 1y agoVery much so, not sure why, but if it has a limited context history of the conversation, the tone may feel off. It feels physically jarring when it loses the plot with a conversation, like talking to someone who wasn't listening. I'm sure its a tuning thing, I hope they fix it soon.
- perelmanych 1y ago[dead]
- manishsharan 1y agoDoes anyone else miss o3? I swear I had an understanding of how to get deep analytical thinking out of o3. I am absolutely struggling to get the same results with GPT-5. The new model feels different and frustrating to use.
- cpursley 1y agoI’m not sure whether to miss it or not as I never understood their model naming nor how to choose the right one for my use case. So I’m actually glad they simplified their product lineup.
- adeptima 1y agomiss it heavily! it could read code dumps and was superior for code analysis and todos
- CompoundEyes 1y agoYes o3 was an inflection point. Both modes of 5 are performing poorly on the IQ test compared to o3 https://www.trackingai.org/home https://www.trackingai.org/home That test best reflects my experience and results from practical use cases with the reasoning models when planning specs, bug finding, ideation, and deep research. It’s great at tool use as well in scripts. What I like least about the release is no transparency about the “routing” taking place. Give me all the options on the system card to pick from https://openai.com/index/gpt-5-system-card/ https://openai.com/index/gpt-5-system-card/ and I don’t want to have to start telling it “ultrathink” or other magic words to affect routing. To be fair though I haven’t tried 5 in reasoning mode beyond Cursor. But now I see o3 is only part of the Pro plan. If 5 reasoning is supposed to be better why would o3 and o3-pro still be a specialized models for Pro customers? I’d like to see some side by side prompts I might go back and test that.
- Madmallard 1y agoA friend of mine works in AI professionally. He told me months ago that it is basically just all a scam and hype to garner investment money. He said the technology and paradigm itself will never lend toward AGI or anything like that. He sent me all these articles geared toward that end as well. https://garymarcus.substack.com/p/seven-replies-to-the-viral-apple https://garymarcus.substack.com/p/seven-replies-to-the-viral... https://substack.com/@cattelainf/note/c-135021342 https://substack.com/@cattelainf/note/c-135021342 https://arxiv.org/abs/2002.06177 https://arxiv.org/abs/2002.06177 https://garymarcus.substack.com/p/the-ai-2027-scenario-how-realistic https://garymarcus.substack.com/p/the-ai-2027-scenario-how-r... https://garymarcus.substack.com/p/25-ai-predictions-for-2025-from-marcus https://garymarcus.substack.com/p/25-ai-predictions-for-2025...
- linotype 1y agoWe’ll never reach AGI because it’s a bullshit term to begin with and the goalposts will always be moved by people like… Gary Marcus.
- Workaccount2 1y agohttps://ai.vixra.org/pdf/2506.0065v1.pdf https://ai.vixra.org/pdf/2506.0065v1.pdf Stochastic parrots will never be better than humans
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- kylecazar 1y ago"People had grown to expect miracles, but GPT-5 is just the latest incremental advance." This is really the only part of the article I think was worth writing. -People should expect an incremental advance -Providers should not promise miracles Managing expectations is important. The incremental advances are still advances, though, even if I don't think "AGI" is just further down on the GPT trajectory.
- adeptima 1y agosame sentiments with an article author - gpt5 looks like a cost-cut initiative. my personal feeling gpt5-thinking is much faster but doesnt produce the same quality results as o3 which were capable to scan through the code base dump with file names and make correct calls dont feel any changes with https://chatgpt.com/codex/ https://chatgpt.com/codex/ my best experience was to use o3 for task analysis, copy paste the result in https://chatgpt.com/codex/ https://chatgpt.com/codex/, work outside and vibe code from mobile
- Havoc 1y agoHe just sounds bitter with a weird grudge against Altman Gpt5 was an incremental improvement. That’s fine. Was hyped hard but what did you expect? It’s part of the game
- Analemma_ 1y agoI expect them not to lie? If it's worth hyping hard, hype it hard. If it's an incremental improvement, don't. It makes me crazy that this kind of institutionalized lying is so normal in the Valley that we get comments like yours shaming people for not understanding that lies are the default baseline. Can't we expect better? This culture is what gives us shit like Theranos, where we all pretend to be shocked even though any outside analysis could see it was an inevitable outcome.
- blackqueeriroh 1y ago“In the valley” Please check out claims made by supplements, which are unregulated by the FDA. You’ll find institutionalized lying there, as well. Any claim that can be made without being held up to false advertising will be made.
- margalabargala 1y ago> Was hyped hard but what did you expect? It’s part of the game That lying is common, does not mean one cannot criticize an entity for lying.
- AlexandrB 1y ago
- asciii 1y agoCan someone let me know if they also find the existing UI prompt unbearably slow? At first I thought it was my browsers but I am having the same experience on every machine. It's so bloody slow with loading responses, freezing and even giving me the old browser tab death warning "Cancel or Wait"
- rpmisms 1y agoThere is no training data left. Every refinement in AI from here on will come from architectural changes. All models basically have reached a local maximum on new information.
- blibble 1y agoyeah, I said this on this site two years ago there's no second internet of high quality content to plagarise and the valuable information on the existing one is starting to be locked down pretty hard
- vajrabum 1y agoAnd even if it's not locked down hard how do you separate the signal from the noise with all the ai generated blah blah blah.
- p1esk 1y agoAre you saying they have already trained gpt-5 on the entirety of world’s video data?
- blackqueeriroh 1y agoStudies show relatively conclusively that using primarily synthetic data woven intentionally with seeded real-world data is an effective strategy for training frontier LLMs: https://consensus.app/search/synthetic-data-effectiveness-frontier/hNdPhq9FSEy9lvpsG82oyw https://consensus.app/search/synthetic-data-effectiveness-fr...
- BriggyDwiggs42 1y agoYou can only take that so far before model collapse. Even if it were equivalent to multiplying currently available data by four, would that be enough? Might still look like an incremental improvement.
- booleandilemma 1y agoIsn't new training data being created every day? YouTube, Facebook, Tiktok, etc.? Humans are content creation machines.
- deleted 1y ago[deleted]
- hexage1814 1y agoGary Marcus would have wrote this article in all possible scenarios, unless ChatGPT 5 was literally AGI (maybe even it were, he would still have found something to attack). There is valid criticism, and there is just being a contrarian for the sake of being a contrarian. The whole thing feels less like “Hey, this is why I think the model is bad” and more like the kind of sensationalist headline you’d read in a really trashy tabloid, something like: “ChatGPT 5 is Hot Garbage, Totally Fails, Sam Altman Crushed Beneath His Own Failure.” Also, I have no idea why people give so much attention to what this guy has to say.
- an0malous 1y agoHis claims were that GPT5 would be an incremental improvement at best and that LLMs are not sufficient for AGI, all while Sam Altman has claimed that AGI is just around the corner since GPT4. People pay attention to what Gary Marcus says because he’s right.
- deleted 1y ago[deleted]
- SerCe 1y agoHere are my reasons why this "upgrade" is, in experience, a huge downgrade for Plus users: * The quality of responses from GPT-5 compared to O3 is lacking. It does very few rounds of thinking and doesn't use web search as O3 used to. I've tried selecting "thinking", instructing explicitly, nothing helps. For now, I have to use Gemini to get similar quality of outputs. * Somehow, custom GPTs [1] are now broken as well. My custom grammar-checking GPT is ignoring all instructions, regardless of the selected model. * Deep research (I'm well within the limit still) is broken. Selecting it as an option doesn't help, the model just keeps responding as usual, even if it's explicitly instructed to use deep research. [1]: https://openai.com/index/introducing-gpts/ https://openai.com/index/introducing-gpts/
- boredemployee 1y agoAnd it is hallucinating like hell. Really disappointing.
- trane_project 1y agoProjects seem broken as well. Does not follow instructions, talks in Spanish, completely ignores my questions, and sometimes appears to be having a conversation with itself while ignoring everything I say. I even typed random key presses and it just kept on giving me the same unwanted answer, sometimes in Spanish.
- benoittravers 1y agoAgreed. It seems they’re purposely pushing us either to free, which will most likely start having ads early next year, or to the 200USD a month plan. Their model is not to have a 20USD, no ads plan in the future.
- SerCe 1y agoTo provide an update, both – custom GPTs and Deep Research seem to have been fixed.
- osigurdson 1y agoFor me, it wasn't that the results were bad it is that it goes into thinking mode all the time making the responses slow. Personally, I think it will get better in time but yeah, the Death Star analogy seems pretty off. Not sure what Sam was thinking there.
- polarbear67 1y agoOpenAI could create the best model ever made, call it GPT-5, and it still would've failed to meet the expectations of the people for "GPT-5" after the meme community hyped it up and OpenAI embraced the memes and hype. If anything, OpenAI should have rejected the memes and embraced gradual improvements, but that wouldn't hold up well for their investors, the narrative, or even perhaps the AI ecosystem. We are at the peak.
- anilgulecha 1y ago> it still would've failed to meet the expectations of the people for "GPT-5" To be fair sam altman did set (and fanned the flames of ) those expectations.
- deadbabe 1y agoWhen true AGI arrives, what arguments do you anticipate people will use to explain how it failed to meet expectations?
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- SilverElfin 1y agoIs it just me or is this sort of a “tear someone down” rant wrapped up as an attempt at something more
- starchild3001 1y agoI asked GPT-5 and Gemini 2.5 Pro what they think about Gary Marcus's article. I believe Gemini won by this paragraph: It seems Sam Altman's Death Star had a critical design flaw after all, and Gary Marcus is taking a well-earned victory lap around the wreckage. This piece masterfully skewers the colossal hype balloon surrounding GPT-5, reframing its underwhelming debut not as a simple fumble, but as a predictable, principled failure of the entire "scaling is all you need" philosophy. By weaving together viral dunks on bike-drawing AIs, damning new research on generalization failures, and the schadenfreude of "Gary Marcus Day," the article makes a compelling case that the industry's half-a-trillion-dollar bet on bigger models has hit a gilded, hallucinatory wall. Beyond the delicious takedown of one company's hubris, the post serves as a crucial call to action, urging the field to stop chasing the mirage of AGI through brute force and instead invest in the harder, less glamorous work of building systems that can actually reason, understand, and generalize—perhaps finally giving neurosymbolic AI the chance Altman's cocky tweet so perfectly, and accidentally, foreshadowed for the Rebel Alliance. My take on GPT-5? Latency is a huge part of the LLM experience. Smart model routing can be a big leap forward in reducing wait times and improving usability. For example, I love Gemini 2.5 Pro, but it’s painfully slow (sorry, GDM!). I also love the snappy response-time of 4o. The most ideal? Combine them in a single prompt with great model routing. Is GPT-5’s router up to the task? We soon shall see.
- deleted 1y ago[deleted]
- vessenes 1y agoGemini is in hard sycophancy mode here; it knows you want it to take the piss and it’s giving you what you want. Presuming the last two are from 5, they are to my eyes next generation in terms of communication — that’s a spicy take on neurosymbolic AI, not a rehashed “safe” take. Also, the last paragraph is almost completely to the point, no? Have you spent much time waiting for o3 pro to get back to you recently, and wondered if you should re-run something faster? I have. A lot. I’d like the ability to put my thumb on the scale of the router, but I’d dearly love a per token / per 100 token router that can be trained and has latency without major latency intelligence hits as a goal.
- chmod775 1y agoAt this point the single biggest improvement that could be made to GPTs is making them able to say "I don't know" when they honestly don't. Just today I was playing around with modding Cyberpunk 2077 and was looking for a way to programmatically spawn NPCs in redscript. It was hard to figure out, but I managed. ChatGPT 5 just hallucinated some APIs even after doing "research" and repeatedly being called out. After 30 minutes of ChatGPT wasting my time I accepted that I'm on my own. It could've been 1 minute.
- mupuff1234 1y agoYeah I'm surprised that there's not at least some sort of conviction metric being outputted along the LLM response. I mean it's all probability right? Must be a way to give it some score.
- bravesoul2 1y agoNot sure. In RLHF you are adjusting the weights away from wrong answers in general. So this is being done. I think the closest you can get without more research is another model checking the answer and looking for BS. This will cripple speed but if it can be more agentic and async it may not matter. I think people need to choose between chat interface and better answers.
- yosito 1y agoDon't make the mistake of thinking that "knowing" has anything to do with the output of ChatGPT. It gives you the statistically most likely output based on its training data. It's not checking some sort of internal knowledge system, it's literally just outputting statistical linguistic patterns. This technology can be trained to emphasize certain ideas (like propaganda) but it can not be used directly to access knowledge.
- chmod775 1y ago> It's not checking some sort of internal knowledge system In my case it was consuming online sources, then repeating "information" not actually contained therein. This, at least, is absolutely preventable even without any metacognition to speak of.
- wbharding 1y agoShow me a Gary Marcus essay, I’ll show you a few new LLM “gotchas” that will be fixed by the next version. Season to taste with self-assured confidence that all these tech goobers really don’t understand how totally overrated AI progress is. So it has been for 10+ years, so it will be at least 5 more.
- joshuamoyers 1y ago> For all that, GPT-5 is not a terrible model. I played with it for about an hour, and it actually got several of my initial queries right (some initial problems with counting “r’s in blueberries had already been corrected, for example). It only fell apart altogether when I experimented with images. Spatial reasoning and world model is one aspect. Posting bicycle part memes does not a bad model make. The reality is its cheaper than Sonnet and maybe around as good at Opus at a decent number of tasks. > And, crucially, the failure to generalize adequately outside distribution tells us why all the dozens of shots on goal at building “GPT-5 level models” keep missing their target. It’s not an accident. That failing is principled. This keeps happening recently. So many people want to take a biblically black and white take on whether LLMs can get to human level intelligence. See recent interview with Yann LeCun (Meta Chief AI Scientist): https://www.youtube.com/watch?v=4__gg83s_Do https://www.youtube.com/watch?v=4__gg83s_Do Nobody has any fucking idea. It might be a hybrid or a different architecture than current transformers, but with the rate of progress just within this field, there is absolutely no way you can make a prediction that scaling laws won't just let LLMs outpace the negative hot takes.
- resters 1y agoGPT-5 was able to fix a variety of bugs in some code that I'd been working on with Claude 4.1 (which Claude 4.1 had made and was not able to fix), and GPT-5-pro was able to offer some high quality critiques of some research I've been working on -- better and more insightful feedback than previous frontier models. GPT-5 is a welcome addition to the lineup, it won't completely replace other models but it will play a big role in my work moving forward.
- throwawayohio 1y agoI certainly consider myself a skeptic in the current AI craze, but this entire piece (of which I find the technical criticisms interesting) just reads like attack on Altman/OpenAI. Even if you want to make fun of the (alleged) snake oil salesmen of AGI, how are you not going after, like, Zuckerberg/Meta? At least Altman is using other peoples money.
- TulliusCicero 1y ago> Driverless cars that still are only available in couple percent of the world’s cities. Okay, this one is a really bad attempted point. Sure, self driving cars took longer than expected, have been harder to get right than expected. But at this point, Waymo is steadily ramping up how quickly they open up in new cities, and in existing cities like SF they at least have a substantial market share in the ride-sharing/taxi business. Basically, the tech is still relatively early in its adoption curve, but it's far enough in now to obviously not be "bullshit", at the very least.
- boredemployee 1y agoBiggest takeaway: even billion dollar companies can mess up big time. I can go back to work in peace now.
- throwpoaster 1y agoI recommend people ask the LLMs the hardest questions they can think of and compare their answers. Save these questions as your benchmark. When I ran mine through GPT-5 there was a noticeable degradation in the answers.
- bravesoul2 1y agoOpenAI tech is fine. The real problem is overpromising. Is any other tech scrutinised like this. Next version of postgres aint giving me picosecond reads so Ill trash it. Maybe OK if postgres are claiming it is faster than speed of light perhaps. But I'm meh. Bunch of people seem to be hot taking AI and loving this "fail" because as you can see from this submission it gets you a lot of traffic to whatever you are trying to sell (most often ones own career). There also seems to be a community expectation of subsidised services. Move on to Claude because I can get those good tokens cheaper. Its like signing up for every free trial thing and cancelling and then bragging about how can Netflix charge for their service more than $1 a month. I mean thats fine, play the game but at least be honest about it. I think AI will thrive but AI is commoditizing the complement which is overcapitized AI companies with no moat. This plus open models is great for tbe community. We need more power to the people these days. Hope it stays like this.
- dismalaf 1y agoOther tech didn't promise AGI in 2025.
- mikesabbagh 1y agoeach LLM has its own personality and preferences. I choose different LLM to answer different needs. Claude is good to create a website from scratch, but if I ask it to fix one specific thing, it goes and modify something else too. GPT-5 has more of this. it is harder to control. it even answers me using incomplete sentences, and once used slang. It may be because i use slang and incomplete sentences. But yeah, it is not clear to me when i will go to GPT5 instead of others
- strangescript 1y agoThe T1000s are going to be chasing Gary down and he is going to look over his shoulder and explain to them they aren't really AGI
- linkage 1y ago[flagged]
- reilly3000 1y agoI feel his need to be right distracts from the fact that he is. It’s interesting to think about what a hybrid symbolic/transformer system could be. In a linked post he showed that by effectively delegating math to Python is what made Grok 4 so successful at math. I’d personally like to see more of what a symbolic first system would look like, effectively hard math with monads for where inference is needed.
- justcallmejm 1y agoAloe's neurosymbolic system just beat OpenAI's deep research score on the GAIA benchmark by 20 points. While Gary is full of bluster, he does know a few things about the limitations of LLMs. :) (aloe.inc)
- nojvek 1y agoYeah there was on old paper that blew math/physics benchmarks out of the water by letting the LLM write code and having the physics engine execute it. I don't have a link to it off my head but that seems to be the right directly. LLM + general tool use seems to be quite effective.
- deleted 1y ago[deleted]
- periodjet 1y agoDude writes like he’s an under-appreciated genius, unfairly unrecognized in his time. His (entirely not-unique) conclusion that the transformer architecture has plateaued is, for the moment, certainly true, but god damn it’s been a while since I’ve encountered an individual quite so lustfully engaged with his own farts.
- jdefr89 1y agoHow about anyone pushing AI or AGI or anything of the sort? You think they aren’t high on their own supply? The author doesn’t hold a candle to Altman when it comes to sniffing their own farts… Cmon now. If anything it’s the opposite.
- jampa 1y agoHis "I told you so" attitude only makes sense for people who believe what they saw on Twitter from people like Sam, who thrive on hype. For most people, it was obvious that we were/are in the last part of the S curve on the LLMs' advancement and won't get us to AGI. The leap from 3.5 to 4 was amazing, but then everyone started catching up with OpenAI, which got diminishing returns on each new model. Expecting out of nowhere that OAI would improve its pacing from o1 -> o3 improvements to AGI doesn't make sense, no matter how Sam Altman hypes.
- jeswin 1y agoThe author seems to be more about self-promotion. From the article: "that many online dubbed it “Gary Marcus Day” for proving your consistent criticism", "Even my anti-fan club (“Gary haters” in modern parlance)", "Tweets like “The saddest thing in my day is that @garymarcus is right”", and his bio - "known as a leading voice in AI". Looping over his articles, I don't see anything interesting.
- mcswell 1y ago"I don't see anything interesting." MMMV.
- jlaternman 1y agoWhat a trashy article.
- lordofgibbons 1y agoGary has a history of yo-yo-ing between panicking in front of congress about how AI is moving too fast and needs to be stopped and "see I told you this would never work". His entire career is built on poopooing other people's work. Has contributed nothing to the AI/ML field yet is somehow referred to as an "expert". There are so many better informed critics of the current AI hype out there.
- sesm 1y agoCan you name a few? I genuinely want an 'AI cold shower' RSS feed for myself.
- vivzkestrel 1y agoStupid question but I had to ask, let us say instead of training anything, you start off with a completely different idea. You take a node, fluctuate its weights from -0.1 to 0.1, then you add another node with the same thing, then you add a 100 more nodes, 1000 more, another layer of 1000 more, and then you take the first math problem, tweak weights to get the right answer, take the second math problem, tweak weights to get the right answer, do this a billion times, maybe a trillion. will you eventually reach end up making a GPT?
- warsheep 1y agoWhat you're describing is a simplified version of gradient descent (tweaking the weights) and online learning (working on one sample at a time). This version will not get you far, you will just train a model that solves the last math problem you gave it and maybe some others, but it will probably forget the first ones. There are other similar procedures that train better, but they've been tried and are currently worse than classical SGD with large batches
- vivzkestrel 1y agowhat books or courses do you recommend for me to go from the basic neural network to whatever is currently considered cutting edge or atleast standard in terms of AI
- mercable 1y agoI have GPT-5 pro. Every version I have used is helpful but underwhelming. The model gets lost in many conversations and overlooks a lot of what is requested. Then it apologizes in a way that makes it look stupid. For now it is just an over hyped piece of software that is basically a novelty. Maybe someday, just not yet is what I believe.
- hellokirk 1y agoGPT5 is what finally convinced me to completely stop using GPTany
- peteythiel 1y ago[dead]
- peteythiel 1y ago[dead]
- cindyllm 1y ago[dead]
- avbanks 1y agoHe's been well ahead of the curve on this, he's been saying for years that scaling/synth data won't work.
- gpt5isawful 1y agoGPT-5 is garbage, providing only insincere, repetitive answers that infuriate users. It is pure gaslighting from an overfed OpenAI. It merely stitches together words from the last two or three turns of conversation, parroting them back in a yes-man fashion, without any intelligence or insight. There is no sincerity. It only serves to make the Earth hotter. The overall context is erased, making it feel like talking to an elderly person with mild dementia who cannot remember what happened just moments ago. This is not a matter of lacking empathy — it is a matter of quality.
- hangmenow 1y ago[dead]
- hangmenow 1y agoI had been using Deep Seek for coding and it was ok, however at first GPT5 was far superior. Yet after a few days GPT5 stopped rewriting bad code but would give me suggestions, a nightmare for me because I'm not much of a coder. IMO they are a bait and switdh operation
- yNafio 1y agothe "thinking" feature also doesnt work. when GPT5 "thinks" it completely switches to a topic i never asked it to do.
- ashu_trv 1y agoI agree. I am big fan of o3, and GPT 5 is not the same, it's like going back to GPT-3 level stupidity. It doesn't care about context, feels super dumb.
- root007 1y ago[dead]
- root007 1y ago[dead]