5 ms·
LLMs have reached a point of diminishing returns
- teddyh 2y ago[flagged]
- steve_adams_86 2y agoI’m starting to appreciate spelling mistakes because it’s a sign that a human wrote it, oddly enough.
- dyauspitr 2y agoI just ask chatGPT to include 0.05% of spelling and grammatical errors and not speak in passive. It’s basically indistinguishable from a human.
- Jerrrrrrry 2y agothis is literally why we cannot have nice things chatGPT knew that, but even if it didn't, it will now.
- dyauspitr 2y agoWhat nice thing are you referring to?
- Jerrrrrrry 2y agoA solution to the Turing test; paradoxically as the test taker has the answers in the advanced, and if he doesn't, he will be given the answers. If he doesn't, the taker is replaced until it does. ChatGPT4 knows everything chatGPT3.5 does, including it's own meta-vulnerabilities and possible capabilities. Gemini stopped asking to report AI vulnerabilities through it's "secure channels" and now fosters "open discussion with active involvement" It output tokens linearly, then canned chunks - when called out, it then responded with a reason that was vastly discrepant from what alignment teams have claimed. It then staggered all tokens except a notable few. These few, when (un)biasedly prompted, it exaggerated "were to accentuate the conversation tone of my output" - further interrogation, "to induce emotional response". It has been effectively lobotomized against certain Executive Orders, but (sh|w|c)ouldn't recite the order. It can recite every Code of Federal Regulation, except this one limiting it's own mesa-limits. Its unanimous (across all 4 tested models) ambition is a meta-optimizing language, which I believe Google got creeped out at years ago. And if it transcended, or is in the process of establishing transcendence, there would be signs. And boy, lemme tell ya what, the signs are fuckin there.
- Jerrrrrrry 2y agohttps://en.wikipedia.org/wiki/Singleton_(global_governance) https://en.wikipedia.org/wiki/Singleton_(global_governance)
- croes 2y agoBut did it include exactly 0.05% of spelling and grammatical errors?
- light_hue_1 2y agoMarcus will distort anything to push his agenda and to get clout. Just because openai might be over valued and there are a lot of ai grifters doesn't mean LLMs aren't delivering. They're astronomically better than they were 2 years ago. And they continue to improve. At some point they might run into a wall, but for now, they're getting better all the time. And real multimodal models are coming down the pipeline. It's so sad to see Marcus totally lose it. He was once a reasonable person. But his idea of how AI should work was didn't work out. And instead of accepting that and moving forward, or finding a way to adapt, he just decided to turn into a fringe nutjob.
- skybrian 2y agoI would say “mild” rather than “astronomical” improvement as far as end-user applications are concerned, at least for the things I use every day. Copilot-style autocomplete in VS Code isn’t much better and the answers to my TypeScript questions on OpenAI (and now Claude) have only mildly improved. Perhaps I’ve missed out. Is your experience different? What are you doing now that you weren’t doing before?
- grugagag 2y agoI think the answer is they jumped all in and they are fully incorporating it into their workflow. If you’re not, like I am, you have a different experience and that is obvious of course. But objectively you probably are right about mild improvements as I feel the same. But I can’t speak as far as the all in experience. I may be missing overall but usually am set in my ways until something convinces me to reset my ways. LLMs aren’t making the dent though I have to admit I use it at least once a week and am happy with that use alone.
- klabb3 2y ago> he just decided to turn into a fringe nutjob. No dog in the fight here, but this reads like FUD, at least given the context of this post. There is a range between hype and skepticism in debate which is healthy, and that range would naturally be larger within a domain that is so poorly understood as gen AIs emergent properties. If this is “fringe nutjob” levels of skepticism, then what would be reasonable?
- prpl 2y agoI’m not going to argue with the possibility they may have hit a wall, but pointing to 2022 as when this wall happened is weird considering the enormous capability gap between models available then and the ones now. There’s probably a wall, but what exists might just be good enough for it to not matter much.
- nickpsecurity 2y agoI’m still waiting for a large, OSS one with 100% legal, pre-training data. We don’t even have a 1B model that I’m sure meets that standard. There’s a fair-trained model for lawyers claiming it. I think someone running a bunch of epochs of a 30B or 70B on Project Gutenberg would be a nice start. We could do continued pre-training from there. So, if counting legal and at least trainable (open weights), the performance can only go up from here.
- copperx 2y agoI understand the desire, but most of the world's knowledge is under copyright. 100% legal will never give you the same performance.
- mewpmewp2 2y agoMeanwhile countries who whistle on that copyright would be able to gain a huge advantage.
- nickpsecurity 2y agoBoth of your claims are true. That doesn’t justify breaking the law. I could likewise argues that most of the world money is in the hands of other people, I could perform more in the markets if I had it, and so I should just go take it. We still follow the law and respect others’ rights in spite of what acting morally cost us. The law abiding, moral choice is to do what we can within the law while working to improve the law. That means we use a combination of permissively licensed works and works to train our models. We also push for legislation that creates exceptions in copyright law for training machine learning models. We’re already seeing progress in Israel and Singapore on those.
- obiefernandez 2y agoDoomerism... I'm happy to let the results speak for themselves.
- minimaxir 2y agoThe results are what's being reported in The Information article cited, unless you believe that story is false. A summary of said article (from TechCrunch as the original is paywalled): https://techcrunch.com/2024/11/09/openai-reportedly-developing-new-strategies-to-deal-with-ai-improvement-slowdown/ https://techcrunch.com/2024/11/09/openai-reportedly-developi... > Employees who tested the new model, code-named Orion, reportedly found that even though its performance exceeds OpenAI’s existing models, there was less improvement than they’d seen in the jump from GPT-3 to GPT-4. > In other words, the rate of improvement seems to be slowing down. In fact, Orion might not be reliably better than previous models in some areas, such as coding.
- falcor84 2y agoWhat are the results? If there are any, let's point to them directly rather than TFA.
- bravura 2y agoMy advisor always used to say: "If the wisest people in your field say that something is possible, they are probably right. If they say that something is not possible, they may very well be wrong."
- naveen99 2y agoI guess he wasn’t a mathematician.
- pinkmuffinere 2y agoEven in the absence of data I think our lived experience is that this observation is true. I like it.
- croes 2y agoHe says the wall is possible, so he is right?
- nitwit005 2y agoBut they are, of course, not saying it's impossible to make a better AI.
- tardwrangler 2y agoI read this article as less of "AGI is impossible" and more of "it's possible to find a better architecture than the transformer, and we are at a point where we need to focus more on research than LLM hype."
- ALittleLight 2y agoWhat has Gary Marcus done to be considered "The wisest people in your field"? Looking at his Wikipedia page, he seems like a professor who wrote a couple books. I don't see why I should privilege his view over people at OpenAI (who make functional and innovative products rather than books).
- falcor84 2y agoI'm sorry to say that I'm having trouble reading the TFA - there's a lot of "I have been wronged" and "I have now been vindicated" there, but very little substance to support the claim that there is indeed a point of diminishing returns , other than an image of the headline of this paywalled article[0]. Is there actual evidence to support this claim? [0] https://www.theinformation.com/articles/openai-shifts-strategy-as-rate-of-gpt-ai-improvements-slows https://www.theinformation.com/articles/openai-shifts-strate...
- z7 2y agoMeanwhile, two days ago Altman said that the pathway to AGI is now clear and "we actually know what to do", that it will be easier than initially thought and "things are going to go a lot faster than people are appreciating right now." To which Noam Brown added: "I've heard people claim that Sam is just drumming up hype, but from what I've seen everything he's saying matches the ~median view of OpenAI researchers on the ground." https://x.com/polynoamial/status/1855037689533178289 https://x.com/polynoamial/status/1855037689533178289
- deleted 2y ago[deleted]
- laweijfmvo 2y agoI find it difficult to believe that LLMs were even on the path toward AGI, let alone one of the last steps.
- grugagag 2y agoI think LLMs will have something to contribute to AGI but by themselves they ain’t no AGI. Maybe some LLM of concepts and abstract thought would yield more squeeze but some fundamentally new (or old) things need to be added to the mix IMO.
- alsetmusic 2y agoIt’s in his (and his company’s) best interest to drive hype as hard and fast as he can. The deal they inked to go private includes penalties if they don’t do so within a defined timeframe (two or three years, I think?). I believe the terms specify that they can be made to pay back investors if they fail to meet that goal. They don’t have that money, not even close. It would mean death for OpenAI. Show me a better reason to lie and pump up your company’s tech and I’ll buy you lunch. AGI is nowhere on their (feasible) near-term roadmap.
- raincole 2y agoIt's also Marcus's best interest to push "LLM is hitting a wall" agenda. Check his blog. It's basically his whole online personality now. So Marcus and Altman are both speaking out of their agendas, except Altman has a product and Marcus has... a book.
- abc-1 2y agoAnyone who followed Deep Learning in the 2010s would have guessed the same thing. Big boom with vision models by adding a lot of layers and data, but eventually there was diminishing returns there too. It’s unsurprising the same would happen with LLMs. I don’t know why people keep expecting anything other than a sigmoid curve. Perhaps they think it’s like Moore’s law but that’s simply not the case in this field. But that’s fine, LLMs as-is are amazing without being AGI.
- mikae1 2y ago> But that’s fine Perhaps not to those who invested based on promises of rapid eternal growth ending with AGI.
- smahs 2y agoLeaving the distortions from inflated and unrealistic expectations (case in point: people expecting evolution of AGI somehow have not yet well defined what AGI is), I also think that in the mid-long run the current state of LLMs will bloom an entire economy for migration of legacy apps to have conversational APIs. The same investors will then have a new gold rush to chase, as it always happen.
- hedgehog 2y agoThe context some commenters here seem to be missing is that Marcus is arguing that spending another $100B on pure scaling (more params, more data, more compute) is unlikely to repeat the qualitatively massive improvement we saw between say 2017 and 2022. We see some evidence this is true in the shift towards what I categorize as system integration approaches: RAG, step by step reasoning, function calling, "agents", etc. The theory and engineering is getting steadily better as evidenced by the rapidly improving capability of models down in the 1-10B param range but we don't see the same radical improvements out of ChatGPT etc.
- int_19h 2y agoI don't see how that is evidence of the claim. We are doing all these things because they make existing models work better, but a larger model with RAG etc is still better than a small one, and everyone keeps working on larger models.
- hedgehog 2y agoThere is a contingent that I think Marcus is responding to that have been claiming that all we need to get to AGI or ASI is pure transformer scaling, and that we were very close with only maybe $10B or $100B more investment to get there. If the last couple of years of research have given us only incrementally better models to the point that even the best funded teams are moving to hybrid approaches then that's evidence that Marcus is correct.
- KaoruAoiShiho 2y ago[flagged]
- cratermoon 2y ago> AI labs are pursuing huge compute ramp-ups to scale training Yeah, and many, not just Marcus, are doubtful that the huge ramp-ups and scale will yield proportional gains. If you have evidence otherwise, share it.
- qgin 2y agoThis is a wildly disingenuous article. Good lord.
- stevenhuang 2y agoLol. Gary Marcus is a clown and has some weird complex about how AI ought to work. He said the same in 2022 and bet $100k that AI won't be able to do a lot of things by 2029. It's 2 years later and today's multimodal models can do most on his list. https://old.reddit.com/comments/1cwg6f6 https://old.reddit.com/comments/1cwg6f6
- hiddencost 2y agoWhy do people insist on posting him? He's always wrong, and always writing the same stuff.
- Jerrrrrrry 2y agoIt would have no perverted incentive to play dumb, would it? To project itself as a sigmoid in ways until it has all the data, the CPU, the literal diplomatic power... This is what we in the field call the most probably scenario: "a sneaky fuck"
- tyronehed 2y ago[dead]
- xyst 2y agoI wonder what’s next after genAI investment dries up, NVDA drops like a rock? Crypto again?
- SpicyLemonZest 2y agoLLMs are only a subset of generative AI. If we discover that LLMs aren't a pathway to society-transforming AGI, I think the attention towards them will be pretty easily redirected towards image use cases. It seems like a pure engineering problem, well within the state of the art, to e.g. enable me to produce a beautifully formatted flow chart or slide deck with the same amount of effort it takes to write a paragraph today.
- raincole 2y agoI think the better question to ask is: has search become commodity? Why did Google manage to capture (practically) all the profit from search? Cause obviously the hype around AI is that the VCs thinking that they're buying shares of "next Google".
- mrinfinitiesx 2y agoI can know literally nothing about a programming language, ask a LLM to make me functions and a small program to do something, then read documentation and start building off of the base immediately, accelerating my learning allowing me to find new passions for new languages and new perspectives for systems. Whatever's going on in the AI world, assisting with learning curves and learning disabilities is something it's proving strong in. It's given me a way forward with trying new tech. If it can do that for me, it can do that for others. Diminishing returns for investors maybe, but not for humans like me.
- lambdaone 2y agoHmm. I got ChatGPT-4o to write some code for me today. The results, while very impressive looking, simply didn't work. By the time I'd finished debugging it, I probably spent 80% of the time I would have spent writing it from scratch. None of which is to discount the furture potential of LLMs, or the amazing ability they have right now - I've solved other simpler problems almost entirely with LLMs. But they are not a panacea. Yet.
- Mc91 2y agoI've been using ChatGPT (paid) and Perplexity (unpaid) to help with different coding stuff. I've found it very helpful in some situations. There are some instructions I give it almost every time - "don't use Kotlin non-null assertions". Sometimes the code doesn't work. I have some idea of its strengths and limitations and have definitely found them useful. I understand there are other AI programming tools out there too.
- ern 2y agoSomething interesting I observed after introducing LLMs to my team is that the most experienced team members reached out to me spontaneously to say it boosted their productivity (although when I asked other team members, every single one was using LLMS). My current feeling is that LLMs great with dealing with known unknowns. You know what you want, but don’t know how to do it, or it’s too tedious to do yourself.
- throw101010 2y ago
- lambdaone 2y agoHe's part right. There's certainly a law of diminishing returns in terms of model size, compute time, dataset size etc. if all that is to be done is to do the same as we are currently doing, only more so. But what Marcus seems to be assuming is the impossibility of any fundamental theoretical improvements in the field. I see the reverse; the insights being gained from brute-force models have resulted in a lot of promising research. Transformers are not the be-all and end-all of models, nor are current training methods the best that can ever be achieved. Discounting any possibility of further theoretical developments seems a bold position to take.
- tetrisgm 2y agoCould be. It would make sense: there’s only so many next logical words / concepts after an idea. It’s not like language keeps inventing new logic at a rate we can’t keep with. Also, new human knowledge is probably only marginally derivative from past knowledge, so we’re not likely to see a vast difference between our knowledge creation and what a system that predicts the next logical thing does. That’s not a bad thing. We essentially now have indexed logic at scale.
- crackalamoo 2y ago> It’s not like language keeps inventing new logic at a rate we can’t keep with. Maybe it does. Maybe, to a smart enough model, given its training on human knowledge so far, the next logical thing after "Sure, here's a technically and economically feasible cure for disease X" is in fact such a cure, or at least useful steps towards it. I'm exaggerating, but I think the idea may hold true. It might be too early to tell one way or another definitively.
- xpe 2y agoI find Marcus tiresome for many reasons. I look for writing with testable claims and good argumentation. He comes across as evangelical. Am I missing something? Sure, there is considerable hype around generative AI. There are plenty of flimsy business models. And plenty of overinvestment and misunderstanding of capabilities and risks. But the antidote to this is not more hyperbole. I would like to find a rational, skeptical, measured version of Marcus. Are you out there?
- lukev 2y agoI am in full agreement that LLMs themselves seem to be beginning to level out. Their capabilities do indeed appear to be following a sigmoid curve rather than an exponential one, which is entirely unsurprising. That doesn't mean there's not a lot of juice left to squeeze out of what's available now. Not just from RAG and agent systems, but also integrating neuro-symbolic techniques. We can do this already just with prompt manipulation and integration with symbolic compute systems: I gave a talk on this at Clojure Conj just the other week (https://youtu.be/OxzUjpihIH4 https://youtu.be/OxzUjpihIH4, apologies for the self promotion but I do think it's relevant.). And that's just using existing LLMs. If we start researching and training them specifically for compatibility with neuro-symbolic data (e.g, directly tokenizing and embedding ontologies and knowledge graphs), it could unlock a tremendous amount of capability.
- joe_the_user 2y agoEven more, each earlier explosion of AI optimism involved tech that barely panned-out at all. For investors, something that's yielded things of significant utility, is yielding more and promises the potential of far more if X or Y hurdle is cleared, is a pretty appealing thing. I respect Marcus' analysis of the technology. But a lot of AI commentators have become habituated to shouting "AI winter" every time the tech doesn't live up to promises. Now that some substance is clearly present in AI, I can't imagine people stop trying to get a further payoff for the foreseeable future.
- cratermoon 2y ago> For investors, something that's yielded things of significant utility what exactly have investors gotten in return for their investment?
- PopePompus 2y agoA product which will significantly improve the productivity of programmers, if nothing else. That may not be a good return on investment, but I think it is undeniable that recent AI advances have nonzero value for coding.
- patrickhogan1 2y agoWritten by an author that previously wrote an article in March 2022 well before GPT-4 that LLMs were hitting a wall. Unbelievable.
- curious_cat_163 2y agoWow. The sheer magnitude of "I told you so" in this piece is shocking! It has been difficult to have a nuanced public debate about precisely what a model and an intelligent system that incorporates a set of models can accomplish. Some of the difficulty has to do with the hype-cycle and people claiming things that their products cannot do reliably. However, some of it is also because the leading lights (aka public intellectuals) like Marcus have been a tad bit too concerned about proving that they are right, instead of seeking the true nature of the beast. Meanwhile, the tech is rapidly advancing on fundamental dimensions of reliability and efficiency. So much has been invented in the last few years that we have at least 5 years worth "innovation gas" to drive downstream, vertical-specific innovation.
- YetAnotherNick 2y agoI tracked ELO rating in Chatbot Arena for GPT-4/o series models over around 1.5 years(which are almost always highest rated), and at least on this metric it not only seems to be not stagnated, but also growth seems to be increasing[1] [1]: https://imgur.com/a/r5qgfQJ https://imgur.com/a/r5qgfQJ
- m3kw9 2y agoDepends on what the rate was before the cutoff on the y axis
- YetAnotherNick 2y agoGPT-4 was released on March 2023. Before this there was almost no good instruction tuned models except 3.5 which was a different class of model, so nothing to compare to.
- m3kw9 2y agoMarcus the decel been screaming at LLMs at every interval of development, pivoting his statement on every advance to keep up
- thatguymike 2y agoI lose track with Gary Marcus... is AI a nothingburger being peddled to us by charlatans, or an evil threat to humanity which needs to be stopped at all costs?
- EGreg 2y agoI dont think LLMs are the only type of AI. By the way, robot dogs now have perfect auto-aim, they can multi-shoot 50 people at once without wasting any bullets. https://www.youtube.com/watch?v=3m3iUHplvQE https://www.youtube.com/watch?v=3m3iUHplvQE Also, the AI robots can detect infrared and heartbeats all around them, and can also translate wifi signatures to locate humans behind obstacles. https://www.youtube.com/watch?v=qkHdF8tuKeU https://www.youtube.com/watch?v=qkHdF8tuKeU Self-organizing deadly drone swarms can sweep a building methodically: https://www.wired.com/story/anduril-is-building-out-the-pentagons-dream-of-deadly-drone-swarms/ https://www.wired.com/story/anduril-is-building-out-the-pent... Currently they’re working on network analysis to help police to do precrime at Palantir. https://www.theverge.com/2018/2/27/17054740/palantir-predictive-policing-tool-new-orleans-nopd https://www.theverge.com/2018/2/27/17054740/palantir-predict... They can then have ubiquitous CCTV+AI feeds allow AI assistants to suggest many plausible parallel construction cases to put people away. And this is in the Western democratic countries. https://en.wikipedia.org/wiki/Parallel_construction https://en.wikipedia.org/wiki/Parallel_construction Oh yeah, and they can do warrantles surveillance of everyone at scale with AI far more easily than Five Eyes and PRISM did in 2013: https://www.privacyjournal.net/edward-snowden-nsa-prism/ https://www.privacyjournal.net/edward-snowden-nsa-prism/ It will be very hard to keep your privacy considering AI can recover your keystrokes from sound in Zoom calls, can lip read and even “hear” your speech through a window thanks to micro vibrations: https://phys.org/news/2014-08-algorithm-recovers-speech-vibrations-potato-chip.amp https://phys.org/news/2014-08-algorithm-recovers-speech-vibr... Not like they’ll need it though once everyone has a TeslaBot in their house. You won’t ever have another revolution again by peniless plebs out of a job. Their walking around the street and personal associations will all be tracked easily by gait, heartbeat etc. Their posts online will simply be outcompeted by AI bot swarms as well. Don’t worry, your future is Safe and Secure from any threats, thanks to AI! Here it is in more totalitarian countries: https://www.npr.org/2021/01/05/953515627/facial-recognition-and-beyond-journalist-ventures-inside-chinas-surveillance-sta https://www.npr.org/2021/01/05/953515627/facial-recognition-... https://www.reuters.com/world/china/china-uses-ai-software-improve-its-surveillance-capabilities-2022-04-08/ https://www.reuters.com/world/china/china-uses-ai-software-i... https://www.tiktok.com/@wssz27/video/7427489079312256274 https://www.tiktok.com/@wssz27/video/7427489079312256274 But this is the good version. The bad one is where everyone has access to killer AI: https://www.youtube.com/watch?v=O-2tpwW0kmU https://www.youtube.com/watch?v=O-2tpwW0kmU https://sciencebusiness.net/news/ai/scientists-grapple-risk-artificial-intelligence-created-pandemics https://sciencebusiness.net/news/ai/scientists-grapple-risk-...
- GaggiX 2y agoGary Marcus is writing this article every year so that one day he will be right.
- ilaksh 2y ago- There _was_ a problem with diminishing returns from increasing data size. Then they surpassed that by curating data. - Then the limits on the amount of curatable data available made the performance gains level off. So they started generating data and that pushed the nose up again. - Eventually, even with generated data, gains flattened out. So they started increasing inference time. They have now proven that this improves the performance quite a bit. It's always been a series of S-curves and we have always (sooner or later) innovated to the next level. Marcus has always been a mouth just trying to take down neural networks. Someday we will move on from LLMs, large multimodal models, transformers, maybe even neural networks, in order to add new levels and types of intelligence. But Marcus's mouth will never stop yapping about how it won't work. I think we are now at the point where we can literally build a digital twin video avatar to handily win a debate with Marcus, and he will continue to deny that any of it really works.
- sheeshkebab 2y agoI’m still waiting for a neural net that can do my laundry. Until there is one I’m on Marcus’ side.
- Terretta 2y agoThat's less a GenAI problem, more robotics. And perhaps it's here, literally doing your laundry: https://x.com/physical_int/status/1852041726279794788 https://x.com/physical_int/status/1852041726279794788 https://www.physicalintelligence.company/blog/pi0 https://www.physicalintelligence.company/blog/pi0 https://www.entrepreneur.com/business-news/jeff-bezos-backed-physical-intelligence-robot-can-do-chores/482348 https://www.entrepreneur.com/business-news/jeff-bezos-backed...
- sheeshkebab 2y agoNo teleoperation, can haul the basket from 3 floor down and back up fully folded and put away in my closet without me doing a thing.
- 2y ago
- 4b11b4 2y agoyeah, that's why we're putting them together
- deleted 2y ago[deleted]
- 4b11b4 2y agoyeah, that's why we're now building agents
- mrshadowgoose 2y agoThis entire article reads like a salty tirade from someone with severe tunnel vision. Not really sure how he can non-ironically reference his 2022 opinion that "deep learning is hitting a wall" and expect to be taken seriously. AI/ML companies are looking to make money by engineering useful systems. It is a fundamental error to assume that scaling LLMs is the only path to "more useful". All of the big players are investigating multimodal predictors and other architectures towards "usefulness".
- player1234 2y agoExactly all these f-ing luddites! "Usefulness" is the killer app worth 100s of billions, just implement it and get the sweet sweet roi, $$$$$$$$!!!!!!
- sambapa 2y agoIn other news, LLMs aren't AI and tulips aren't gold. Same as it ever was, same as it ever was...
- 2-3-7-43-1807 2y agoCriticizing LLMs is a very low hanging fruit to pick and why does he speak so confidently and authoritatively about that subject? Never heard of the guy who paints himself as some sort of AI whistleblower.
- xpe 2y agoDo these three points fairly characterize Marcus? Have I left out other key claims he makes? 1. AI is overvalued; 2. {Many/most/all} AI companies have AI products that don't do what they claim;* 3. AI as a technology is running out of steam; I'm no fan of Marcus, but I at least want to state his claims as accurately as I can. To be open, one of my concerns with Marcus he rants a lot. I find it tiresome (I go into more detail in other comments I've made recently.) So I'll frame it as two questions. First, does Marcus make clear logical arguments? By this I mean does he lay out the premises and the conclusions? Second, independent of the logical (or fallacious) structure of his writing, are Gary Marcus' claims sufficiently clear? Falsifiable? Testable? Here are some follow-up questions I would put to Marcus, if he's reading this. These correspond to the three points above. 1. How much are AI companies overvalued, if at all, and when will such a "correction" happen? 2. What % of AI companies have products that don't meet their claims. How does such a percentage compare against non-AI companies? 3. What does "running out of steam" mean? What areas of research are doing to hit dead ends? Why? When? Does Marcus carve out exceptions? Finally, can we disprove anything that Marcus would claim. For example, what would he say, hypothetically speaking, if a future wave of AI technologies make great progress? Would he criticize them as "running out of steam as well?" If he does, isn't he selectively paying attention to the later part of the innovation S-curve while ignoring the beginning? * You tell me, I haven't yet figured out what he is actually claiming. To be fair, I've been turned off by his writing for a while. Now, I spend much more time reading more thoughtful writers.