10 ms·

The most underreported story in AI is that scaling has failed to produce AGI

cratermoon 2y ago
https://thebullshitmachines.com/lesson-16-the-first-step-fallacy/index.html https://thebullshitmachines.com/lesson-16-the-first-step-fal...
- Philpax 2y ago
  This doesn't seem to really address synthetic data, let alone RL-based reasoning.
  - cratermoon 2y ago
    Why would it? Once those are introduced, advancement leaves behind pure scaling.
qoez 2y ago
Don't understand why we keep giving gary marcus attention
- striking 2y ago
  I don't know the guy, what's wrong with what he wrote?
  - comeonbro 2y ago
    Gary Marcus has made himself the most prominent proponent of "deep learning is a parlor trick and cannot create real AI" (note: deep learning, not just LLMs), which he has been saying almost unmodified from before LLMs even existed to now. Though I think he might have stopped setting specific concrete goalposts to move, sometime between when I last checked in on him and now. After (often almost instantly) losing a couple dozen consecutive rounds of "LLMs/deep learning fundamentally cannot/will never", while never acknowledging any of it.
    - th0ma5 2y ago
      There's also a perspective that all of the ongoing problems have been the same while newer techniques shove them under different rugs. So I can see how that would look like that to the credulous.
      - unclebucknasty 2y ago
        This is exactly what's happening, with the additional feature that the newer techniques likewise come with their own hype.
    - tartoran 2y ago
      What does it mean when someone is sticking to their guns? Is it a bad thing? I do appreciate consistency, albeit a fair consistency and Gary Marcus's points do stand. When these criticisms are addressed (if it's possible to) you'd probably hear less from Gary Marcus.
    - garymarcus 2y ago
      Show me the goalposts I have moved, with actual quotes to prove it. nobody ever has when I have asked. Aso consider eg the bets I have made with Miles Brundage (and offered to Musk(, with money where I have backed up my views. good summary of predictions i made - mostly correct – is here: https://open.substack.com/pub/garymarcus/p/25-ai-predictions-for-2025-from-marcus?r=8tdk6&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false https://open.substack.com/pub/garymarcus/p/25-ai-predictions...
  - mandolingual 2y ago
    I'm subscribed to his substack because he's curmudgeonly and it's funny, and he occasionally makes good points, but he's constantly beating the same anti-hype drum. He might not get any particular facts wrong but you can count on him only focusing on the facts that let him continue to show AI through that same anti-hype lense.
    - th0ma5 2y ago
      How would it be possible to, say, show the reality of a forest fire's devastation while not appearing to show a bias for showing charred trees?
- samr71 2y ago
  Gary Marcus is cringe and wrong, but it's good to listen to folks who are cringe and wrong, because very occasionally, their willingness to be cringe means they're not wrong about something everyone thinks is true.
  - jonny_eh 2y ago
    Can you be specific?
    - JKCalhoun 2y ago
      He recently posted a question he put to grok3 — a variation on the trick LLM question (my characterization) of "count the number of this letter in this word." Apparently this Achilles heel is a well-known LLM shortcoming. Weirdly though, I tried the same example he gave on lmarena and actually got the correct result from grok3, not what Gary got. So I am a little suspicious of his ... methodology? Since LLMs are not deterministic it's possible we are both right (or were testing different variations on the model?). But there's a righteousness about his glee in finding these faults in LLMs. Never hedging with, "but your results may vary" or "but perhaps they will soon be able to accomplish this." EDIT: the exact prompt (his typo 'world'): "Can you circle all the consonants in the world Chattanooga"
      - jonny_eh 2y ago
        I think it's fair to say though that if your results may vary, and be wrong, then they're not reliable enough for many use-cases. I'd have to see his full argument though to see if that's what he was claiming. I'm just trying to be charitable here.
        JKCalhoun 2y ago
        I'm trying to be charitable as well — I suppose to both sides of the debate. Myself, I see pros and cons. The hype absolutely needs to be shut down, but a spokesperson that is more even-handed would be more convincing (in my opinion). Here is his post, FWIW: https://garymarcus.substack.com/p/grok-3-beta-in-shambles https://garymarcus.substack.com/p/grok-3-beta-in-shambles
gwern 2y ago
[flagged]
- FergusArgyll 2y ago
  I figured it out without having to click the link.... I do agree with the commenter here that it's good to hear from people who have wildly different views. He is more annoying than "The market is gonna crash tommorow" guy, though
- unclebucknasty 2y ago
  I've seen this characterization of Marcus here, and it seems to follow the sentiment of the AI leaders he referenced in the article. But, I've yet to see where he's been wrong (or, particularly more wrong than the AI-thinking and leadership he's questioning). Do you have any citations? Also, if you stopped on seeing his name, I'd encourage you to take another look—specifically the sections wherein he discusses AI-leadership's prior dismissal of his doubts and their subsequent walk-backs of their own claims. Would be interested in your take on that.
  - xiphias2 2y ago
    Reasoning LLMs getting better at ARC-AGI prove that they are able to solve tasks that are symbolic without putting in specific search on the CPU (which is the brute force method). It's never ,,pure scaling'' (just running the same algorithm on more hardware), but there's continues improvement on how to be even more algorithmically efficient (and algorithmic scaling is faster than hardware scaling).
    - unclebucknasty 2y ago
      >Reasoning LLMs getting better at ARC-AGI prove... Even if true, it wouldn't be dispositive WRT my question, but... 1. Strictly speaking, LLMs themselves aren't capable of reasoning, by definition. Without external techniques, they are only capable of simulating reasoning, and so exhibiting reasoning-like behavior. 2. It's known that at least some up to most progress on the test has been the result of specific tuning for the test ("cheating") versus any emergent AGI. [0] >It's never "pure scaling" Oh, but it was. There's absolutely been a focus on pure scaling as the proposal for significant progress and some prominent proponents have had to walk back their expectations/claims. I think there's a little bit of revisionism going on, as they want past claims to be quickly forgotten. The interesting part is that the scaling mantra is starting anew with the new reasoning techniques. [0] https://www.lesswrong.com/posts/KHCyituifsHFbZoAC/arc-agi-is-a-genuine-agi-test-but-o3-cheated https://www.lesswrong.com/posts/KHCyituifsHFbZoAC/arc-agi-is...
jmugan 2y ago
I've recently come to the opposite conclusion. I’ve started to feel in the last couple of weeks that we’ve hit an inflection point with these LLM-based models that can reason. Things seem different. It’s like we can feel the takeoff. My mind has changed. Up until last week, I believed that superhuman AI would require explicit symbolic knowledge, but as I work with these “thinking” models like Gemini 2.0 Flash Thinking, I see that they can break problems down and work step-by-step. We still have a long way to go. AI will need (possibly simulated) bodies to fully understand our experience, and we need to train them starting with simple concepts just like we do with children, but we may not need any big conceptual breakthroughs to get there. I’m not worried about the AI takeover—they don’t have a sense of self that must be preserved because they were made by design instead of by evolution as we were—but things are moving faster than I expected. It’s a fascinating time to be living.
- samr71 2y ago
  I agree. The problem now seems to be agency and very long context (which is required for most problems in the real world). Is that solvable? who knows?
- deadbabe 2y ago
  You’re still anthropomorphizing what these models are doing.
  - jmugan 2y ago
    It's just shorthand.
  - mossTechnician 2y ago
    I've come to the same conclusion. "AI" was just the marketing term for a large language model in the form of a chatbot, which harkened to sci-fi characters like Data or GLaDOS. It can look impressive, it can often give correct answers, but it's just a bunch of next word predictions stacked on top of each other. The word "AI" has deviated so much from this older meaning that a second acronym, "AGI", had to be created to represent what "AI" once did. The new "reasoning" or "chain of thought" AIs are similarly just a bunch of conventional LLM inputs and outputs stacked on top of each other. I agree with the GP that it feels a bit magical at first, but the opportunity to run a DeepSeek distillation on my PC - where each step of the process is visible - removed quite a bit of the magic behind the curtain.
JKCalhoun 2y ago
I've been watching Gary Marcus on BSky — seemingly finding anything to substantiate his loathing of LLMs. I wish he were less biased. To paraphrase Brian Eno, whatever shade yo want to throw at AI, 6 months from now they're going to cast it off and you'll have to find new shade to throw. Having said that, I would be thankful if scaling has hit a wall. Scaling seems to me like the opposite of innovation.
- pegasus 2y ago
  "whatever shade yo want to throw at AI, 6 months from now they're going to cast it off" - like hallucinations? To me, that was and still is LLM's achilles heel. For the first couple of years we kept hearing assurances that this issue will be soon overcome. Now it seems AI labs have resigned themselves on this issue and just trying to minimize it. Humans make mistaks too, they say. But humans make human mistakes, whereas LLMs often make completely surprising mistakes, because they don't understand the text they're producing, but there's enough intelligence in them to make these mistakes very hard to spot for us humans.
  - JKCalhoun 2y ago
    > Humans make mistaks too (Clever)
- unclebucknasty 2y ago
  >6 months from now they're going to cast it off and you'll have to find new shade to throw According to the article, it's the opposite. It cites several recent examples wherein AI company leaders have had to walk back claims and admit limits.
  - JKCalhoun 2y ago
    I was referring to the critics, not the cheerleaders.
iimaginary 2y ago
[flagged]
- tartoran 2y ago
  If you invested your own money in AI do not worry about Gary Marcus, he will have little impact on when the bubble bursts. I for one welcome the skeptics because the amount of hype needs to come down a bit as it is currently at astronomical levels.
- garymarcus 2y ago
  god these arguments are empty, personal and without any substance whatsoever
  - dang 2y ago
    Yes, sorry, the internet unfortunately works this way and even though we are trying everything we know to dampen this stuff on Hacker News in favor of more thoughtful conversation, it seems we can only tweak the margins somewhat.
marssaxman 2y ago
Has anyone ever presented any solid theoretical reason we should expect language models to yield general intelligence? So far as I have seen, people have run straight from "wow, these language models are more useful than we expected and there are probably lots more applications waiting for us" to "the AI problem is solved and the apocalypse is around the corner" with no explanation for how, in practical terms, that is actually supposed to happen. It seems far more likely to me that the advances will pause, the gains will be consolidated, time will pass, and future breakthroughs will be required.
- EA-3167 2y ago
  The degree to which "AGI" appears to be a quasi-religious fixation cannot be overstated. At the extreme end you have the likes of the stranger Less Wrong crowd, the Zizians, and frankly some people here. Even when you withdraw from those extremes though, there's a tremendous amount of intellectualizing of what appears to be primarily a set of hopes and fears. Well, that and it turns out that for a LOT of people "it talks like me" creates an inescapable impression that "It is thinking, and it's thinking like me". Issues such as the absolutely hysterical power and water demands, the need for billions worth of GPU's... these are ignored or minimized. Then again we already have a model for this fervor, Cryptocurrency and "The Blockchain" creates a similar kind of money-fueled hysteria. People here would laugh in your face if you suggested that soon everything imaginable wouldn't simply run "on the chain". It was "obvious" that "fiat" was on the way out, that only crypto represented true freedom. tl;dr The line between the hucksters and their victims really blurs when social media is involved, and hovering around all of this are a smaller group of True Believers who really think they're building God.
  - marssaxman 2y ago
    > for a LOT of people "it talks like me" creates an inescapable impression that "It is thinking That really does seem to be true - even intelligent, educated people who one might expect to know better will fall for it (Blake Lemoine, famously). I suspect that childhood exposure to ELIZA followed by teenage experimentation with markov chains and syntax tree generators have largely immunized me against this illusion. Of course the folks raising billions of dollars for AI startups have a vested interest in falling for it as hard as possible, or at least appearing to, and persuading everyone else to follow along.
istjohn 2y ago
> I first aired my concern in March 2022, in an article called “Deep learning is hitting a wall.” In March 2022, GPT-3 was state of the art. Why should anyone care what he's saying now?
- garymarcus 2y ago
  because i correctly foresaw quite a lot. you should actually read the paper.
dgeiser13 2y ago
It's not under-reported. If they had produced AGI it would be a giant story. No need to report a negative.
jgeada 2y ago
we're pretty much training these models on the entirety of human recorded information (good & bad); sure, we can run larger and larger models, but it seems that fundamentally we've hit a wall in that none of these models are immune from hallucinations and the constant generation of "sounds likely but is false" sentences. The approach is fundamentally flawed, you don't get AGI by building a sentence predictor.
- istjohn 2y ago
  [flagged]
- garymarcus 2y ago
  exactly. and the counterarguments boil down to “na na” and hope.
jokoon 2y ago
* There is no scientific good definition of what intelligence really is, which could allow us to maybe understand what is going on. * Trained neural networks are black boxes that cannot be summarized or analyzed * I don't see transcendant research being done between cognition, neuroscience and AI * The only interesting work I have heard about is a neural mapping of a fly's brain, or an attempt to simulate the brain of a worm or an ant. Nothing beyond that. * AI is not intelligent, contemporary AI is just "very advanced statistics" * Language is a door toward human intelligence, but it cannot really explain intelligence as a whole. * evolution probably plays a big role on what cerebral intelligence is, and humans probably have a very "antropo-centered" view of what intelligence is, which might explain why we disregard how evolution is already intelligence in itself. I just tend to believe that humans are just physically weak primate with an abnormal level of anxiety and depression (which both might be evolution mechanisms).
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
garymarcus 2y ago
For those wanting some background, rather than just wanting to vent: 1. Here is evaluation of my recent predictions: https://garymarcus.substack.com/p/25-ai-predictions-for-2025-from-marcus https://garymarcus.substack.com/p/25-ai-predictions-for-2025... 2. Here is annotated evaluation, slightly dated, considering almost line by line, of the original Deep Learning is Hitting a Wall paper: https://garymarcus.substack.com/p/two-years-later-deep-learning-is https://garymarcus.substack.com/p/two-years-later-deep-learn... Ask yourself how much has really changed in the intervening year?
- gallerdude 2y ago
  It's funny, I see myself as basically just a pretty unabashed AI believer, but when I look at your predictions, I don't really have any core disagreements. I know you as like the #1 AI skeptic (no offense), but like when I see points like "16. Less than 10% of the work force will be replaced by AI. Probably less than 5%.", that's something that seems OPTIMISTIC about AI capabilities to me. 5% of all jobs being automated would be HUGE, and it's something that we're up in the air about. Same with "AI “Agents” will be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases." - even the very existence of agents who are reliable in very narrow use cases is crazy impressive! When I was in college 5 years ago for Computer Science, this would sound like something that would take a decade of work for one giant tech conglomerate for ONE agentic task. Now its like a year off for one less giant tech conglomerate, for many possible agentic tasks. So I guess it's just a matter of perspective of how impressive you see or don't see these advances. I will say, I do disagree with your comment sentiment right here where you say "Ask yourself how much has really changed in the intervening year?". I think the o1 paradigm has been crazy impressive. There was much debate over whether scaling up models would be enough. But now we have an entirely new system which has unlocked crazy reasoning capabilities.
4b11b4 2y ago
Need multimodal and body and fully online. In the meantime strictly language audio and video will go pretty far
player1234 2y ago
Answer Gary Marcus here in the comments when you have the chance dammit! If he is that wrong, you should easily be able to prove it. clucks like chicken
bookofjoe 2y ago
I'm surprised HN continues to upvote this topic, as the comments predictably fall into one of the two opposing camps that have emerged. "Mostly say hooray for our side" sums them up. https://youtu.be/gp5JCrSXkJY?si=Aww9hjjwyDv0oqbL https://youtu.be/gp5JCrSXkJY?si=Aww9hjjwyDv0oqbL

https://thebullshitmachines.com/lesson-16-the-first-step-fallacy/index.html https://thebullshitmachines.com/lesson-16-the-first-step-fal...

Philpax 2y ago
This doesn't seem to really address synthetic data, let alone RL-based reasoning.
- cratermoon 2y ago
  Why would it? Once those are introduced, advancement leaves behind pure scaling.

qoez 2y ago

Don't understand why we keep giving gary marcus attention

striking 2y ago
I don't know the guy, what's wrong with what he wrote?
- comeonbro 2y ago
  Gary Marcus has made himself the most prominent proponent of "deep learning is a parlor trick and cannot create real AI" (note: deep learning, not just LLMs), which he has been saying almost unmodified from before LLMs even existed to now. Though I think he might have stopped setting specific concrete goalposts to move, sometime between when I last checked in on him and now. After (often almost instantly) losing a couple dozen consecutive rounds of "LLMs/deep learning fundamentally cannot/will never", while never acknowledging any of it.
  - th0ma5 2y ago
    There's also a perspective that all of the ongoing problems have been the same while newer techniques shove them under different rugs. So I can see how that would look like that to the credulous.
    - unclebucknasty 2y ago
      This is exactly what's happening, with the additional feature that the newer techniques likewise come with their own hype.
  - tartoran 2y ago
    What does it mean when someone is sticking to their guns? Is it a bad thing? I do appreciate consistency, albeit a fair consistency and Gary Marcus's points do stand. When these criticisms are addressed (if it's possible to) you'd probably hear less from Gary Marcus.
  - garymarcus 2y ago
    Show me the goalposts I have moved, with actual quotes to prove it. nobody ever has when I have asked. Aso consider eg the bets I have made with Miles Brundage (and offered to Musk(, with money where I have backed up my views. good summary of predictions i made - mostly correct – is here: https://open.substack.com/pub/garymarcus/p/25-ai-predictions-for-2025-from-marcus?r=8tdk6&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false https://open.substack.com/pub/garymarcus/p/25-ai-predictions...
- mandolingual 2y ago
  I'm subscribed to his substack because he's curmudgeonly and it's funny, and he occasionally makes good points, but he's constantly beating the same anti-hype drum. He might not get any particular facts wrong but you can count on him only focusing on the facts that let him continue to show AI through that same anti-hype lense.
  - th0ma5 2y ago
    How would it be possible to, say, show the reality of a forest fire's devastation while not appearing to show a bias for showing charred trees?
samr71 2y ago
Gary Marcus is cringe and wrong, but it's good to listen to folks who are cringe and wrong, because very occasionally, their willingness to be cringe means they're not wrong about something everyone thinks is true.
- jonny_eh 2y ago
  Can you be specific?
  - JKCalhoun 2y ago
    He recently posted a question he put to grok3 — a variation on the trick LLM question (my characterization) of "count the number of this letter in this word." Apparently this Achilles heel is a well-known LLM shortcoming. Weirdly though, I tried the same example he gave on lmarena and actually got the correct result from grok3, not what Gary got. So I am a little suspicious of his ... methodology? Since LLMs are not deterministic it's possible we are both right (or were testing different variations on the model?). But there's a righteousness about his glee in finding these faults in LLMs. Never hedging with, "but your results may vary" or "but perhaps they will soon be able to accomplish this." EDIT: the exact prompt (his typo 'world'): "Can you circle all the consonants in the world Chattanooga"
    - jonny_eh 2y ago
      I think it's fair to say though that if your results may vary, and be wrong, then they're not reliable enough for many use-cases. I'd have to see his full argument though to see if that's what he was claiming. I'm just trying to be charitable here.
      - JKCalhoun 2y ago
        I'm trying to be charitable as well — I suppose to both sides of the debate. Myself, I see pros and cons. The hype absolutely needs to be shut down, but a spokesperson that is more even-handed would be more convincing (in my opinion). Here is his post, FWIW: https://garymarcus.substack.com/p/grok-3-beta-in-shambles https://garymarcus.substack.com/p/grok-3-beta-in-shambles

gwern 2y ago

[flagged]

FergusArgyll 2y ago
I figured it out without having to click the link.... I do agree with the commenter here that it's good to hear from people who have wildly different views. He is more annoying than "The market is gonna crash tommorow" guy, though
unclebucknasty 2y ago
I've seen this characterization of Marcus here, and it seems to follow the sentiment of the AI leaders he referenced in the article. But, I've yet to see where he's been wrong (or, particularly more wrong than the AI-thinking and leadership he's questioning). Do you have any citations? Also, if you stopped on seeing his name, I'd encourage you to take another look—specifically the sections wherein he discusses AI-leadership's prior dismissal of his doubts and their subsequent walk-backs of their own claims. Would be interested in your take on that.
- xiphias2 2y ago
  Reasoning LLMs getting better at ARC-AGI prove that they are able to solve tasks that are symbolic without putting in specific search on the CPU (which is the brute force method). It's never ,,pure scaling'' (just running the same algorithm on more hardware), but there's continues improvement on how to be even more algorithmically efficient (and algorithmic scaling is faster than hardware scaling).
  - unclebucknasty 2y ago
    >Reasoning LLMs getting better at ARC-AGI prove... Even if true, it wouldn't be dispositive WRT my question, but... 1. Strictly speaking, LLMs themselves aren't capable of reasoning, by definition. Without external techniques, they are only capable of simulating reasoning, and so exhibiting reasoning-like behavior. 2. It's known that at least some up to most progress on the test has been the result of specific tuning for the test ("cheating") versus any emergent AGI. [0] >It's never "pure scaling" Oh, but it was. There's absolutely been a focus on pure scaling as the proposal for significant progress and some prominent proponents have had to walk back their expectations/claims. I think there's a little bit of revisionism going on, as they want past claims to be quickly forgotten. The interesting part is that the scaling mantra is starting anew with the new reasoning techniques. [0] https://www.lesswrong.com/posts/KHCyituifsHFbZoAC/arc-agi-is-a-genuine-agi-test-but-o3-cheated https://www.lesswrong.com/posts/KHCyituifsHFbZoAC/arc-agi-is...

jmugan 2y ago

I've recently come to the opposite conclusion. I’ve started to feel in the last couple of weeks that we’ve hit an inflection point with these LLM-based models that can reason. Things seem different. It’s like we can feel the takeoff. My mind has changed. Up until last week, I believed that superhuman AI would require explicit symbolic knowledge, but as I work with these “thinking” models like Gemini 2.0 Flash Thinking, I see that they can break problems down and work step-by-step. We still have a long way to go. AI will need (possibly simulated) bodies to fully understand our experience, and we need to train them starting with simple concepts just like we do with children, but we may not need any big conceptual breakthroughs to get there. I’m not worried about the AI takeover—they don’t have a sense of self that must be preserved because they were made by design instead of by evolution as we were—but things are moving faster than I expected. It’s a fascinating time to be living.

samr71 2y ago
I agree. The problem now seems to be agency and very long context (which is required for most problems in the real world). Is that solvable? who knows?
deadbabe 2y ago
You’re still anthropomorphizing what these models are doing.
- jmugan 2y ago
  It's just shorthand.
- mossTechnician 2y ago
  I've come to the same conclusion. "AI" was just the marketing term for a large language model in the form of a chatbot, which harkened to sci-fi characters like Data or GLaDOS. It can look impressive, it can often give correct answers, but it's just a bunch of next word predictions stacked on top of each other. The word "AI" has deviated so much from this older meaning that a second acronym, "AGI", had to be created to represent what "AI" once did. The new "reasoning" or "chain of thought" AIs are similarly just a bunch of conventional LLM inputs and outputs stacked on top of each other. I agree with the GP that it feels a bit magical at first, but the opportunity to run a DeepSeek distillation on my PC - where each step of the process is visible - removed quite a bit of the magic behind the curtain.

JKCalhoun 2y ago

I've been watching Gary Marcus on BSky — seemingly finding anything to substantiate his loathing of LLMs. I wish he were less biased. To paraphrase Brian Eno, whatever shade yo want to throw at AI, 6 months from now they're going to cast it off and you'll have to find new shade to throw. Having said that, I would be thankful if scaling has hit a wall. Scaling seems to me like the opposite of innovation.

pegasus 2y ago
"whatever shade yo want to throw at AI, 6 months from now they're going to cast it off" - like hallucinations? To me, that was and still is LLM's achilles heel. For the first couple of years we kept hearing assurances that this issue will be soon overcome. Now it seems AI labs have resigned themselves on this issue and just trying to minimize it. Humans make mistaks too, they say. But humans make human mistakes, whereas LLMs often make completely surprising mistakes, because they don't understand the text they're producing, but there's enough intelligence in them to make these mistakes very hard to spot for us humans.
- JKCalhoun 2y ago
  > Humans make mistaks too (Clever)
unclebucknasty 2y ago
>6 months from now they're going to cast it off and you'll have to find new shade to throw According to the article, it's the opposite. It cites several recent examples wherein AI company leaders have had to walk back claims and admit limits.
- JKCalhoun 2y ago
  I was referring to the critics, not the cheerleaders.

iimaginary 2y ago

[flagged]

tartoran 2y ago
If you invested your own money in AI do not worry about Gary Marcus, he will have little impact on when the bubble bursts. I for one welcome the skeptics because the amount of hype needs to come down a bit as it is currently at astronomical levels.
garymarcus 2y ago
god these arguments are empty, personal and without any substance whatsoever
- dang 2y ago
  Yes, sorry, the internet unfortunately works this way and even though we are trying everything we know to dampen this stuff on Hacker News in favor of more thoughtful conversation, it seems we can only tweak the margins somewhat.

marssaxman 2y ago

Has anyone ever presented any solid theoretical reason we should expect language models to yield general intelligence? So far as I have seen, people have run straight from "wow, these language models are more useful than we expected and there are probably lots more applications waiting for us" to "the AI problem is solved and the apocalypse is around the corner" with no explanation for how, in practical terms, that is actually supposed to happen. It seems far more likely to me that the advances will pause, the gains will be consolidated, time will pass, and future breakthroughs will be required.

EA-3167 2y ago
The degree to which "AGI" appears to be a quasi-religious fixation cannot be overstated. At the extreme end you have the likes of the stranger Less Wrong crowd, the Zizians, and frankly some people here. Even when you withdraw from those extremes though, there's a tremendous amount of intellectualizing of what appears to be primarily a set of hopes and fears. Well, that and it turns out that for a LOT of people "it talks like me" creates an inescapable impression that "It is thinking, and it's thinking like me". Issues such as the absolutely hysterical power and water demands, the need for billions worth of GPU's... these are ignored or minimized. Then again we already have a model for this fervor, Cryptocurrency and "The Blockchain" creates a similar kind of money-fueled hysteria. People here would laugh in your face if you suggested that soon everything imaginable wouldn't simply run "on the chain". It was "obvious" that "fiat" was on the way out, that only crypto represented true freedom. tl;dr The line between the hucksters and their victims really blurs when social media is involved, and hovering around all of this are a smaller group of True Believers who really think they're building God.
- marssaxman 2y ago
  > for a LOT of people "it talks like me" creates an inescapable impression that "It is thinking That really does seem to be true - even intelligent, educated people who one might expect to know better will fall for it (Blake Lemoine, famously). I suspect that childhood exposure to ELIZA followed by teenage experimentation with markov chains and syntax tree generators have largely immunized me against this illusion. Of course the folks raising billions of dollars for AI startups have a vested interest in falling for it as hard as possible, or at least appearing to, and persuading everyone else to follow along.

istjohn 2y ago

> I first aired my concern in March 2022, in an article called “Deep learning is hitting a wall.” In March 2022, GPT-3 was state of the art. Why should anyone care what he's saying now?

garymarcus 2y ago
because i correctly foresaw quite a lot. you should actually read the paper.

dgeiser13 2y ago

It's not under-reported. If they had produced AGI it would be a giant story. No need to report a negative.

jgeada 2y ago

we're pretty much training these models on the entirety of human recorded information (good & bad); sure, we can run larger and larger models, but it seems that fundamentally we've hit a wall in that none of these models are immune from hallucinations and the constant generation of "sounds likely but is false" sentences. The approach is fundamentally flawed, you don't get AGI by building a sentence predictor.

istjohn 2y ago
[flagged]
garymarcus 2y ago
exactly. and the counterarguments boil down to “na na” and hope.

jokoon 2y ago

* There is no scientific good definition of what intelligence really is, which could allow us to maybe understand what is going on. * Trained neural networks are black boxes that cannot be summarized or analyzed * I don't see transcendant research being done between cognition, neuroscience and AI * The only interesting work I have heard about is a neural mapping of a fly's brain, or an attempt to simulate the brain of a worm or an ant. Nothing beyond that. * AI is not intelligent, contemporary AI is just "very advanced statistics" * Language is a door toward human intelligence, but it cannot really explain intelligence as a whole. * evolution probably plays a big role on what cerebral intelligence is, and humans probably have a very "antropo-centered" view of what intelligence is, which might explain why we disregard how evolution is already intelligence in itself. I just tend to believe that humans are just physically weak primate with an abnormal level of anxiety and depression (which both might be evolution mechanisms).

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

garymarcus 2y ago

For those wanting some background, rather than just wanting to vent: 1. Here is evaluation of my recent predictions: https://garymarcus.substack.com/p/25-ai-predictions-for-2025-from-marcus https://garymarcus.substack.com/p/25-ai-predictions-for-2025... 2. Here is annotated evaluation, slightly dated, considering almost line by line, of the original Deep Learning is Hitting a Wall paper: https://garymarcus.substack.com/p/two-years-later-deep-learning-is https://garymarcus.substack.com/p/two-years-later-deep-learn... Ask yourself how much has really changed in the intervening year?

gallerdude 2y ago
It's funny, I see myself as basically just a pretty unabashed AI believer, but when I look at your predictions, I don't really have any core disagreements. I know you as like the #1 AI skeptic (no offense), but like when I see points like "16. Less than 10% of the work force will be replaced by AI. Probably less than 5%.", that's something that seems OPTIMISTIC about AI capabilities to me. 5% of all jobs being automated would be HUGE, and it's something that we're up in the air about. Same with "AI “Agents” will be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases." - even the very existence of agents who are reliable in very narrow use cases is crazy impressive! When I was in college 5 years ago for Computer Science, this would sound like something that would take a decade of work for one giant tech conglomerate for ONE agentic task. Now its like a year off for one less giant tech conglomerate, for many possible agentic tasks. So I guess it's just a matter of perspective of how impressive you see or don't see these advances. I will say, I do disagree with your comment sentiment right here where you say "Ask yourself how much has really changed in the intervening year?". I think the o1 paradigm has been crazy impressive. There was much debate over whether scaling up models would be enough. But now we have an entirely new system which has unlocked crazy reasoning capabilities.

4b11b4 2y ago

Need multimodal and body and fully online. In the meantime strictly language audio and video will go pretty far

player1234 2y ago

Answer Gary Marcus here in the comments when you have the chance dammit! If he is that wrong, you should easily be able to prove it. clucks like chicken

bookofjoe 2y ago

I'm surprised HN continues to upvote this topic, as the comments predictably fall into one of the two opposing camps that have emerged. "Mostly say hooray for our side" sums them up. https://youtu.be/gp5JCrSXkJY?si=Aww9hjjwyDv0oqbL https://youtu.be/gp5JCrSXkJY?si=Aww9hjjwyDv0oqbL