5 ms·

I think his claim basically boils down to "if you're expecting AI, LLMs don't cut it". And I think he's basically right on that count. There's a lot of tooling

I think his claim basically boils down to "if you're expecting AI, LLMs don't cut it". And I think he's basically right on that count. There's a lot of tooling and harnessing being put in place to course correct them on the job, and from the other angle standards are simply being lowered to accommodate them. So they can be made to be useful, but they're still not what you would want from an actual AI. Marcus wants to augment them with symbolic AI. I don't know how feasible that is, but he's not fundamentally against AI, he's just against the notion that LLMs are AI. Which given how they've been marketed and how the public is encouraged to think about them, is a worthwhile point to make.

logicprog 7mo ago
I used to be a Gary Marcus fan, but I guess what confuses me is... I'm not really sure at that point what 'actual' AI means? It seems like the definition of actual AI is something like perfect AI — it has to be fully observable, interpretable, reason perfectly, have perfect factual recall, continual learning, infinite context windows, perfect instruction following, and so on. I feel like at that point, maybe nothing could ever be 'actual' AI? We typically use AI to mean some kind of algorithm or program that lets computers do intellectual work that was previously considered to be the exclusive domain of humans, especially if it involves problem solving or pattern matching or reasoning. Just look at Donald Knuth's recent posts about what Claude was able to do — seems like AI to me? Yeah, it is in perfect AI, but it's still AI. And it's not clear to me that the imperfections that LLMs have mean that they can't be extremely useful and revolutionary as a form of AI. Yes, they make weird mistakes a lot, and they don't think at all like humans do. But I am of the opinion that there are a lot of forms of intelligence, and human intelligence is just one of them. And every kind of intelligence comes with its own different gamut of continual errors that it will tend to make, blind spots and biases. The fact that LLMs have issues that are different from the form of intelligence humans have and also different from what computers have issues with doesn't discount them from being intelligent to me. I also think the framing of agentic harnesses as being bolted onto LLM's in order to "make them useful", but agentic harness plus LLMs not counting as an AI system itself very odd — I think it's pretty clear to me at least that "the AI", if you want to talk about it, is the neurosymbolic cybernetic feedback system that combines the harness and the LLM. The LLM is only the sort of fuzzy pattern matching logic and creativity core; the harness provides verification feedback loops, the ability to interact with and explore the outside world, the ability to bring in programming language interpreters and so on in order to do more rigid symbolic logic, observability, systems for storing and recalling memory for continual learning, and so on, and I think a lot of these, especially feedback loops, resolve a lot of issues that LLMs seem to inherently face, such as hallucinations. Moreover, LLMs are now substantially trained with writing code and using tools and interacting with the world and existing in harnesses in mind. At this point, I would have to guess that more than half of their training is actually devoted to rewarding them for correctly using all of these symbolic tools and solving problems in a simulated world than just predicting the next token. I also think that LLMs, as a sort of core engine of an agentic harness, are allowing computers to do things we'd never really dreamed they could do before, that symbolic systems by themselves never really achieved, and as I said before, if you're looking for neurosymbolic AI — as Marcus says he is — then this is basically how it's going to have to look unless you want to fall down the expert system rabbit hole again.
- albatross79 7mo ago
  [flagged]
  - logicprog 7mo ago
    That's because of a fundamental perceptual limitation — tokenization. That's like saying that humans aren't intelligent because we can't perceive UV light. That's a shallow gotcha at best, it doesn't get to the heart of the matter at all. Also, models will just write code for that sort of thing now.
    - albatross79 7mo ago
      The heart of the matter is a system that produces correct output in a wide variety of scenarios, including ones outside our scope of experience or understanding. That would be a one definition of AI. LLMs are not that, and I don't understand the desperate insistence that they are. Just accept what it is. A good auto complete is useful, a crappy AI is a marketing scam.
      - logicprog 7mo ago
        > a system that produces correct output in a wide variety of scenarios, including ones outside our scope of experience or understanding Where does this definition come from? I certainly don't agree with it, and I am not sure who does, besides yourself and Marcus. Also it seems that you're saying AI does, in fact, mean 'perfect' AI, basically? > A good auto complete is useful, a crappy AI is a marketing scam. LLM agents do a lot more than auto complete now, and using them less like auto complete and more like 'AI' (via agents) has actually made them more useful and less crappy! Also, I don't think framing how modern RLVR'd LLMs operate now as as auto complete even makes a whole lot of sense in the first place.
        albatross79 7mo ago
        It's a common sense definition that covers the broad features of AI as most people would have imagined it before LLMs came along. If you have a better one I'm open to it. Also, agents are just LLMs with harnessing. Attaching a plow to a horse makes it more useful too, but it doesn't change the intrinsic nature of a horse.
antonvs 7mo ago
> "if you're expecting AI, LLMs don't cut it". And I think he's basically right on that count. This is one of those comments whose truth value depends entirely on a constantly shifting definition of “AI”. The ability of modern models to functionally understand, answer questions, and make recommendations about software codebases is superhuman at this point, relative to most human software developers. What is that, if not artificial intelligence? Perhaps you’re thinking of something more like AGI, but even there the terminology is loaded and ambiguous. The models are general enough to answer questions well on a vast range of subjects, and they exhibit understanding (again, functionally speaking this is true - whether someone wants to call them stochastic parrots is beside the point.) The appellation of “intelligence” applies just as well as in the coding case, it’s artificial, and it’s general. > a worthwhile point to make. I disagree. Without clear, justified definitions, it’s an incoherent, poorly specified point that seems to be driven by a desire to maintain a specific conclusion regardless of the evidence.
- albatross79 7mo ago
  [flagged]
  - logicprog 7mo ago
    I'm not sure what you mean by "everyone knew what AI meant"? It's always been an extremely vague term, since its inception — and LLMs do, actually, enable computers to do things I, at least, would've called basically AGI in 2020.
    - albatross79 7mo ago
      There's nothing vague about it. Intelligence is understanding. I think you're just motivated to be confused because you want to fit this thing into that box and it doesn't really fit. So maybe output is an that matters, are we auto completers, can you be intelligent without knowing, etc etc.
      - logicprog 7mo ago

logicprog 7mo ago

I used to be a Gary Marcus fan, but I guess what confuses me is... I'm not really sure at that point what 'actual' AI means? It seems like the definition of actual AI is something like perfect AI — it has to be fully observable, interpretable, reason perfectly, have perfect factual recall, continual learning, infinite context windows, perfect instruction following, and so on. I feel like at that point, maybe nothing could ever be 'actual' AI? We typically use AI to mean some kind of algorithm or program that lets computers do intellectual work that was previously considered to be the exclusive domain of humans, especially if it involves problem solving or pattern matching or reasoning. Just look at Donald Knuth's recent posts about what Claude was able to do — seems like AI to me? Yeah, it is in perfect AI, but it's still AI. And it's not clear to me that the imperfections that LLMs have mean that they can't be extremely useful and revolutionary as a form of AI. Yes, they make weird mistakes a lot, and they don't think at all like humans do. But I am of the opinion that there are a lot of forms of intelligence, and human intelligence is just one of them. And every kind of intelligence comes with its own different gamut of continual errors that it will tend to make, blind spots and biases. The fact that LLMs have issues that are different from the form of intelligence humans have and also different from what computers have issues with doesn't discount them from being intelligent to me. I also think the framing of agentic harnesses as being bolted onto LLM's in order to "make them useful", but agentic harness plus LLMs not counting as an AI system itself very odd — I think it's pretty clear to me at least that "the AI", if you want to talk about it, is the neurosymbolic cybernetic feedback system that combines the harness and the LLM. The LLM is only the sort of fuzzy pattern matching logic and creativity core; the harness provides verification feedback loops, the ability to interact with and explore the outside world, the ability to bring in programming language interpreters and so on in order to do more rigid symbolic logic, observability, systems for storing and recalling memory for continual learning, and so on, and I think a lot of these, especially feedback loops, resolve a lot of issues that LLMs seem to inherently face, such as hallucinations. Moreover, LLMs are now substantially trained with writing code and using tools and interacting with the world and existing in harnesses in mind. At this point, I would have to guess that more than half of their training is actually devoted to rewarding them for correctly using all of these symbolic tools and solving problems in a simulated world than just predicting the next token. I also think that LLMs, as a sort of core engine of an agentic harness, are allowing computers to do things we'd never really dreamed they could do before, that symbolic systems by themselves never really achieved, and as I said before, if you're looking for neurosymbolic AI — as Marcus says he is — then this is basically how it's going to have to look unless you want to fall down the expert system rabbit hole again.

albatross79 7mo ago
[flagged]
- logicprog 7mo ago
  That's because of a fundamental perceptual limitation — tokenization. That's like saying that humans aren't intelligent because we can't perceive UV light. That's a shallow gotcha at best, it doesn't get to the heart of the matter at all. Also, models will just write code for that sort of thing now.
  - albatross79 7mo ago
    The heart of the matter is a system that produces correct output in a wide variety of scenarios, including ones outside our scope of experience or understanding. That would be a one definition of AI. LLMs are not that, and I don't understand the desperate insistence that they are. Just accept what it is. A good auto complete is useful, a crappy AI is a marketing scam.
    - logicprog 7mo ago
      > a system that produces correct output in a wide variety of scenarios, including ones outside our scope of experience or understanding Where does this definition come from? I certainly don't agree with it, and I am not sure who does, besides yourself and Marcus. Also it seems that you're saying AI does, in fact, mean 'perfect' AI, basically? > A good auto complete is useful, a crappy AI is a marketing scam. LLM agents do a lot more than auto complete now, and using them less like auto complete and more like 'AI' (via agents) has actually made them more useful and less crappy! Also, I don't think framing how modern RLVR'd LLMs operate now as as auto complete even makes a whole lot of sense in the first place.
      - albatross79 7mo ago
        It's a common sense definition that covers the broad features of AI as most people would have imagined it before LLMs came along. If you have a better one I'm open to it. Also, agents are just LLMs with harnessing. Attaching a plow to a horse makes it more useful too, but it doesn't change the intrinsic nature of a horse.

antonvs 7mo ago

> "if you're expecting AI, LLMs don't cut it". And I think he's basically right on that count. This is one of those comments whose truth value depends entirely on a constantly shifting definition of “AI”. The ability of modern models to functionally understand, answer questions, and make recommendations about software codebases is superhuman at this point, relative to most human software developers. What is that, if not artificial intelligence? Perhaps you’re thinking of something more like AGI, but even there the terminology is loaded and ambiguous. The models are general enough to answer questions well on a vast range of subjects, and they exhibit understanding (again, functionally speaking this is true - whether someone wants to call them stochastic parrots is beside the point.) The appellation of “intelligence” applies just as well as in the coding case, it’s artificial, and it’s general. > a worthwhile point to make. I disagree. Without clear, justified definitions, it’s an incoherent, poorly specified point that seems to be driven by a desire to maintain a specific conclusion regardless of the evidence.

albatross79 7mo ago
[flagged]
- logicprog 7mo ago
  I'm not sure what you mean by "everyone knew what AI meant"? It's always been an extremely vague term, since its inception — and LLMs do, actually, enable computers to do things I, at least, would've called basically AGI in 2020.
  - albatross79 7mo ago
    There's nothing vague about it. Intelligence is understanding. I think you're just motivated to be confused because you want to fit this thing into that box and it doesn't really fit. So maybe output is an that matters, are we auto completers, can you be intelligent without knowing, etc etc.
    - logicprog 7mo ago