7 ms·

I had the funny thought that this is exactly what a sentient AI would write "stop looking here, there is nothing to see, move along." :-) I (like vannevar appa

I had the funny thought that this is exactly what a sentient AI would write "stop looking here, there is nothing to see, move along." :-) I (like vannevar apparently) didn't feel Cyc was going anywhere useful, there were ideas there, but not coherent enough to form a credible basis for even a hypothesis of how a system could be constructed that would embody them. I was pretty impressed by McCarthy's blocks world demo, later he and a student formalized some of the rules for creating 'context'[1] for AI to operate within, I continue to think that will be crucial to solving some of the mess that LLMs create. For example, the early failures of LLMs suggesting that you could make salad crunchy by adding rocks was a classic context failure, data from the context of 'humor' and data from the context of 'recipes' intertwined. Because existing models have no context during training, there is nothing in the model that 'tunes' the output based on context. And you get rocks in your salad. [1] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=29935775a3ae302bdaffef9dadc8f090659ea7c5 https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...

musicale 2y ago
> there remains no evidence of its general intelligence This seems like a high bar to reach. We all know that symbolic AI didn't scale as well as LLMs trained on huge amounts of data. However, as you note, it also tried to address many things that LLMs still don't do well.
- adastra22 2y ago
  Such as what? What can GOFAI do well that LLMs still cannot?
  - musicale 2y ago
    "Tried to address" is not the same as "can do well." I was responding to PP, but some other (maybe obvious?) examples are logical reasoning and explainability. As PP suggests, some of the classical symbolic ideas may be applicable or complementary to current approaches.
    - musicale 2y ago
      See also (Lenat's final paper): D. Lenat, G. Marcus, "Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc", https://arxiv.org/abs/2308.04445 https://arxiv.org/abs/2308.04445
  - sgt101 2y ago
    I think logical reasoning - so reasoning about logical problems, especially those with transitive relations like two way implication. A way round this is to get them to write prolog relations and then reason over them... with prolog. This isn't a fail - it's what things like prolog do, and not what things like nns do. If I was asked to solve these problems I would write prolog too. I think quite a lot of planning. I think scheduling - I tried something recently and GPT4 wrote python code which worked for very naive cases but then failed at any scale. Basically though - trusted reasoning. Where you need a precise and correct answer LLM's aren't any good. They fail in the limit. But where you need a generally decent answer they are amazing. You just can't rely on it. Whereas GOFAI you can, because if you couldn't the community thew it out and said it was impossible!
    - miki123211 2y ago
      This has always been the case with ML. ML is good at fuzzy stuff, where you don't have a clear definition of a problem (what is spam? what is porn?), "I know it when I see it", or when you don't have a clear mathematical algorithm to solve the problem (think "distinguishing dogs and cats"). When you have both (think sorting arrays, adding numbers), traditional programming (and that includes Prolog and the symbolic AI stuff) is much better. LLMs will always be much worse than traditional computer programs at adding large numbers, just as traditional computer programs will always be worse at telling whether the person in the image is wearing proper safety equipment. For best results, you need to combine both. Use LLMs for the "fuzzy stuff", converting imprecise English or pictures into JSON, Python, Wolfram, Prolog or some other representation that a computer can understand and work with, and then use the computer to do the rest. Let's say you're trying to calculate how much proteins there are per 100 grams of a product, you have a picture of the label, but the label only gives you proteins per serving and the serving size in imperial units. The naive way most programmers try is to ask an LLM to give them proteins per 100g, which is obviously going to fail in some cases. The correct way is to ask the LLM for whatever unit it likes, and then do the conversion on the backend.
    - adastra22 2y ago
      I guess that's a fine distinction I don't make. If the problem requires the AI to write a prolog program to solve, and it is capable of writing the necessary prolog code, then I don't see the practical or philosophical difference from taking the transitive step and saying the AI solved it. If I asked you to solve an air traffic control problem and you did so by writing prolog, no one would try to claim you weren't capable of solving it. Agentic LLMs can solve complicated reasoning and scheduling problems, by writing special-purpose solutions (which might resemble the things we call GOFAI). It's the nature of AGI--which LLMs assuredly are--that they can solve problems by inventing specialized tools, just as we do.
  - mepian 2y ago
    Theorem proving.
  - YeGoblynQueenne 2y ago
    SAT solving, verification and model checking, automated theorem proving, planning and scheculing, knowledge representation and reasoning. Those are fields of AI research where LLMs have nothing to offer.
    - adastra22 2y ago
      I can ask Claude 3.7 to write me a program that does SAT solving, theorem proving, or scheduling, and it generally gets it right on the first try.
      - mepian 2y ago
        Would you actually use this program for real-world applications of theorem proving, e.g. validating an integrated circuit design before spending millions on its manufacturing?
        adastra22 2y ago
        I do, yes.
      - deleted 2y ago
        [deleted]
      - YeGoblynQueenne 2y ago
        Demonstrate.
        adastra22 2y ago
        It would take you all of 5 seconds to try in Claude yourself. I do this work on a daily basis; I know its value.
        YeGoblynQueenne 2y ago
        Do you mean you create SAT solvers with Claude on a daily basis? What is the use case for that?
- ChuckMcM 2y ago
  This is exactly correct, LLMs did scale with huge data, symbolic AI did not. So why? One of the things I periodically ask people working on LLMs is "what does a 'parameter' represent? The simplistic answer is 'it's a weight in a neural net node' but that doesn't much closer. Consider something like a bloom filter where a '0' bit represents the nth bit of all hashes of strings this filter has not seen. I would be interested in reading a paper that does a good job of explaining what a parameter ends up representing in an LLM model.[1] I suspect that McCarthy was on to something with the context thing. Organic intelligence certainly fails in creative ways without context it would not be disqualifying to have AI fail in similarly spectacular ways. [1] I made a bit of progress on this considering it to be the permeability for progress such that the higher the weight the easier it was to 'pass thorough' this particular neuron but the cyclic nature of the graph makes a purely topological explanation pretty obtuse :-).
  - krackers 2y ago
    >I would be interested in reading a paper that does a good job of explaining what a parameter ends up representing in an LLM model. https://distill.pub/2020/circuits/ https://distill.pub/2020/circuits/ https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...
    - ChuckMcM 2y ago
      That's an interesting paper and worth reading. Not sure it has answered my question but I did learn some things from it that I had not considered. This was the quote I resonated with :-) "... the discoveries we highlight here only capture a small fraction of the mechanisms of the model." It sometimes feels a bit like papers on cellular biology with DNA discussions in which descriptions of the enzymes and proteins involved are insightful but the mechanism that operates the reaction remains opaque.
  - YeGoblynQueenne 2y ago
    >> This is exactly correct, LLMs did scale with huge data, symbolic AI did not. So why? Like the rock salad you're mixing up two disparate contexts here. Symbolic AI like SAT solvers and planners is not trying to learn from data and there's no context in which it has to "scale with huge data". Instead, what modern SAT solvers and planners do is even harder than "scaling with data" - which, after all, today means having imba hardware and using it well. SAT solving and planning can't do that: SAT is NP-complete and planning is PSPACE-complete so it really doesn't matter how much you "scale" your hardware, those are not problems you can solve by scaling, ever. And yet, today both SAT and planning are solved problems. NP complete? Nowadays, that's a piece of cake. There are dedicated solvers for all the classical sub-categories of SAT and modern planners can solve planning problems that require sequences of thousands of actions. Hell, modern planners can even play Atari games from pixels alone, and do very well indeed [1]. So how did symbolic AI manage those feats? Not with bigger computers but precisely with the approach that the article above seems to think has failed to produce any results: heuristic search. In SAT solving, the dominant approach is an algorithm called "Conflict Driven Clause Learning", that is designed to exploit the special structure of SAT problems. In Planning and Scheduling, heuristic search was always used, but work really took off in the '90s when people realised that they could automatically estimate a heuristic cost function from the structure of a planning problem. There are parallel and similar approaches everywhere you look at, in classical AI problems, like verification, theorem proving, etc, and that work has even produced a few Turing awards [2]. But do you hear about that work at all, when you hear about AI research? No, because it works, and so it's not AI. But it works, it runs on normal hardware, it doesn't need "scale" and it doesn't need data. You're measuring the wrong thing with the wrong stick. ____________ [1] Planning with Pixels in (Almost) Real Time: https://arxiv.org/pdf/1801.03354 https://arxiv.org/pdf/1801.03354 Competitive results with humans and RL. Bet you didn't know that. [2] E.g. Pnueli for temporal logic in verification, or Clarke, Emerson and Sifakis, for model checking.
- otabdeveloper4 2y ago
  Well, we haven't tried symbolic AI with huge amounts of data. It's a hard problem. (And ironically this problem is much easier now that we have LLMs to help us clean and massage textual data.)

musicale 2y ago

> there remains no evidence of its general intelligence This seems like a high bar to reach. We all know that symbolic AI didn't scale as well as LLMs trained on huge amounts of data. However, as you note, it also tried to address many things that LLMs still don't do well.

adastra22 2y ago
Such as what? What can GOFAI do well that LLMs still cannot?
- musicale 2y ago
  "Tried to address" is not the same as "can do well." I was responding to PP, but some other (maybe obvious?) examples are logical reasoning and explainability. As PP suggests, some of the classical symbolic ideas may be applicable or complementary to current approaches.
  - musicale 2y ago
    See also (Lenat's final paper): D. Lenat, G. Marcus, "Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc", https://arxiv.org/abs/2308.04445 https://arxiv.org/abs/2308.04445
- sgt101 2y ago
  I think logical reasoning - so reasoning about logical problems, especially those with transitive relations like two way implication. A way round this is to get them to write prolog relations and then reason over them... with prolog. This isn't a fail - it's what things like prolog do, and not what things like nns do. If I was asked to solve these problems I would write prolog too. I think quite a lot of planning. I think scheduling - I tried something recently and GPT4 wrote python code which worked for very naive cases but then failed at any scale. Basically though - trusted reasoning. Where you need a precise and correct answer LLM's aren't any good. They fail in the limit. But where you need a generally decent answer they are amazing. You just can't rely on it. Whereas GOFAI you can, because if you couldn't the community thew it out and said it was impossible!
  - miki123211 2y ago
    This has always been the case with ML. ML is good at fuzzy stuff, where you don't have a clear definition of a problem (what is spam? what is porn?), "I know it when I see it", or when you don't have a clear mathematical algorithm to solve the problem (think "distinguishing dogs and cats"). When you have both (think sorting arrays, adding numbers), traditional programming (and that includes Prolog and the symbolic AI stuff) is much better. LLMs will always be much worse than traditional computer programs at adding large numbers, just as traditional computer programs will always be worse at telling whether the person in the image is wearing proper safety equipment. For best results, you need to combine both. Use LLMs for the "fuzzy stuff", converting imprecise English or pictures into JSON, Python, Wolfram, Prolog or some other representation that a computer can understand and work with, and then use the computer to do the rest. Let's say you're trying to calculate how much proteins there are per 100 grams of a product, you have a picture of the label, but the label only gives you proteins per serving and the serving size in imperial units. The naive way most programmers try is to ask an LLM to give them proteins per 100g, which is obviously going to fail in some cases. The correct way is to ask the LLM for whatever unit it likes, and then do the conversion on the backend.
  - adastra22 2y ago
    I guess that's a fine distinction I don't make. If the problem requires the AI to write a prolog program to solve, and it is capable of writing the necessary prolog code, then I don't see the practical or philosophical difference from taking the transitive step and saying the AI solved it. If I asked you to solve an air traffic control problem and you did so by writing prolog, no one would try to claim you weren't capable of solving it. Agentic LLMs can solve complicated reasoning and scheduling problems, by writing special-purpose solutions (which might resemble the things we call GOFAI). It's the nature of AGI--which LLMs assuredly are--that they can solve problems by inventing specialized tools, just as we do.
- mepian 2y ago
  Theorem proving.
- YeGoblynQueenne 2y ago
  SAT solving, verification and model checking, automated theorem proving, planning and scheculing, knowledge representation and reasoning. Those are fields of AI research where LLMs have nothing to offer.
  - adastra22 2y ago
    I can ask Claude 3.7 to write me a program that does SAT solving, theorem proving, or scheduling, and it generally gets it right on the first try.
    - mepian 2y ago
      Would you actually use this program for real-world applications of theorem proving, e.g. validating an integrated circuit design before spending millions on its manufacturing?
      - adastra22 2y ago
        I do, yes.
    - deleted 2y ago
      [deleted]
    - YeGoblynQueenne 2y ago
      Demonstrate.
      - adastra22 2y ago
        It would take you all of 5 seconds to try in Claude yourself. I do this work on a daily basis; I know its value.
        YeGoblynQueenne 2y ago
        Do you mean you create SAT solvers with Claude on a daily basis? What is the use case for that?
ChuckMcM 2y ago
This is exactly correct, LLMs did scale with huge data, symbolic AI did not. So why? One of the things I periodically ask people working on LLMs is "what does a 'parameter' represent? The simplistic answer is 'it's a weight in a neural net node' but that doesn't much closer. Consider something like a bloom filter where a '0' bit represents the nth bit of all hashes of strings this filter has not seen. I would be interested in reading a paper that does a good job of explaining what a parameter ends up representing in an LLM model.[1] I suspect that McCarthy was on to something with the context thing. Organic intelligence certainly fails in creative ways without context it would not be disqualifying to have AI fail in similarly spectacular ways. [1] I made a bit of progress on this considering it to be the permeability for progress such that the higher the weight the easier it was to 'pass thorough' this particular neuron but the cyclic nature of the graph makes a purely topological explanation pretty obtuse :-).
- krackers 2y ago
  >I would be interested in reading a paper that does a good job of explaining what a parameter ends up representing in an LLM model. https://distill.pub/2020/circuits/ https://distill.pub/2020/circuits/ https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...
  - ChuckMcM 2y ago
    That's an interesting paper and worth reading. Not sure it has answered my question but I did learn some things from it that I had not considered. This was the quote I resonated with :-) "... the discoveries we highlight here only capture a small fraction of the mechanisms of the model." It sometimes feels a bit like papers on cellular biology with DNA discussions in which descriptions of the enzymes and proteins involved are insightful but the mechanism that operates the reaction remains opaque.
- YeGoblynQueenne 2y ago
  >> This is exactly correct, LLMs did scale with huge data, symbolic AI did not. So why? Like the rock salad you're mixing up two disparate contexts here. Symbolic AI like SAT solvers and planners is not trying to learn from data and there's no context in which it has to "scale with huge data". Instead, what modern SAT solvers and planners do is even harder than "scaling with data" - which, after all, today means having imba hardware and using it well. SAT solving and planning can't do that: SAT is NP-complete and planning is PSPACE-complete so it really doesn't matter how much you "scale" your hardware, those are not problems you can solve by scaling, ever. And yet, today both SAT and planning are solved problems. NP complete? Nowadays, that's a piece of cake. There are dedicated solvers for all the classical sub-categories of SAT and modern planners can solve planning problems that require sequences of thousands of actions. Hell, modern planners can even play Atari games from pixels alone, and do very well indeed [1]. So how did symbolic AI manage those feats? Not with bigger computers but precisely with the approach that the article above seems to think has failed to produce any results: heuristic search. In SAT solving, the dominant approach is an algorithm called "Conflict Driven Clause Learning", that is designed to exploit the special structure of SAT problems. In Planning and Scheduling, heuristic search was always used, but work really took off in the '90s when people realised that they could automatically estimate a heuristic cost function from the structure of a planning problem. There are parallel and similar approaches everywhere you look at, in classical AI problems, like verification, theorem proving, etc, and that work has even produced a few Turing awards [2]. But do you hear about that work at all, when you hear about AI research? No, because it works, and so it's not AI. But it works, it runs on normal hardware, it doesn't need "scale" and it doesn't need data. You're measuring the wrong thing with the wrong stick. ____________ [1] Planning with Pixels in (Almost) Real Time: https://arxiv.org/pdf/1801.03354 https://arxiv.org/pdf/1801.03354 Competitive results with humans and RL. Bet you didn't know that. [2] E.g. Pnueli for temporal logic in verification, or Clarke, Emerson and Sifakis, for model checking.
otabdeveloper4 2y ago
Well, we haven't tried symbolic AI with huge amounts of data. It's a hard problem. (And ironically this problem is much easier now that we have LLMs to help us clean and massage textual data.)