7 ms·

> "4. Prolog is good at solving reasoning problems." Plain Prolog's way of solving reasoning problems is effectively: for person in [martha, brian, sarah,

> "4. Prolog is good at solving reasoning problems." Plain Prolog's way of solving reasoning problems is effectively: for person in [martha, brian, sarah, tyrone]: if timmy.parent == person: print "solved!" You hard code some options, write a logical condition with placeholders, and Prolog brute-forces every option in every placeholder. It doesn't do reasoning. Arguably it lets a human express reasoning problems better than other languages by letting you write high level code in a declarative way, instead of allocating memory and choosing data types and initializing linked lists and so on, so you can focus on the reasoning, but that is no benefit to an LLM which can output any language as easily as any other. And that might have been nice compared to Pascal in 1975, it's not so different to modern garbage collected high level scripting languages. Arguably Python or JavaScript will benefit an LLM most because there are so many training examples inside it, compared to almost any other langauge.

nuc1e0n 1y ago
What makes you think your brain isn't also brute forcing potential solutions subconciously and only surfacing the useful results?
- mkirsten 1y ago
  Can you try calculating 101 * 70 in your head?
  - hacker_homie 1y ago
    I think therefore I am calculator?
  - ang_cire 1y ago
    Um, that's really easy to do in your head, there's no carrying or anything? 7,070 7 * 101 = 707 * 10 = 7,070 And computers don't brute-force multiplication either, so I'm not sure how this is relevant to the comment above?
    - polotics 1y ago
      I think it is very relevant, because no brute-forcing is involved in this solution.
      - nutjob2 1y ago
        That's not true, the 'brute force' part is searching for a shortcut that works.
        numpy-thagoras 1y ago
        The brute force got reduced down to fast heuristics, like Arthur Benjamin's Mathemagics.
    - jabbywocker 1y ago
      It’s almost like you’re proving the point of his reply…
  - johnisgood 1y ago
    Very easy to solve, just like it is easy to solve many other ones once you know the tricks. I recommend this book: https://www.amazon.com/Secrets-Mental-Math-Mathemagicians-Calculation/dp/0307338401 https://www.amazon.com/Secrets-Mental-Math-Mathemagicians-Ca...
    - jabbywocker 1y ago
      Completely missing the point on purpose?
      - johnisgood 1y ago
        Elaborate.
        jabbywocker 1y ago
        you don’t solve it by brute forcing possible solutions until one sticks
        johnisgood 11mo ago
        Yeah, read it in another comment. Why do you think doing calculations in your head is brute-forcing? Many people can do it flawlessly, without even knowing of these "tricks". They just know. Is that brute-force?
        jabbywocker 11mo ago
        [flagged]
        johnisgood 11mo ago
        You are not replying to what I said. I am not going to repeat myself, see the parent comment you replied to.
  - froggit 1y ago
    I can absolutely try this. Doesn't mean i'll solve it. If i solve it there's no guarantee i'll be correct. Math gets way harder when i don't have a legitimate need to do it. This falls in the "no legit need" so my mind went right to "100 * 70, good enough."
    - NetMageSCW 11mo ago
      Or you could do (100 + 1)*70 => 100*70 + 1*70
- dbtc 1y ago
  Just intuition ;)
- troupo 1y ago
  human brains are insanely powerful pattern matching and shortcut-taking machines. There's very little brute forcing going on.
  - nutjob2 1y ago
    Your second sentence contradicts your first.
    - troupo 1y ago
      Pray tell how it contradicts the first. Just note: human pattern matching is not Haskell/Erlang/ML pattern matching. It doesn't go [1] through all possible matches of every possible combination of all available criteria [1] If it does, it's the most powerful computing device imaginable.
      - antonvs 1y ago
        > [1] If it does, it's the most powerful computing device imaginable. Well, my brain perhaps. Not sure about the rest of y'all.
      - versteegen 11mo ago
        I 100% agree with nutjob :| There are hundreds of trillions of synapses in the brain, and much of what they do (IANANS) could reasonably be described as pattern matching: mostly sitting idle waiting for patterns. (Since dendritic trees perform a lot of computation (for example, combining inputs at each branch), if you want to count the number of pattern matchers in the branch you can't just count neurons. A neuron can recognise more than one pattern.) So yes, thanks to its insanely parallel architecture, the brain is also an insanely brute force pattern matcher, constantly matching against who knows how many trillions of previously seen patterns. (BTW IMHO this is why LLMs work so well) (I do recognise the gap in my argument: are all those neurons actually receiving inputs to match against, or are they 'gated'? But we're really just arguing about semantics of applying "brute force", a CS term, to a neural architecture, where it has no definition.)
- jodrellblank 1y ago
  Because I can solve problems that would take the age of the universe to brute force, without waiting the age of the universe. So can you: start counting at 1, increment the counter up to 10^8000, then print the counter value. Prolog: 1, 2, 3, 4, 5 ... You and me instantly: 10^8000
  - numpy-thagoras 1y ago
    There's a whole lot of undecidable (or effectively undecidable) edge cases that can be adequately covered. As a matter of fact, Decidability Logic is compatible with Prolog.
  - nuc1e0n 11mo ago
    The brain can still use other means of working in addition to brute forcing solutions. For example, how would you go about solving the chess puzzle of eight queens that doesn't involve going through the potential positions and then filtering out the options that don't match the criteria for the solution? Prolog can also evaluate mathematical expressions directly as well.
riku_iki 1y ago
Even in your example (which is obviously not correct representation of prolog), that code will work X orders magnitude faster and with 100% reliability compared to much more inferior LLM reasoning capabilities.
- szundi 1y ago
  This is not the point though
  - Onavo 1y ago
    Algorithmically there's nothing wrong with using BFS/DFS to do reasoning as long as the logic is correct and the search space is constrained sufficiently. The hard part has always been doing the constraining, which LLMs seem to be rather good at.
  - riku_iki 1y ago
    > This is not the point though could you expand what is the point? That authors opinion without much justification is that this is not reasoning?
cpill 1y ago
Of course it does "reasoning", what do you think reasoning is? From a quick google: "the action of thinking about something in a logical, sensible way". Prolog searches through a space of logical proposition (constraints) and finds conditions that lead to solutions (if one exists). (a) Trying adding another 100 or 1000 interlocking proposition to your problem. It will find solutions or tell you one doesn't exist. (b) You can verify the solutions yourself. You don't get that with imperative descriptions of problems. (b) Good luck sandboxing Python or JavaScript with the treat of prompt injection still unsolved.
- jodrellblank 1y ago
  Of course it doesn't "do reasoning", why do you think "following the instructions you gave it in the stupidest way imaginable" is 'obviously' reasoning? I think one definition of reasoning is being able to come up with any better-than-brute-force thing that you haven't been explicitly told to use on this problem. Prolog isn't "thinking". Not about anything, not about your problem, your code, its implementation, or any background knowledge. Prolog cannot reason that your problem is isomorphic to another problem with a known solution. It cannot come up with an expression transform that hasn't been hard-coded into the interpreter which would reduce the amount of work involved in getting to a solution. It cannot look at your code, reason about it, and make a logical leap over some of the code without executing it (in a way that hasn't been hard-coded into it by the programmer/implementer). It cannot reason that your problem would be better solved with SLG resolution (tabling) instead of SLD resolution (depth first search). The point of my example being pseudo-Python was to make it clear that plain Prolog (meaning no constraint solver, no metaprogramming), is not reasoning. It's no more reasoning than that Python loop is reasoning. If you ask me to find the largest Prime number between 1 and 1000, I might think to skip even numbers, I might think to search down from 1000 instead of up from 1. I might not come up with a good strategy but I will reason about the problem. Prolog will not. You code what it will do, and it will slavishly do what you coded. If you code counting 1-1000 it will do that. If you code Sieve of Eratosthenes it will do that instead.
  - hunterpayne 1y ago
    Its a Horn clause interpreter. Maybe lookup what that is before commenting on it. Clearly you don't have a good grasp of Computer Science concepts or math based upon your comments here. You also don't seem to understand the AI/ML definition of reasoning (which is based in formal logic, much like Prolog itself). Python and Prolog are based upon completely different kinds of math. The only thing they share is that they are both Turing complete. But being Turing complete isn't a strong or complete mathematical definition of a programming language. This is especially true for Prolog which is very different from other languages, especially Python. You shouldn't even think of Prolog as a programming language, think of it as a type of logic system (or solver).
ux266478 1y ago
Everything you've written here is an invalid over-reduction, I presume because you aren't terribly well versed with Prolog. Your simplification is not only outright erroneous in a few places, but essentially excludes every single facet of Prolog that makes it a turing complete logic language. What you are essentially presenting Prolog as would be like presenting C as a language where all you can do is perform operations on constants, not even being able to define functions or preprocessor macros. To assert that's what C is would be completely and obviously ludicrous, but not so many people are familiar enough with Prolog or its underlying formalisms to call you out on this. Firstly, we must set one thing straight: Prolog definitionally does reasoning. Formal reasoning. This isn't debatable, it's a simple fact. It implements resolution (a computationally friendly inference rule over computationally-friendly logical clauses) that's sound and refutation complete, and made practical through unification. Your example is not even remotely close to how Prolog actually works, and excludes much of the extra-logical aspects that Prolog implements. Stripping it of any of this effectively changes the language beyond recognition. > Plain Prolog's way of solving reasoning problems is effectively: No. There is no cognate to what you wrote anywhere in how Prolog works. What you have here doesn't even qualify as a forward chaining system, though that's what it's closest to given it's somewhat how top-down systems work with their ruleset. For it to even approach a weaker forward chaining system like CLIPS, that would have to be a list of rules which require arbitrary computation and may mutate the list of rules it's operating on. A simple iteration over a list testing for conditions doesn't even remotely cut it, and again that's still not Prolog even if we switch to a top-down approach by enabling tabling. > You hard code some options A Prolog knowledgebase is not hardcoded. > write a logical condition with placeholders A horn clause is not a "logical condition", and those "placeholders" are just normal variables. > and Prolog brute-forces every option in every placeholder. Absolutely not. It traverses a graph proving things, and when it cannot prove something it backtracks and tries a different route, or otherwise fails. This is of course without getting into impure Prolog, or the extra-logical aspects it implements. It's a fundamentally different foundation of computation which is entirely geared towards formal reasoning. > And that might have been nice compared to Pascal in 1975, it's not so different to modern garbage collected high level scripting languages. It is extremely different, and the only reason you believe this is because you don't understand Prolog in the slightest, as indicated by the unsoundness of essentially everything you wrote. Prolog is as different from something like Javascript as a neural network with memory is.
tannhaeuser 1y ago
Prolog was introduced to capture natural language - in a logic/symbolic way that didn't prove as powerful as today's LLM for sure, but this still means there is a large corpus of direct English to Prolog mappings available for training, and also the mapping rules are much more straightforward by design. You can pretty much translate simple sentences 1:1 into Prolog clauses as in the classic boring example % "the boy eats the apple" eats(boy, apple). This is being taken advantage of in Prolog code generation using LLMs. In the Quantum Prolog example, the LLM is also instructed not to generate search strategies/algorithms but just planning domain representation and action clauses for changing those domain state clauses which is natural enough in vanilla Prolog. The results are quite a bit more powerful, close to end user problems, and upward in the food chain compared to the usual LLM coding tasks for Python and JavaScript such as boilerplate code generation and similarly idiosyncratic problems.
- jodrellblank 11mo ago
  "large corpus" - large compared to the amount of Python on Github or the amount of JavaScript on all the webpages Google has ever indexed? Quantum Prolog doesn't have any relevant looking DuckDuckGo results, I found it in an old comment of yours here[1] but the link goes to a redirect which is blocked by uBlock rules and on to several more redirects beyond which I didn't get to a page. In your linked comment you write: > "has convenient built-in recursive-decent parsing with backtracking built-in into the language semantics, but also has bottom-up parsing facilities for defining operator precedence parsers. That's why it's very convenient for building DSLs" which I agree with, for humans. What I am arguing is that LLMs don't have the same notion of "convenient". Them dumping hundreds of lines of convoluted 'unreadable' Python (or C or Go or anything) to implement "half of Common Lisp" or "half of a Prolog engine" for a single task is fine, they don't have to read it, and it gets the same result. What would be different is if it got a significantly better result, which I would find interesting but haven't seen a good reason why it would. [1] https://news.ycombinator.com/item?id=40523633 https://news.ycombinator.com/item?id=40523633
hunterpayne 1y ago
Its a Horn clause resolver...that's exactly the kind of reasoning that LLMs are bad at. I have no idea how to graft Prolog to an LLM but if you can graft any programming language to it, you can graft Prolog more easily. Also, that you push Python and JavaScript makes me think you don't know many languages. Those are terrible languages to try to graft to anything. Just because you only know those 2 languages doesn't make them good choices for something like this. Learn a real language Physicist.
- numpy-thagoras 1y ago
  We would begin by having a Prolog server of some kind (I have no idea if Prolog is parallelized but it should very well be if we're dealing with Horn Clauses). There would be MCP bindings to said server, which would be accessible upon request. The LLM would provide a message, it could even formulate Prolog statements per a structured prompt, and then await the result, and then continue.
- jodrellblank 1y ago
  > Also, that you push Python and JavaScript I didn't push them. > Those are terrible languages to try to graft to anything. Web browsers, Blender, LibreOffice and Excel all use those languages for embedded scripting. They're fine. > Just because you only know those 2 languages doesn't make them good choices for something like this. You misunderstood my claim and are refuting something different. I said there is more training data for LLMs to use to generate Python and JavaScript, than Prolog.
  - hunterpayne 1y ago
    I'm not. Python and JS are scripting languages. And in this case, we want something that models formal logic. We are hammering in a nail, you picked up a screwdriver and I am telling you to use a claw hammer.
    - Karrot_Kream 11mo ago
      What does this comment even mean? A claw hammer? By formal definitions, all 3 languages are Turing complete and can express programs of the same computational complexity.
YeGoblynQueenne 1y ago
>> You hard code some options, write a logical condition with placeholders, and Prolog brute-forces every option in every placeholder. It doesn't do reasoning. SLD-Resolution with unification (Prolog's automated theorem proving algorithm) is the polar opposite of brute force: as the proof proceeds, the cardinality of the set of possible answers [1] decreases monotonically. Unification itself is nothing but a dirty hack to avoid having to ground the Herbrand base of a predicate before completing a proof; which is basically going from an NP-complete problem to a linear-time one (on average). Besides which I find it very difficult to see how a language with an automated theorem prover for an interpreter "doesn't do reasoning". If automated theorem proving is not reasoning, what is? ___________________ [1] More precisely, the resolution closure.
- jodrellblank 11mo ago
  > "as the proof proceeds, the cardinality of the set of possible answers [1] decreases" In the sense that it cuts off part of the search tree where answers cannot be found? member(X, [1,2,3,4]), X > 5, slow_computation(X, 0.001). will never do the slow_computation - but if it did, it would come up with the same result. How is that the polar opposite of brute force, rather than an optimization of brute-force? If a language has tail call optimization then it can handle deeper recursive calls with less memory. Without TCO it would do the same thing and get the same result but using more memory, assuming it had enough memory. TCO and non-TCO aren't polar opposites, they are almost the same.
  - xabush 11mo ago
    I don't understand (the point of) your example. In all branches of the search `X > 5` will never be `true` so yeah `slow_computation` will not be reached. How does that relate to your point of it being "brute force" >> but if it did, it would come up with the same result Meaning either changing the condition or the order of the clauses. How do you expect Prolog to proceed to `slow_computation` when you have declared a statement (X > 5) that is always false before it.
    - 11mo ago
superdisk 11mo ago
Contrary to what everyone else is saying, I think you're completely correct. Using it for AI or "reasoning" is a hopeless dead end, even if people wish otherwise. However I've found that Prolog is an excellent language for expressing certain types of problems in a very concise way, like parsers, compilers, and assemblers (and many more). The whole concept of using a predicate in different modes is actually very useful in a pragmatic way for a lot of problems. When you add in the constraint solving extensions (CLP(Z) and CLP(B) and so on) it becomes even more powerful, since you can essentially mix vanilla Prolog code with solver tools.
- YeGoblynQueenne 11mo ago
  The reason why you can write parsers with Prolog is because you can cast the problem of determining whether a string belongs to a language or not as a proof, and, in Prolog, express it as a set of Definite Clauses, particularly with the syntactic sugar of Definite Clause Grammars that give you an executable grammar that acts as both acceptor and generator and is equivalent to a left-corner parser. Now, with that in mind, I'd like to understand how you and the OP reconcile the ability to carry out a formal proof with the inability to do reasoning. How is it not reasoning, if you're doing a proof? If a proof is not reasoning, then what is?
  - jodrellblank 11mo ago
    Clearly people write parsers in C and C++ and Pascal and OCAML, etc. What does it mean to come in with "the reason you can write parsers with Prolog..."? I'm not claiming that reason is incorrect, I'm handwaving it away as irrelevant and academic. Like saying that Lisp map() is better than Python map() because Lisp map is based on formal Lambda Calculus and Python map is an inferior imitation for blub programmers. When a programmer maps a function over a list and gets a result, it's a distinction without a difference. When a programmer writes a getchar() peek() and goto state machine parser with no formalism, it works, what difference does the formalism behind the implementation practically make? Yes maybe the Prolog way means concise code is easier for a human to tell whether the code is a correct expression of the intent, but an LLM won't look at it like that. Whatever the formalism brings, it isn't enough that every parser task is done in Prolog in the last 50 years. Therefore it isn't any particular interest or benefit, except academic. > both acceptor and generator Also academically interesting but practically useless due to the combinatorial explosion of "all possible valid grammars" after the utterly basic "aaaaabbbbbbbbbbbb" examples. > "how you and the OP reconcile the ability to carry out a formal proof with the inability to do reasoning. How is it not reasoning, if you're doing a proof? If a proof is not reasoning, then what is?" If drawing a painting is art, is it art if a computer pulls up a picture of a painting and shows it on screen? No. If a human coded the proof into a computer, the human is reasoning, the computer isn't. If the computer comes up with the proof, the computer is reasoning. Otherwise you're in a situation where dominos falling over is "doing reasoning" because it can be expressed formally as a chain of connected events where the last one only falls if the whole chain is built properly, and that's absurdum.
disambiguation 11mo ago
This sparked a really fascinating discussion, I don't know if anyone will see this but thanks everyone for sharing your thoughts :) I understand your point - to an LLM there's no meaningful difference between once turing complete language and another. I'll concede that I don't have a counter argument, and perhaps it doesn't need to be prolog - though my hunch is that LLM's tend to give better results when using purpose built tools for a given type of problem. The only loose end I want to address is the idea of "doing reasoning." This isn't an AGI proposal (I was careful to say "good at writing prolog") just an augmentation that (as a user) I haven't yet seen applied in practice. But neither have I seen it convincingly dismissed. The idea is the LLM would act like an NLP parser that gradually populates a prolog ontology, like building a logic jail one brick at a time. The result would be a living breathing knowledge base which constrains and informs the LLM's outputs. The punchline is that I don't even know any prolog myself, I just think it's a neat idea.

nuc1e0n 1y ago

What makes you think your brain isn't also brute forcing potential solutions subconciously and only surfacing the useful results?

mkirsten 1y ago
Can you try calculating 101 * 70 in your head?
- hacker_homie 1y ago
  I think therefore I am calculator?
- ang_cire 1y ago
  Um, that's really easy to do in your head, there's no carrying or anything? 7,070 7 * 101 = 707 * 10 = 7,070 And computers don't brute-force multiplication either, so I'm not sure how this is relevant to the comment above?
  - polotics 1y ago
    I think it is very relevant, because no brute-forcing is involved in this solution.
    - nutjob2 1y ago
      That's not true, the 'brute force' part is searching for a shortcut that works.
      - numpy-thagoras 1y ago
        The brute force got reduced down to fast heuristics, like Arthur Benjamin's Mathemagics.
  - jabbywocker 1y ago
    It’s almost like you’re proving the point of his reply…
- johnisgood 1y ago
  Very easy to solve, just like it is easy to solve many other ones once you know the tricks. I recommend this book: https://www.amazon.com/Secrets-Mental-Math-Mathemagicians-Calculation/dp/0307338401 https://www.amazon.com/Secrets-Mental-Math-Mathemagicians-Ca...
  - jabbywocker 1y ago
    Completely missing the point on purpose?
    - johnisgood 1y ago
      Elaborate.
      - jabbywocker 1y ago
        you don’t solve it by brute forcing possible solutions until one sticks
        johnisgood 11mo ago
        Yeah, read it in another comment. Why do you think doing calculations in your head is brute-forcing? Many people can do it flawlessly, without even knowing of these "tricks". They just know. Is that brute-force?
        jabbywocker 11mo ago
        [flagged]
        johnisgood 11mo ago
        You are not replying to what I said. I am not going to repeat myself, see the parent comment you replied to.
- froggit 1y ago
  I can absolutely try this. Doesn't mean i'll solve it. If i solve it there's no guarantee i'll be correct. Math gets way harder when i don't have a legitimate need to do it. This falls in the "no legit need" so my mind went right to "100 * 70, good enough."
  - NetMageSCW 11mo ago
    Or you could do (100 + 1)*70 => 100*70 + 1*70
dbtc 1y ago
Just intuition ;)
troupo 1y ago
human brains are insanely powerful pattern matching and shortcut-taking machines. There's very little brute forcing going on.
- nutjob2 1y ago
  Your second sentence contradicts your first.
  - troupo 1y ago
    Pray tell how it contradicts the first. Just note: human pattern matching is not Haskell/Erlang/ML pattern matching. It doesn't go [1] through all possible matches of every possible combination of all available criteria [1] If it does, it's the most powerful computing device imaginable.
    - antonvs 1y ago
      > [1] If it does, it's the most powerful computing device imaginable. Well, my brain perhaps. Not sure about the rest of y'all.
    - versteegen 11mo ago
      I 100% agree with nutjob :| There are hundreds of trillions of synapses in the brain, and much of what they do (IANANS) could reasonably be described as pattern matching: mostly sitting idle waiting for patterns. (Since dendritic trees perform a lot of computation (for example, combining inputs at each branch), if you want to count the number of pattern matchers in the branch you can't just count neurons. A neuron can recognise more than one pattern.) So yes, thanks to its insanely parallel architecture, the brain is also an insanely brute force pattern matcher, constantly matching against who knows how many trillions of previously seen patterns. (BTW IMHO this is why LLMs work so well) (I do recognise the gap in my argument: are all those neurons actually receiving inputs to match against, or are they 'gated'? But we're really just arguing about semantics of applying "brute force", a CS term, to a neural architecture, where it has no definition.)
jodrellblank 1y ago
Because I can solve problems that would take the age of the universe to brute force, without waiting the age of the universe. So can you: start counting at 1, increment the counter up to 10^8000, then print the counter value. Prolog: 1, 2, 3, 4, 5 ... You and me instantly: 10^8000
- numpy-thagoras 1y ago
  There's a whole lot of undecidable (or effectively undecidable) edge cases that can be adequately covered. As a matter of fact, Decidability Logic is compatible with Prolog.
- nuc1e0n 11mo ago
  The brain can still use other means of working in addition to brute forcing solutions. For example, how would you go about solving the chess puzzle of eight queens that doesn't involve going through the potential positions and then filtering out the options that don't match the criteria for the solution? Prolog can also evaluate mathematical expressions directly as well.

riku_iki 1y ago

Even in your example (which is obviously not correct representation of prolog), that code will work X orders magnitude faster and with 100% reliability compared to much more inferior LLM reasoning capabilities.

szundi 1y ago
This is not the point though
- Onavo 1y ago
  Algorithmically there's nothing wrong with using BFS/DFS to do reasoning as long as the logic is correct and the search space is constrained sufficiently. The hard part has always been doing the constraining, which LLMs seem to be rather good at.
- riku_iki 1y ago
  > This is not the point though could you expand what is the point? That authors opinion without much justification is that this is not reasoning?

cpill 1y ago

Of course it does "reasoning", what do you think reasoning is? From a quick google: "the action of thinking about something in a logical, sensible way". Prolog searches through a space of logical proposition (constraints) and finds conditions that lead to solutions (if one exists). (a) Trying adding another 100 or 1000 interlocking proposition to your problem. It will find solutions or tell you one doesn't exist. (b) You can verify the solutions yourself. You don't get that with imperative descriptions of problems. (b) Good luck sandboxing Python or JavaScript with the treat of prompt injection still unsolved.

jodrellblank 1y ago
Of course it doesn't "do reasoning", why do you think "following the instructions you gave it in the stupidest way imaginable" is 'obviously' reasoning? I think one definition of reasoning is being able to come up with any better-than-brute-force thing that you haven't been explicitly told to use on this problem. Prolog isn't "thinking". Not about anything, not about your problem, your code, its implementation, or any background knowledge. Prolog cannot reason that your problem is isomorphic to another problem with a known solution. It cannot come up with an expression transform that hasn't been hard-coded into the interpreter which would reduce the amount of work involved in getting to a solution. It cannot look at your code, reason about it, and make a logical leap over some of the code without executing it (in a way that hasn't been hard-coded into it by the programmer/implementer). It cannot reason that your problem would be better solved with SLG resolution (tabling) instead of SLD resolution (depth first search). The point of my example being pseudo-Python was to make it clear that plain Prolog (meaning no constraint solver, no metaprogramming), is not reasoning. It's no more reasoning than that Python loop is reasoning. If you ask me to find the largest Prime number between 1 and 1000, I might think to skip even numbers, I might think to search down from 1000 instead of up from 1. I might not come up with a good strategy but I will reason about the problem. Prolog will not. You code what it will do, and it will slavishly do what you coded. If you code counting 1-1000 it will do that. If you code Sieve of Eratosthenes it will do that instead.
- hunterpayne 1y ago
  Its a Horn clause interpreter. Maybe lookup what that is before commenting on it. Clearly you don't have a good grasp of Computer Science concepts or math based upon your comments here. You also don't seem to understand the AI/ML definition of reasoning (which is based in formal logic, much like Prolog itself). Python and Prolog are based upon completely different kinds of math. The only thing they share is that they are both Turing complete. But being Turing complete isn't a strong or complete mathematical definition of a programming language. This is especially true for Prolog which is very different from other languages, especially Python. You shouldn't even think of Prolog as a programming language, think of it as a type of logic system (or solver).

ux266478 1y ago

Everything you've written here is an invalid over-reduction, I presume because you aren't terribly well versed with Prolog. Your simplification is not only outright erroneous in a few places, but essentially excludes every single facet of Prolog that makes it a turing complete logic language. What you are essentially presenting Prolog as would be like presenting C as a language where all you can do is perform operations on constants, not even being able to define functions or preprocessor macros. To assert that's what C is would be completely and obviously ludicrous, but not so many people are familiar enough with Prolog or its underlying formalisms to call you out on this. Firstly, we must set one thing straight: Prolog definitionally does reasoning. Formal reasoning. This isn't debatable, it's a simple fact. It implements resolution (a computationally friendly inference rule over computationally-friendly logical clauses) that's sound and refutation complete, and made practical through unification. Your example is not even remotely close to how Prolog actually works, and excludes much of the extra-logical aspects that Prolog implements. Stripping it of any of this effectively changes the language beyond recognition. > Plain Prolog's way of solving reasoning problems is effectively: No. There is no cognate to what you wrote anywhere in how Prolog works. What you have here doesn't even qualify as a forward chaining system, though that's what it's closest to given it's somewhat how top-down systems work with their ruleset. For it to even approach a weaker forward chaining system like CLIPS, that would have to be a list of rules which require arbitrary computation and may mutate the list of rules it's operating on. A simple iteration over a list testing for conditions doesn't even remotely cut it, and again that's still not Prolog even if we switch to a top-down approach by enabling tabling. > You hard code some options A Prolog knowledgebase is not hardcoded. > write a logical condition with placeholders A horn clause is not a "logical condition", and those "placeholders" are just normal variables. > and Prolog brute-forces every option in every placeholder. Absolutely not. It traverses a graph proving things, and when it cannot prove something it backtracks and tries a different route, or otherwise fails. This is of course without getting into impure Prolog, or the extra-logical aspects it implements. It's a fundamentally different foundation of computation which is entirely geared towards formal reasoning. > And that might have been nice compared to Pascal in 1975, it's not so different to modern garbage collected high level scripting languages. It is extremely different, and the only reason you believe this is because you don't understand Prolog in the slightest, as indicated by the unsoundness of essentially everything you wrote. Prolog is as different from something like Javascript as a neural network with memory is.

tannhaeuser 1y ago

Prolog was introduced to capture natural language - in a logic/symbolic way that didn't prove as powerful as today's LLM for sure, but this still means there is a large corpus of direct English to Prolog mappings available for training, and also the mapping rules are much more straightforward by design. You can pretty much translate simple sentences 1:1 into Prolog clauses as in the classic boring example % "the boy eats the apple" eats(boy, apple). This is being taken advantage of in Prolog code generation using LLMs. In the Quantum Prolog example, the LLM is also instructed not to generate search strategies/algorithms but just planning domain representation and action clauses for changing those domain state clauses which is natural enough in vanilla Prolog. The results are quite a bit more powerful, close to end user problems, and upward in the food chain compared to the usual LLM coding tasks for Python and JavaScript such as boilerplate code generation and similarly idiosyncratic problems.

jodrellblank 11mo ago
"large corpus" - large compared to the amount of Python on Github or the amount of JavaScript on all the webpages Google has ever indexed? Quantum Prolog doesn't have any relevant looking DuckDuckGo results, I found it in an old comment of yours here[1] but the link goes to a redirect which is blocked by uBlock rules and on to several more redirects beyond which I didn't get to a page. In your linked comment you write: > "has convenient built-in recursive-decent parsing with backtracking built-in into the language semantics, but also has bottom-up parsing facilities for defining operator precedence parsers. That's why it's very convenient for building DSLs" which I agree with, for humans. What I am arguing is that LLMs don't have the same notion of "convenient". Them dumping hundreds of lines of convoluted 'unreadable' Python (or C or Go or anything) to implement "half of Common Lisp" or "half of a Prolog engine" for a single task is fine, they don't have to read it, and it gets the same result. What would be different is if it got a significantly better result, which I would find interesting but haven't seen a good reason why it would. [1] https://news.ycombinator.com/item?id=40523633 https://news.ycombinator.com/item?id=40523633

hunterpayne 1y ago

Its a Horn clause resolver...that's exactly the kind of reasoning that LLMs are bad at. I have no idea how to graft Prolog to an LLM but if you can graft any programming language to it, you can graft Prolog more easily. Also, that you push Python and JavaScript makes me think you don't know many languages. Those are terrible languages to try to graft to anything. Just because you only know those 2 languages doesn't make them good choices for something like this. Learn a real language Physicist.

numpy-thagoras 1y ago
We would begin by having a Prolog server of some kind (I have no idea if Prolog is parallelized but it should very well be if we're dealing with Horn Clauses). There would be MCP bindings to said server, which would be accessible upon request. The LLM would provide a message, it could even formulate Prolog statements per a structured prompt, and then await the result, and then continue.
jodrellblank 1y ago
> Also, that you push Python and JavaScript I didn't push them. > Those are terrible languages to try to graft to anything. Web browsers, Blender, LibreOffice and Excel all use those languages for embedded scripting. They're fine. > Just because you only know those 2 languages doesn't make them good choices for something like this. You misunderstood my claim and are refuting something different. I said there is more training data for LLMs to use to generate Python and JavaScript, than Prolog.
- hunterpayne 1y ago
  I'm not. Python and JS are scripting languages. And in this case, we want something that models formal logic. We are hammering in a nail, you picked up a screwdriver and I am telling you to use a claw hammer.
  - Karrot_Kream 11mo ago
    What does this comment even mean? A claw hammer? By formal definitions, all 3 languages are Turing complete and can express programs of the same computational complexity.

YeGoblynQueenne 1y ago

>> You hard code some options, write a logical condition with placeholders, and Prolog brute-forces every option in every placeholder. It doesn't do reasoning. SLD-Resolution with unification (Prolog's automated theorem proving algorithm) is the polar opposite of brute force: as the proof proceeds, the cardinality of the set of possible answers [1] decreases monotonically. Unification itself is nothing but a dirty hack to avoid having to ground the Herbrand base of a predicate before completing a proof; which is basically going from an NP-complete problem to a linear-time one (on average). Besides which I find it very difficult to see how a language with an automated theorem prover for an interpreter "doesn't do reasoning". If automated theorem proving is not reasoning, what is? ___________________ [1] More precisely, the resolution closure.

jodrellblank 11mo ago
> "as the proof proceeds, the cardinality of the set of possible answers [1] decreases" In the sense that it cuts off part of the search tree where answers cannot be found? member(X, [1,2,3,4]), X > 5, slow_computation(X, 0.001). will never do the slow_computation - but if it did, it would come up with the same result. How is that the polar opposite of brute force, rather than an optimization of brute-force? If a language has tail call optimization then it can handle deeper recursive calls with less memory. Without TCO it would do the same thing and get the same result but using more memory, assuming it had enough memory. TCO and non-TCO aren't polar opposites, they are almost the same.
- xabush 11mo ago
  I don't understand (the point of) your example. In all branches of the search `X > 5` will never be `true` so yeah `slow_computation` will not be reached. How does that relate to your point of it being "brute force" >> but if it did, it would come up with the same result Meaning either changing the condition or the order of the clauses. How do you expect Prolog to proceed to `slow_computation` when you have declared a statement (X > 5) that is always false before it.
  - 11mo ago

superdisk 11mo ago

Contrary to what everyone else is saying, I think you're completely correct. Using it for AI or "reasoning" is a hopeless dead end, even if people wish otherwise. However I've found that Prolog is an excellent language for expressing certain types of problems in a very concise way, like parsers, compilers, and assemblers (and many more). The whole concept of using a predicate in different modes is actually very useful in a pragmatic way for a lot of problems. When you add in the constraint solving extensions (CLP(Z) and CLP(B) and so on) it becomes even more powerful, since you can essentially mix vanilla Prolog code with solver tools.

YeGoblynQueenne 11mo ago
The reason why you can write parsers with Prolog is because you can cast the problem of determining whether a string belongs to a language or not as a proof, and, in Prolog, express it as a set of Definite Clauses, particularly with the syntactic sugar of Definite Clause Grammars that give you an executable grammar that acts as both acceptor and generator and is equivalent to a left-corner parser. Now, with that in mind, I'd like to understand how you and the OP reconcile the ability to carry out a formal proof with the inability to do reasoning. How is it not reasoning, if you're doing a proof? If a proof is not reasoning, then what is?
- jodrellblank 11mo ago
  Clearly people write parsers in C and C++ and Pascal and OCAML, etc. What does it mean to come in with "the reason you can write parsers with Prolog..."? I'm not claiming that reason is incorrect, I'm handwaving it away as irrelevant and academic. Like saying that Lisp map() is better than Python map() because Lisp map is based on formal Lambda Calculus and Python map is an inferior imitation for blub programmers. When a programmer maps a function over a list and gets a result, it's a distinction without a difference. When a programmer writes a getchar() peek() and goto state machine parser with no formalism, it works, what difference does the formalism behind the implementation practically make? Yes maybe the Prolog way means concise code is easier for a human to tell whether the code is a correct expression of the intent, but an LLM won't look at it like that. Whatever the formalism brings, it isn't enough that every parser task is done in Prolog in the last 50 years. Therefore it isn't any particular interest or benefit, except academic. > both acceptor and generator Also academically interesting but practically useless due to the combinatorial explosion of "all possible valid grammars" after the utterly basic "aaaaabbbbbbbbbbbb" examples. > "how you and the OP reconcile the ability to carry out a formal proof with the inability to do reasoning. How is it not reasoning, if you're doing a proof? If a proof is not reasoning, then what is?" If drawing a painting is art, is it art if a computer pulls up a picture of a painting and shows it on screen? No. If a human coded the proof into a computer, the human is reasoning, the computer isn't. If the computer comes up with the proof, the computer is reasoning. Otherwise you're in a situation where dominos falling over is "doing reasoning" because it can be expressed formally as a chain of connected events where the last one only falls if the whole chain is built properly, and that's absurdum.

disambiguation 11mo ago

This sparked a really fascinating discussion, I don't know if anyone will see this but thanks everyone for sharing your thoughts :) I understand your point - to an LLM there's no meaningful difference between once turing complete language and another. I'll concede that I don't have a counter argument, and perhaps it doesn't need to be prolog - though my hunch is that LLM's tend to give better results when using purpose built tools for a given type of problem. The only loose end I want to address is the idea of "doing reasoning." This isn't an AGI proposal (I was careful to say "good at writing prolog") just an augmentation that (as a user) I haven't yet seen applied in practice. But neither have I seen it convincingly dismissed. The idea is the LLM would act like an NLP parser that gradually populates a prolog ontology, like building a logic jail one brick at a time. The result would be a living breathing knowledge base which constrains and informs the LLM's outputs. The punchline is that I don't even know any prolog myself, I just think it's a neat idea.