5 ms·

Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setu

by pkoird 3y ago

Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.

westoncb 3y ago
I think this is an important perspective. It helps clarify prompting as well because, used in a certain way, they are effectively natural language specification of constraints which have the effect of 'partially configuring' the network so that its inference selects from the configuration space of its still free params For example, if you tell it to reply in JSON (and it obeys), you've just constrained its search space in a particular way. There is space for very interesting informal programming that can be done from this perspective, setting up constraints and then allowing inference to solve within them. I've been using this heavily. When I was first getting deep into LLM stuff a few months ago and contemplating latent space my main characterization was that much of its high level behavior can be usefully grappled with by viewing it as a kind of 'learned geometric prolog'. I did a bunch of illustrations and talked about some of these ideas here if anyone's curious: https://x.com/Westoncb/status/1757910205478703277 https://x.com/Westoncb/status/1757910205478703277 (I think I mostly dropped the prolog terminology in that presentation because not everyone knows about it)
thesz 3y ago
https://en.wikipedia.org/wiki/Cyc#MathCraft https://en.wikipedia.org/wiki/Cyc#MathCraft Quote: One Cyc application aims to help students doing math at a 6th grade level, helping them much more deeply understand that subject matter... Unlike almost all other educational software, where the computer plays the role of the teacher, this application of Cyc, called MathCraft, has Cyc play the role of a fellow student who is always slightly more confused than you, the user, are about the subject. This is from 2017. I haven't seen anything like this using LMs in 2017 and I suspect it is still hard for LLMs today. Cyc is the huge reasoning engine. You can call it Prolog, if you want. I won't.
- ogogmad 3y ago
  > I suspect it is still hard for LLMs I just gave it to Claude: https://imgur.com/a/fQQOy1d https://imgur.com/a/fQQOy1d
  - Sakos 3y ago
    1) This is ridiculously cool 2) The "action" text gives me such I put on my wizard hat vibes
  - thesz 3y ago
    MathCraft [1] is little more [2] than chat. [1] https://www.cyc.com/mathcraft/ [2] https://www.youtube.com/watch?v=pbrp7MzBDm0
pjmlp 3y ago
The moment they are fully there, just like an executable generated from a Prolog compiler, eventually it is time to realize programing is no more, except for the selected few ones from big corps that create LLM compilers.
deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
mirekrusin 3y ago
It also has further reaching consequences. It creates foundation for reinforcement learning without human feedback - a missing piece of puzzle. Simplifying: propose plausible theorem, try to find provable solution, reinforce reasoning/solution path, move proved statement into axioms, repeat. (super)intelligence has many dimentions. One of less explored ones is exploiting concurrency in thought chains. It's something very un-natural to us, but there is a lot of gain if you're able to branch and collect feedback from dead ends and progress from different directions being taken at the same time.
ginko 3y ago
I was thinking of if AI systems could be composed of multiple internal agents that when prompted are supposed to discuss internally and agree on what to reply in the end. Some of the agents could be LLMs but others could be logic engines or database frontends for instance.
DebtDeflation 3y ago
>where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog. Graph RAG is an emerging design pattern where factual information is retrieved from knowledge graphs to augment the textual information retrieved from document stores in classic RAG before submitting to an LLM. What you are proposing is essentially using the LLM to build the knowledge graph in the first place. Would be interesting to see the two techniques combined along with some sort of planner/optimizer in the middle.
astrange 3y ago
The problem is that Prolog is already "AI" as in expert system GOFAI - that approach already failed, so you shouldn't believe people claiming that an entirely different connectionist approach is going to prove that their failed alternative worked all along.
- codesnik 3y ago
  maybe that's because prolog from 80ies basically operated in 1bit space and had just 1000s of parameters, crafted by hand. But nowadays...
nickpsecurity 3y ago
In truth, I thought it would be a great idea to train them on turning English and code into formal specs and logic language. Then, we could use all our tools in those areas work from there. Combining LLM’s with rewriting logic, like Maude or K Framework, would be the most, powerful option. The rewriting tools plus LLM’s could probably rapidly develop static analyzers and code porting tools.
pyinstallwoes 3y ago
Hehe, this was the exact thinking going across my mind in considering underlying architecture for a LLM OS/Word processor that was self aware of the documents across a project for consistency.
gpm 3y ago
I envision a thread of research in the exact opposite direction. Take a logic program/theorem/database query/... and use a LLM to guide a search for a solution/proof/query plan/...
- ithkuil 3y ago
  I wonder if the problem that LLMs solved was not the lack of "intelligence" of logic driven systems but the lack of a particular intelligence that is so crucial for is to make effective use of the interaction with such tools, namely the ability to actually understand our natural language. That feature by itself is not enough, but can be a very effective glue to be used with other components of an intelligent system. The analogy with the human brain would be the broca area vs. the rest of the brain. Now, there are open questions about whether the _architecture_ that underpins the LLMs is also good enough to be used as a substrate for other functions and what's the most effective way for having these different components of the system communicate between each other. The analogy with the human brain can guide us (as well as lead us astray), in that our brain, like biological systems often do, re-purposes the basic building blocks to create different subsystems. It's not clear to me at which level we'll find the most effective re-purposable building blocks. It's easy to try (and people do) to use the top-level LLM system as such a building block and have it produce plans, connect it to external systems that feed information back and have it iterate again on it (ab)using it's language processing as an API with the environment. The human analogy of that is when we use external tools to extend our cognitive capacity, like when we do arithmetic using pencil and paper or when we scribble some notes to help us think. I think this level is useful and real but I wonder if we also need to give more power to some lower levels too. Granted, some of that "power" can already be emerging during the training of the LLMs but I wonder if some more specialized blocks might enhance the effectiveness
- winwang 3y ago
  LLM the Ultimate Heuristic?
  - kevindamm 3y ago
    It would be convenient if true. I'm a little skeptical it will go that far, but at the same time humans seem to better understand a tricky problem after an internal monologue about it or a quick conversation with a rubber ducky.. so maybe?
lachlan_gray 3y ago
You might enjoy parts of this interview with Stephen Wolfram https://m.youtube.com/watch?v=PdE-waSx-d8 https://m.youtube.com/watch?v=PdE-waSx-d8 He’s very much on this kind of beat. In general I have a feeling that there are orders of magnitude to gain by successfully applying computer science to “language algorithms”. Feels like we are exploring very narrow paths of computations that can be performed with language. Like we have an x86 cpu and we are building pocket calculators. So much untapped Re prolog I had a similar intuition at some point and tried to make a stack based programming language that uses a language model as a kind of control/logic unit https://github.com/LachlanGray/silas https://github.com/LachlanGray/silas I was missing a bunch of cs background at the time so I didn’t get very far, but I feel like there’s a lot to be done for this kind of thing
sitkack 3y ago
You aren't half jesting, you have been thinking about this for months as have I.

by pkoird 3y ago

westoncb 3y ago

I think this is an important perspective. It helps clarify prompting as well because, used in a certain way, they are effectively natural language specification of constraints which have the effect of 'partially configuring' the network so that its inference selects from the configuration space of its still free params For example, if you tell it to reply in JSON (and it obeys), you've just constrained its search space in a particular way. There is space for very interesting informal programming that can be done from this perspective, setting up constraints and then allowing inference to solve within them. I've been using this heavily. When I was first getting deep into LLM stuff a few months ago and contemplating latent space my main characterization was that much of its high level behavior can be usefully grappled with by viewing it as a kind of 'learned geometric prolog'. I did a bunch of illustrations and talked about some of these ideas here if anyone's curious: https://x.com/Westoncb/status/1757910205478703277 https://x.com/Westoncb/status/1757910205478703277 (I think I mostly dropped the prolog terminology in that presentation because not everyone knows about it)

thesz 3y ago

https://en.wikipedia.org/wiki/Cyc#MathCraft https://en.wikipedia.org/wiki/Cyc#MathCraft Quote: One Cyc application aims to help students doing math at a 6th grade level, helping them much more deeply understand that subject matter... Unlike almost all other educational software, where the computer plays the role of the teacher, this application of Cyc, called MathCraft, has Cyc play the role of a fellow student who is always slightly more confused than you, the user, are about the subject. This is from 2017. I haven't seen anything like this using LMs in 2017 and I suspect it is still hard for LLMs today. Cyc is the huge reasoning engine. You can call it Prolog, if you want. I won't.

ogogmad 3y ago
> I suspect it is still hard for LLMs I just gave it to Claude: https://imgur.com/a/fQQOy1d https://imgur.com/a/fQQOy1d
- Sakos 3y ago
  1) This is ridiculously cool 2) The "action" text gives me such I put on my wizard hat vibes
- thesz 3y ago
  MathCraft [1] is little more [2] than chat. [1] https://www.cyc.com/mathcraft/ [2] https://www.youtube.com/watch?v=pbrp7MzBDm0

pjmlp 3y ago

The moment they are fully there, just like an executable generated from a Prolog compiler, eventually it is time to realize programing is no more, except for the selected few ones from big corps that create LLM compilers.

deleted 3y ago

[deleted]

deleted 3y ago

[deleted]

mirekrusin 3y ago

It also has further reaching consequences. It creates foundation for reinforcement learning without human feedback - a missing piece of puzzle. Simplifying: propose plausible theorem, try to find provable solution, reinforce reasoning/solution path, move proved statement into axioms, repeat. (super)intelligence has many dimentions. One of less explored ones is exploiting concurrency in thought chains. It's something very un-natural to us, but there is a lot of gain if you're able to branch and collect feedback from dead ends and progress from different directions being taken at the same time.

ginko 3y ago

I was thinking of if AI systems could be composed of multiple internal agents that when prompted are supposed to discuss internally and agree on what to reply in the end. Some of the agents could be LLMs but others could be logic engines or database frontends for instance.

DebtDeflation 3y ago

>where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog. Graph RAG is an emerging design pattern where factual information is retrieved from knowledge graphs to augment the textual information retrieved from document stores in classic RAG before submitting to an LLM. What you are proposing is essentially using the LLM to build the knowledge graph in the first place. Would be interesting to see the two techniques combined along with some sort of planner/optimizer in the middle.

astrange 3y ago

The problem is that Prolog is already "AI" as in expert system GOFAI - that approach already failed, so you shouldn't believe people claiming that an entirely different connectionist approach is going to prove that their failed alternative worked all along.

codesnik 3y ago
maybe that's because prolog from 80ies basically operated in 1bit space and had just 1000s of parameters, crafted by hand. But nowadays...

nickpsecurity 3y ago

In truth, I thought it would be a great idea to train them on turning English and code into formal specs and logic language. Then, we could use all our tools in those areas work from there. Combining LLM’s with rewriting logic, like Maude or K Framework, would be the most, powerful option. The rewriting tools plus LLM’s could probably rapidly develop static analyzers and code porting tools.

pyinstallwoes 3y ago

Hehe, this was the exact thinking going across my mind in considering underlying architecture for a LLM OS/Word processor that was self aware of the documents across a project for consistency.

gpm 3y ago

I envision a thread of research in the exact opposite direction. Take a logic program/theorem/database query/... and use a LLM to guide a search for a solution/proof/query plan/...

ithkuil 3y ago
I wonder if the problem that LLMs solved was not the lack of "intelligence" of logic driven systems but the lack of a particular intelligence that is so crucial for is to make effective use of the interaction with such tools, namely the ability to actually understand our natural language. That feature by itself is not enough, but can be a very effective glue to be used with other components of an intelligent system. The analogy with the human brain would be the broca area vs. the rest of the brain. Now, there are open questions about whether the _architecture_ that underpins the LLMs is also good enough to be used as a substrate for other functions and what's the most effective way for having these different components of the system communicate between each other. The analogy with the human brain can guide us (as well as lead us astray), in that our brain, like biological systems often do, re-purposes the basic building blocks to create different subsystems. It's not clear to me at which level we'll find the most effective re-purposable building blocks. It's easy to try (and people do) to use the top-level LLM system as such a building block and have it produce plans, connect it to external systems that feed information back and have it iterate again on it (ab)using it's language processing as an API with the environment. The human analogy of that is when we use external tools to extend our cognitive capacity, like when we do arithmetic using pencil and paper or when we scribble some notes to help us think. I think this level is useful and real but I wonder if we also need to give more power to some lower levels too. Granted, some of that "power" can already be emerging during the training of the LLMs but I wonder if some more specialized blocks might enhance the effectiveness
winwang 3y ago
LLM the Ultimate Heuristic?
- kevindamm 3y ago
  It would be convenient if true. I'm a little skeptical it will go that far, but at the same time humans seem to better understand a tricky problem after an internal monologue about it or a quick conversation with a rubber ducky.. so maybe?

lachlan_gray 3y ago

You might enjoy parts of this interview with Stephen Wolfram https://m.youtube.com/watch?v=PdE-waSx-d8 https://m.youtube.com/watch?v=PdE-waSx-d8 He’s very much on this kind of beat. In general I have a feeling that there are orders of magnitude to gain by successfully applying computer science to “language algorithms”. Feels like we are exploring very narrow paths of computations that can be performed with language. Like we have an x86 cpu and we are building pocket calculators. So much untapped Re prolog I had a similar intuition at some point and tried to make a stack based programming language that uses a language model as a kind of control/logic unit https://github.com/LachlanGray/silas https://github.com/LachlanGray/silas I was missing a bunch of cs background at the time so I didn’t get very far, but I feel like there’s a lot to be done for this kind of thing

sitkack 3y ago

You aren't half jesting, you have been thinking about this for months as have I.