16 ms·

Large Language Models Are Neurosymbolic Reasoners

107 points by optimalsolver 3y ago

egberts1 3y ago
Yep, just along as it is fed biased data, it can still reason based on faulty info.
zer00eyz 3y ago
I... "It is a tale Told by an idiot, full of sound and fury Signifying nothing." This is sort of water is wet kind of research. Im glad they did it but it's not exactly moving the ball down the field.
sharts 3y ago
Not really
- ranguna 3y ago
  This reminds me of a meme that goes something along these lines: Joins a university; Studies for their bachelor's degree; Gets their degree after 3-5+ years; Studies for their master's; Gets their master's after 2-4+ years; Studies for their PhD while working on their thesis for a few more years; Participares in intensive discussions with their peers, and investigates day and night; Sends their thesis for peer review; Reworks their thesis according to the review; Finally publishes their thesis in an academic journal; Someone on the internet, reads their thesis title: bulsh*t -- Would you care to elaborate on your comment?
  - randysalami 3y ago
    Is the master’s really 2-4 years? I always thought it was closer to 1 or 2.
    - ranguna 3y ago
      Maybe it depends on the master's. The ones I know are 2 years if you follow the standard plan, but I wouldn't be surprised if there were 1 year master's
clooper 3y ago
I was recently thinking how every neural network is equivalent to a lookup table where the input is all numbers up to what can be expressed within the context window and the output is the result of the arithmetic operations applied to that number. So every neural network is equivalent to T = {(i, f(i)) : i < K} where K is the constant which determines the context window and f is the numerical function implemented by the network. Can someone ask a neural network if my reasoning is valid and correct? The main practical issue is the size of the table but I don't see any theoretical reasons why this is incorrect. The neural network is simply a compressed representation of the uncompressed lookup table. Given that the two representations are theoretically equivalent and a lookup table does not perform any reasoning we can conclude that no neural network is actually doing any thinking other than uncompressing the table and looking up the value corresponding to the input number. Modern neural networks have some randomness but that doesn't change the table in any meaningful way because instead of the output being a number it becomes a distribution over some finite range which can again be turned into a table with some tuples.
- bubblyworld 3y ago
  This is an old argument against determinism - I think a serious challenge is that: 1. Modern physics suggests you can implement such a lookup table for any subset of our universe. 2. We are a subset of the universe. 3. Therefore we are representable by lookup tables too. ...so your argument appears to prove too much, namely that humans aren't thinking beings either. Which is fine, but personally I don't think that's a useful definition of "thinking".
  - clooper 3y ago
    How are people lookup tables? In the case of neural networks the representation of the table is obvious, it's just numbers. What would be the equivalent table for the liver? My argument isn't abstract. Neural networks really are just numerical functions which can be expanded into their equivalent graph representations.
    - josh-stylo 3y ago
bubblyworld 3y ago
The authors get LLMs to perform pretty well in a variety of IF-style text based games. Which is pretty cool, these kinds of games are played and read in natural language, which makes them pretty hard to write AIs for normally. Something I'd love to see one day is modern AI applied to other kinds of text based games like nethack. Last I checked nobody had managed to solve the problem of nethack AI without using hard coded heuristics and goals!
- wiz21c 3y ago
  Yes IF-style for sure but that's not Zork either. Cool work though, but having those solving Zork or Mystery House or whatever would be sooo cool !
- still_grokking 3y ago
  Is it actually possible to beat Nethack without reading up some "spoilers" upfront? I've never heard of anybody who managed to do that. Even when you read up all kinds of info about the game before you attempt a run it's extremely hard to reach higher levels, yet beat the game. (I myself never reached any later levels despite I know some tricks by now. Tricks impossible to infer from just playing the game; you need to read them up…) Imho there is no winning strategy for Nethack. It's some random stuff "you need to know" to progress even a little bit paired with complete rule of the dice while encountering maximally nonsensical "puzzles". But OK, maybe I'm just too dumb for this game and don't see the "logic" behind the things the game presents.
  - bubblyworld 3y ago
    I'm not sure. I've beaten it a number of times, but only using tons of spoilers as you say. That said, there are players who consistently win almost every game, which is crazy (there's an online nethack server somewhere, can't remember the name offhand, but you can search player stats and some of them are insane). Edit: here's one, a player with a 60% win rate, not as crazy as I initially thought but if you've ever played nethack... https://alt.org/nethack/player-stats.php?player=Stroller https://alt.org/nethack/player-stats.php?player=Stroller
deleted 3y ago
[deleted]
lkrubner 3y ago
Possibly off-topic, but does anyone know where I can read up on LLMs? I've posted an "Ask HN" here, in the hopes some people can inform me about how I can keep up on what's new: https://news.ycombinator.com/item?id=39688911 https://news.ycombinator.com/item?id=39688911
- thinkingemote 3y ago
  Search box in the bottom can help look for introduction tutorials recommendations etc
lngnmn2 3y ago
[dead]
neurallambda 3y ago
I'm trying to tackle this problem more head-on, by outfitting LLMs with lambda calculus, stacks, queues, etc. directly in their internals, operating over their latent space. [1] I'll read your paper, but, LLMs famously fail horribly at "multi jump" reasoning, which to me means they can't reason at all. They can merely output a reflection of the human reasoning that was baked into the training data, and they can also recombine it combinatorially. Eager to see if you've solved this! [1] https://github.com/neurallambda/neurallambda https://github.com/neurallambda/neurallambda
- eru 3y ago
  Humans also fail much of the time at 'multi jump' reasoning. You have to prod them.
  - littlestymaar 3y ago
    Except no human (non-colorblind at least) past three years old thinks bananas have the same color as the sky (see the example given in the repo, that's a mistake literally no human could make)
    - nielsbot 3y ago
      Maybe not no human :) But probably 99.99% of them.
      - littlestymaar 3y ago
        You're technically right, my two and a half had only be familiar with colors for six months, but I think it's fine to say that toddlers aren't reaching standard level of human intelligence ;)
    - mewpmewp2 3y ago
      For what it's worth, I tried it on ChatGPT and this was its response: "The color of the daytime sky is commonly blue. The common household fruit that is also blue would be blueberries. Blueberries typically grow in acidic soil. The pH of the soil they grow in is usually between 4.5 and 5.5."
ashz8888 3y ago
I did something similar [1], where the text based game being a Text Interface (TI) that provides model a view and a set of actions. The model repeatedly interacts with TI to achieve a goal. In the current implementation, Text Interface allows the model to list files, open a file, and search within a file to find what it needs to satisfy the goal. With my current set of prompts, the model is able to backtrack when it fails to find relevant info in the current file. However, I couldn't get it work with GPT-3.5. Only GPT-4 is capable to reason its way through the Text Interface. [1] https://github.com/ash80/backtracking_gpt https://github.com/ash80/backtracking_gpt

107 points by optimalsolver 3y ago

Yep, just along as it is fed biased data, it can still reason based on faulty info.

I... "It is a tale Told by an idiot, full of sound and fury Signifying nothing." This is sort of water is wet kind of research. Im glad they did it but it's not exactly moving the ball down the field.

sharts 3y ago

Not really

ranguna 3y ago
This reminds me of a meme that goes something along these lines: Joins a university; Studies for their bachelor's degree; Gets their degree after 3-5+ years; Studies for their master's; Gets their master's after 2-4+ years; Studies for their PhD while working on their thesis for a few more years; Participares in intensive discussions with their peers, and investigates day and night; Sends their thesis for peer review; Reworks their thesis according to the review; Finally publishes their thesis in an academic journal; Someone on the internet, reads their thesis title: bulsh*t -- Would you care to elaborate on your comment?
- randysalami 3y ago
  Is the master’s really 2-4 years? I always thought it was closer to 1 or 2.
  - ranguna 3y ago
    Maybe it depends on the master's. The ones I know are 2 years if you follow the standard plan, but I wouldn't be surprised if there were 1 year master's

clooper 3y ago

I was recently thinking how every neural network is equivalent to a lookup table where the input is all numbers up to what can be expressed within the context window and the output is the result of the arithmetic operations applied to that number. So every neural network is equivalent to T = {(i, f(i)) : i < K} where K is the constant which determines the context window and f is the numerical function implemented by the network. Can someone ask a neural network if my reasoning is valid and correct? The main practical issue is the size of the table but I don't see any theoretical reasons why this is incorrect. The neural network is simply a compressed representation of the uncompressed lookup table. Given that the two representations are theoretically equivalent and a lookup table does not perform any reasoning we can conclude that no neural network is actually doing any thinking other than uncompressing the table and looking up the value corresponding to the input number. Modern neural networks have some randomness but that doesn't change the table in any meaningful way because instead of the output being a number it becomes a distribution over some finite range which can again be turned into a table with some tuples.

bubblyworld 3y ago
This is an old argument against determinism - I think a serious challenge is that: 1. Modern physics suggests you can implement such a lookup table for any subset of our universe. 2. We are a subset of the universe. 3. Therefore we are representable by lookup tables too. ...so your argument appears to prove too much, namely that humans aren't thinking beings either. Which is fine, but personally I don't think that's a useful definition of "thinking".
- clooper 3y ago
  How are people lookup tables? In the case of neural networks the representation of the table is obvious, it's just numbers. What would be the equivalent table for the liver? My argument isn't abstract. Neural networks really are just numerical functions which can be expanded into their equivalent graph representations.
  - josh-stylo 3y ago

bubblyworld 3y ago

The authors get LLMs to perform pretty well in a variety of IF-style text based games. Which is pretty cool, these kinds of games are played and read in natural language, which makes them pretty hard to write AIs for normally. Something I'd love to see one day is modern AI applied to other kinds of text based games like nethack. Last I checked nobody had managed to solve the problem of nethack AI without using hard coded heuristics and goals!

wiz21c 3y ago
Yes IF-style for sure but that's not Zork either. Cool work though, but having those solving Zork or Mystery House or whatever would be sooo cool !
still_grokking 3y ago
Is it actually possible to beat Nethack without reading up some "spoilers" upfront? I've never heard of anybody who managed to do that. Even when you read up all kinds of info about the game before you attempt a run it's extremely hard to reach higher levels, yet beat the game. (I myself never reached any later levels despite I know some tricks by now. Tricks impossible to infer from just playing the game; you need to read them up…) Imho there is no winning strategy for Nethack. It's some random stuff "you need to know" to progress even a little bit paired with complete rule of the dice while encountering maximally nonsensical "puzzles". But OK, maybe I'm just too dumb for this game and don't see the "logic" behind the things the game presents.
- bubblyworld 3y ago
  I'm not sure. I've beaten it a number of times, but only using tons of spoilers as you say. That said, there are players who consistently win almost every game, which is crazy (there's an online nethack server somewhere, can't remember the name offhand, but you can search player stats and some of them are insane). Edit: here's one, a player with a 60% win rate, not as crazy as I initially thought but if you've ever played nethack... https://alt.org/nethack/player-stats.php?player=Stroller https://alt.org/nethack/player-stats.php?player=Stroller

deleted 3y ago

[deleted]

lkrubner 3y ago

Possibly off-topic, but does anyone know where I can read up on LLMs? I've posted an "Ask HN" here, in the hopes some people can inform me about how I can keep up on what's new: https://news.ycombinator.com/item?id=39688911 https://news.ycombinator.com/item?id=39688911

thinkingemote 3y ago
Search box in the bottom can help look for introduction tutorials recommendations etc

lngnmn2 3y ago

[dead]

neurallambda 3y ago

I'm trying to tackle this problem more head-on, by outfitting LLMs with lambda calculus, stacks, queues, etc. directly in their internals, operating over their latent space. [1] I'll read your paper, but, LLMs famously fail horribly at "multi jump" reasoning, which to me means they can't reason at all. They can merely output a reflection of the human reasoning that was baked into the training data, and they can also recombine it combinatorially. Eager to see if you've solved this! [1] https://github.com/neurallambda/neurallambda https://github.com/neurallambda/neurallambda

eru 3y ago
Humans also fail much of the time at 'multi jump' reasoning. You have to prod them.
- littlestymaar 3y ago
  Except no human (non-colorblind at least) past three years old thinks bananas have the same color as the sky (see the example given in the repo, that's a mistake literally no human could make)
  - nielsbot 3y ago
    Maybe not no human :) But probably 99.99% of them.
    - littlestymaar 3y ago
      You're technically right, my two and a half had only be familiar with colors for six months, but I think it's fine to say that toddlers aren't reaching standard level of human intelligence ;)
  - mewpmewp2 3y ago
    For what it's worth, I tried it on ChatGPT and this was its response: "The color of the daytime sky is commonly blue. The common household fruit that is also blue would be blueberries. Blueberries typically grow in acidic soil. The pH of the soil they grow in is usually between 4.5 and 5.5."

ashz8888 3y ago

I did something similar [1], where the text based game being a Text Interface (TI) that provides model a view and a set of actions. The model repeatedly interacts with TI to achieve a goal. In the current implementation, Text Interface allows the model to list files, open a file, and search within a file to find what it needs to satisfy the goal. With my current set of prompts, the model is able to backtrack when it fails to find relevant info in the current file. However, I couldn't get it work with GPT-3.5. Only GPT-4 is capable to reason its way through the Text Interface. [1] https://github.com/ash80/backtracking_gpt https://github.com/ash80/backtracking_gpt