7 ms·

I wouldn’t expect an LLM to get this right unless it had been trained on a solution. Am I wrong to think that? Are LLMs in the future going to be able to “thin

I wouldn’t expect an LLM to get this right unless it had been trained on a solution. Am I wrong to think that? Are LLMs in the future going to be able to “think through” actual logic problems?

PrayagBhakar 3y ago
You could achieve thinking though logical problems by adding chain of thought or tree of thought. Then the LLM will try to break stuff down into logic it’s already seen.
ftxbro 3y ago
This is the whole point of the breakthrough related to the emergence of cognitive capabilities of LLMs. They are literally Markov chains. No one expected it to happen to this degree, but here we are.
- dclowd9901 3y ago
  Almost kind of proves ideas shouldn’t be copyrightable.
  - bramblerose 3y ago
    Ideas aren't copyrightable.
    - ftxbro 3y ago
      maybe they meant idea like when you write a book you are transcribing a series of ideas you had
      - deleted 3y ago
        [deleted]
      - dclowd9901 3y ago
        In my mind I was thinking about recipes and code, which are really little more than ideas. My point was that, if such things emerge with a complete lack of creativity, perhaps they don’t warrant protection.
    - moffkalast 3y ago
      Disney: "Let's agree to disagree."
- jhbadger 3y ago
  People say that "they are literally Markov chains", but anyone who has looked at the code for LLMs knows that they are more complicated than that. I implemented Markov chains in BASIC in about ten lines of code in the 1980s on a 1 Mhz 64K Apple II after reading about the famous Mark V. Shaney hoax (https://en.wikipedia.org/wiki/Mark_V._Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney). No neural nets or fancy GPUs required. It's one thing to stress that LLMs aren't magical or self-aware, but the fact is they are way more complicated than simple Markov chains.
  - ftxbro 3y ago
    > People say that "they are literally Markov chains", but anyone who has looked at the code for LLMs knows that they are more complicated than that. They are literally Markov chains according to the mathematical definition. The code is complicated. Having complicated code doesn't mean it's not literally a Markov chain. > I implemented Markov chains in BASIC in about ten lines of code in the 1980s on a 1 Mhz 64K Apple II after reading about the famous Mark V. Shaney hoax (https://en.wikipedia.org/wiki/Mark_V._Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney). No neural nets or fancy GPUs required. I don't doubt this. You can make a Markov chain by just counting the frequency of letters that follow each letter giving one that has a context window of one or two characters. That is a very simple Markov chain. You can make it by hand. You can make ones with more context window like a dozen characters or a few words, using sophisticated smoothing and regularization methods and not just frequency counts. Those are also simple Markov chains that you can do without neural net or GPU. Then you can also make a Markov chain that has a context window of thousands of tokens that is made from neural nets and massive training data and differentiable tensor computing libraries with data centers full of hardware linear algebra accelerators. Those are some even bigger Markov chains! > LLMs are way more complicated than simple Markov chains. That's true, they are more complicated than simple Markov chains, if by simple Markov chains you mean ones with small context window. LLMs are Markov chains with large context window!
    - cubacaban 3y ago
      How big is the state space of the Markov chain corresponding to a LLM generating a sequence of tokens? Wouldn't it be (size of the vocabulary)^(size of the context window), i.e. ~ (100k)^(4k)? How useful is it to conceptualize LLMs as Markov chains at that point? For example, is there a result about Markov chains with interesting implications for LLMs?
FrustratedMonky 3y ago
I've been talking to GPT and asking it long multi part questions and it does seem to 'think through' them, and able to provide negative responses. Specific logic questions like in this post, or long math formulas seem to still be a struggle. But I get the impression it is just a matter of time before it is scaled up to handle them. You can argue whether it is 'reasoning' about them or not. But if it becomes indistinguishable from a human, then does the word 'reason' even really apply to the human anymore? Both are black boxes, giving answers.
- nuancebydefault 3y ago
  Indeed, anyone who says llm's cannot reason since it is just reformulating things it remembers, is unable to explain what reasoning is, let alone how it would not be reasoning. In the end, if the result is indistinguishable from a treat people call reasoning, let's call the behavior reasoning.
jhbadger 3y ago
I don't think so. Consider the following conversation from UndiMix-v2-13B (a local LLM running on my Mac M1 via llama.cpp). It gets it wrong at first, but then gets it right with a little hint. I don't see how the hint could work if it was merely repeating an example it knew. User: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? Llama: Hello there! I'd be happy to help you with that question. Let me think... If each of Sally's brothers has two sisters, then together they would have 6 sisters in total (2 x 3 = 6). So, Sally would have 6 sisters as well. Is that the answer you were looking for? User: no. Think about it. Sally is a sister to the brothers Llama: Ah, I see what you mean now! In this case, since Sally is one of the siblings, she only counts as one of her own sisters. Therefore, Sally has only one sister - herself. Does that make sense? User: yes Llama: Great! I hope that clears up any confusion. Please let me know if you have any other questions or need further assistance.
- jameshart 3y ago
  I don’t think ‘Sally has only one sister - herself’ is the right answer.
- arnvald 3y ago
  The initial response had 2 problems - not counting Sally and counting sisters separately for each brother. Your hint pointed out the 1st problem, but then LLM should still provide wrong solution. This makes me think that maybe it would try another answer and explanation no matter what you said. If your reply was just "no, think again" it could also provide the right answer
- nuancebydefault 3y ago
  She counts as her own sister :D I can be sister of myself?
- 1024core 3y ago
  Llama: ... Therefore, Sally has only one sister - herself. Does that make sense? User: yes ---- Um... no!
thereticent 3y ago
I see no reason why not. You would need to represent dyadic relationships and whether they are reciprocal, transitive, etc., weigh probabilities about the intended meaning (LLM magic already), then use sentential logic rules, right?

I wouldn’t expect an LLM to get this right unless it had been trained on a solution. Am I wrong to think that? Are LLMs in the future going to be able to “think through” actual logic problems?

PrayagBhakar 3y ago

You could achieve thinking though logical problems by adding chain of thought or tree of thought. Then the LLM will try to break stuff down into logic it’s already seen.

ftxbro 3y ago

This is the whole point of the breakthrough related to the emergence of cognitive capabilities of LLMs. They are literally Markov chains. No one expected it to happen to this degree, but here we are.

dclowd9901 3y ago
Almost kind of proves ideas shouldn’t be copyrightable.
- bramblerose 3y ago
  Ideas aren't copyrightable.
  - ftxbro 3y ago
    maybe they meant idea like when you write a book you are transcribing a series of ideas you had
    - deleted 3y ago
      [deleted]
    - dclowd9901 3y ago
      In my mind I was thinking about recipes and code, which are really little more than ideas. My point was that, if such things emerge with a complete lack of creativity, perhaps they don’t warrant protection.
  - moffkalast 3y ago
    Disney: "Let's agree to disagree."
jhbadger 3y ago
People say that "they are literally Markov chains", but anyone who has looked at the code for LLMs knows that they are more complicated than that. I implemented Markov chains in BASIC in about ten lines of code in the 1980s on a 1 Mhz 64K Apple II after reading about the famous Mark V. Shaney hoax (https://en.wikipedia.org/wiki/Mark_V._Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney). No neural nets or fancy GPUs required. It's one thing to stress that LLMs aren't magical or self-aware, but the fact is they are way more complicated than simple Markov chains.
- ftxbro 3y ago
  > People say that "they are literally Markov chains", but anyone who has looked at the code for LLMs knows that they are more complicated than that. They are literally Markov chains according to the mathematical definition. The code is complicated. Having complicated code doesn't mean it's not literally a Markov chain. > I implemented Markov chains in BASIC in about ten lines of code in the 1980s on a 1 Mhz 64K Apple II after reading about the famous Mark V. Shaney hoax (https://en.wikipedia.org/wiki/Mark_V._Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney). No neural nets or fancy GPUs required. I don't doubt this. You can make a Markov chain by just counting the frequency of letters that follow each letter giving one that has a context window of one or two characters. That is a very simple Markov chain. You can make it by hand. You can make ones with more context window like a dozen characters or a few words, using sophisticated smoothing and regularization methods and not just frequency counts. Those are also simple Markov chains that you can do without neural net or GPU. Then you can also make a Markov chain that has a context window of thousands of tokens that is made from neural nets and massive training data and differentiable tensor computing libraries with data centers full of hardware linear algebra accelerators. Those are some even bigger Markov chains! > LLMs are way more complicated than simple Markov chains. That's true, they are more complicated than simple Markov chains, if by simple Markov chains you mean ones with small context window. LLMs are Markov chains with large context window!
  - cubacaban 3y ago
    How big is the state space of the Markov chain corresponding to a LLM generating a sequence of tokens? Wouldn't it be (size of the vocabulary)^(size of the context window), i.e. ~ (100k)^(4k)? How useful is it to conceptualize LLMs as Markov chains at that point? For example, is there a result about Markov chains with interesting implications for LLMs?

FrustratedMonky 3y ago

I've been talking to GPT and asking it long multi part questions and it does seem to 'think through' them, and able to provide negative responses. Specific logic questions like in this post, or long math formulas seem to still be a struggle. But I get the impression it is just a matter of time before it is scaled up to handle them. You can argue whether it is 'reasoning' about them or not. But if it becomes indistinguishable from a human, then does the word 'reason' even really apply to the human anymore? Both are black boxes, giving answers.

nuancebydefault 3y ago
Indeed, anyone who says llm's cannot reason since it is just reformulating things it remembers, is unable to explain what reasoning is, let alone how it would not be reasoning. In the end, if the result is indistinguishable from a treat people call reasoning, let's call the behavior reasoning.

jhbadger 3y ago

I don't think so. Consider the following conversation from UndiMix-v2-13B (a local LLM running on my Mac M1 via llama.cpp). It gets it wrong at first, but then gets it right with a little hint. I don't see how the hint could work if it was merely repeating an example it knew. User: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? Llama: Hello there! I'd be happy to help you with that question. Let me think... If each of Sally's brothers has two sisters, then together they would have 6 sisters in total (2 x 3 = 6). So, Sally would have 6 sisters as well. Is that the answer you were looking for? User: no. Think about it. Sally is a sister to the brothers Llama: Ah, I see what you mean now! In this case, since Sally is one of the siblings, she only counts as one of her own sisters. Therefore, Sally has only one sister - herself. Does that make sense? User: yes Llama: Great! I hope that clears up any confusion. Please let me know if you have any other questions or need further assistance.

jameshart 3y ago
I don’t think ‘Sally has only one sister - herself’ is the right answer.
arnvald 3y ago
The initial response had 2 problems - not counting Sally and counting sisters separately for each brother. Your hint pointed out the 1st problem, but then LLM should still provide wrong solution. This makes me think that maybe it would try another answer and explanation no matter what you said. If your reply was just "no, think again" it could also provide the right answer
nuancebydefault 3y ago
She counts as her own sister :D I can be sister of myself?
1024core 3y ago
Llama: ... Therefore, Sally has only one sister - herself. Does that make sense? User: yes ---- Um... no!

thereticent 3y ago

I see no reason why not. You would need to represent dyadic relationships and whether they are reciprocal, transitive, etc., weigh probabilities about the intended meaning (LLM magic already), then use sentential logic rules, right?