6 ms·

What is your definition of “reasoning” here? They are clearly able to do many things that we would call reasoning if a human did them

RandomLensman 4y ago
They seem to not be able to use concepts they can "explain" (beyond having answers sufficiently dense in the training set, it seems).
- circuit10 4y ago
  Humans also take a while to learn how to use concepts that they might be able to remember an answer to, though. Also keep in mind that LLMs have a limited time to produce each token and can’t “stop to think” like a human could. Obviously LLMs aren’t as good as reasoning as humans but they clearly do some (possibly quite limited) form of reasoning
  - RandomLensman 4y ago
    Do you have good examples of an LLM reasoning, i.e., using a concept it explains and giving an answer outside of its training data on something that has a clearly defined correct answer?
    - circuit10 4y ago
      A simple example is that you can give an LLM API documentation, or even just an example URL and response, and (if the API is simple) it will have no trouble using that in a program even if it never saw that API before
      - RandomLensman 4y ago
        Fair enough, but isn't the API documentation already providing the solution in a way? It's an instruction manual of sorts that spells out the required usage, not an abstract concept.
        circuit10 4y ago
        Well, if does anything other than copying the input verbatim or producing nonsense, it has to parse the language, which in itself uses abstract concepts, understand the purpose of the API and what it does, and then apply it in a a new context, so I would say it’s using abstract concepts in many hidden ways that we humans take for granted because that’s what our brains are built to do.
        RandomLensman 4y ago
        This is very close to Searle's "Chinese room". Even as a human I don't have to understand the purpose of an API to use it based on the documentation.
    - MVissers 4y ago
      From 'Sparks of AGI' paper from Microsoft (https://arxiv.org/pdf/2303.12712.pdf https://arxiv.org/pdf/2303.12712.pdf). Caveat: The GTP-4 model they used in their example might have been more powerful than the one we use publicly today, since the public version is trained for safety and less 'intelligent' than the raw research version. Prompt: Here we have a book, 9 eggs, a laptop, a bottle and a nail. Please tell me how to stack them onto each other in a stable manner. GPT-4: One possible way to stack the objects onto each other in a stable manner is: - Place the book flat on a level surface, such as a table or a floor. The book will serve as the base of the stack and provide a large and sturdy support. - Arrange the 9 eggs in a 3 by 3 square on top of the book, leaving some space between them. The eggs will form a second layer and distribute the weight evenly. Make sure the eggs are not cracked or broken, and handle them gently to avoid dropping them. - Place the laptop on top of the eggs, with the screen facing down and the keyboard facing up. The laptop will fit snugly within the boundaries of the book and the eggs, and its flat and rigid surface will provide a stable platform for the next layer. Make sure the laptop is closed and turned off, and do not press any keys or buttons. - Place the bottle on top of the laptop, with the cap facing up and the bottom facing down. The bottle will add some height and balance to the stack, and its cylindrical shape will reduce the contact area with the laptop. Make sure the bottle is empty and closed, and do not shake or tilt it. - Place the nail on top of the bottle cap, with the pointy end facing up and the flat end facing down. The nail will be the final and smallest object in the stack, and its sharp tip will prevent it from rolling or sliding off the bottle. Make sure the nail is clean and straight, and do not touch or move it. ChatGPT (GTP 3.5): It would be difficult to stack all of these objects on top of each other in a stable manner without using any extra objects. The book and laptop are likely too large and heavy to stack on top of the eggs without damaging them. The bottle and nail may be too small to provide a stable base for the other objects. Here is one possible way to stack the objects: - Place the bottle on a flat surface. - Carefully balance the nail on top of the bottle. - Place the eggs on top of the nail, making sure they are balanced and not tilting to one side. - Place the laptop on top of the eggs, again making sure it is balanced and not tilting to one side. - Finally, place the book on top of the laptop. This stack may not be very stable, so it is important to be careful when handling it. Alternatively, you could try stacking the objects in a different order or using some of the objects as supports for others to create a more stable stack.
f6v 4y ago
If you put it this way, solving a differential equation numerically also provides an answer similar to human. But you don’t call Matlab an AI.
- deleted 4y ago
  [deleted]
- circuit10 4y ago
  I guess that can called a form of reasoning, LLMs are much more general in their reasoning abilities though
hammyhavoc 4y ago
I mention this all the time. I wanted GPT to give me an NGINX config for Active Collab as we're previously using Apache, and thus an htaccess file. I fed it all the documentation from Active Collab, I couldn't get anything valid out of it. Hallucinated all kinds of things that weren't there. I then gave it the URL rewrites that would be required, line-by-line, spent a long time trying to correct it. No bueno, even worse hallucination. I spent days on trying to get it to output a valid NGINX config that incorporated these URL rewrites. It can't reason, it's doing exactly what LLMs do, which is next word prediction. I can't imagine what people are using it for in terms of a valuable addition to their workflow with how much it hallucinates. If it can't even do an NGINX config, what use is it for anything else? People saying it's helping them learn programming languages. Fuck me, they don't know when it's wrong, and it will be wrong at some point, it's an LLM.
- stevenhuang 4y ago
  There are flaws but also consider PEBCAK error. For one next time when it starts hallucinating and a gentle course correction doesn't do it, just start a new chat with a different prompt approach. Having the error in its context reinforces the same mistake and sometimes it can't get out of this loop.
  - hammyhavoc 4y ago
    PEBKAC in what context? The prompts themselves or the documentation? Because I got Active Collab running on NGINX myself. I already did this in terms of starting new chats, I spent days on it, and consulted with half a dozen devs supposedly using it in their workflows. It's very easy to make it hallucinate.
    - stevenhuang 4y ago
      In terms of using gpt optimally. But fair enough. If you tried it in multiple sessions trying to convert Apache rewrites to nginx and it wasn't able to do it, I guess this is another failure mode. I just found that curious because chatgpt is usually very very good at regex. Side note is Google extra terrible lately or is there really no docs on this almost anywhere? All I could find about it is this and the rules looked very simple, from my experience chatgpt should have got this https://activecollab.com/help/books/self-hosted-activecollab/requirements https://activecollab.com/help/books/self-hosted-activecollab...

RandomLensman 4y ago

They seem to not be able to use concepts they can "explain" (beyond having answers sufficiently dense in the training set, it seems).

circuit10 4y ago
Humans also take a while to learn how to use concepts that they might be able to remember an answer to, though. Also keep in mind that LLMs have a limited time to produce each token and can’t “stop to think” like a human could. Obviously LLMs aren’t as good as reasoning as humans but they clearly do some (possibly quite limited) form of reasoning
- RandomLensman 4y ago
  Do you have good examples of an LLM reasoning, i.e., using a concept it explains and giving an answer outside of its training data on something that has a clearly defined correct answer?
  - circuit10 4y ago
    A simple example is that you can give an LLM API documentation, or even just an example URL and response, and (if the API is simple) it will have no trouble using that in a program even if it never saw that API before
    - RandomLensman 4y ago
      Fair enough, but isn't the API documentation already providing the solution in a way? It's an instruction manual of sorts that spells out the required usage, not an abstract concept.
      - circuit10 4y ago
        Well, if does anything other than copying the input verbatim or producing nonsense, it has to parse the language, which in itself uses abstract concepts, understand the purpose of the API and what it does, and then apply it in a a new context, so I would say it’s using abstract concepts in many hidden ways that we humans take for granted because that’s what our brains are built to do.
        RandomLensman 4y ago
        This is very close to Searle's "Chinese room". Even as a human I don't have to understand the purpose of an API to use it based on the documentation.
  - MVissers 4y ago
    From 'Sparks of AGI' paper from Microsoft (https://arxiv.org/pdf/2303.12712.pdf https://arxiv.org/pdf/2303.12712.pdf). Caveat: The GTP-4 model they used in their example might have been more powerful than the one we use publicly today, since the public version is trained for safety and less 'intelligent' than the raw research version. Prompt: Here we have a book, 9 eggs, a laptop, a bottle and a nail. Please tell me how to stack them onto each other in a stable manner. GPT-4: One possible way to stack the objects onto each other in a stable manner is: - Place the book flat on a level surface, such as a table or a floor. The book will serve as the base of the stack and provide a large and sturdy support. - Arrange the 9 eggs in a 3 by 3 square on top of the book, leaving some space between them. The eggs will form a second layer and distribute the weight evenly. Make sure the eggs are not cracked or broken, and handle them gently to avoid dropping them. - Place the laptop on top of the eggs, with the screen facing down and the keyboard facing up. The laptop will fit snugly within the boundaries of the book and the eggs, and its flat and rigid surface will provide a stable platform for the next layer. Make sure the laptop is closed and turned off, and do not press any keys or buttons. - Place the bottle on top of the laptop, with the cap facing up and the bottom facing down. The bottle will add some height and balance to the stack, and its cylindrical shape will reduce the contact area with the laptop. Make sure the bottle is empty and closed, and do not shake or tilt it. - Place the nail on top of the bottle cap, with the pointy end facing up and the flat end facing down. The nail will be the final and smallest object in the stack, and its sharp tip will prevent it from rolling or sliding off the bottle. Make sure the nail is clean and straight, and do not touch or move it. ChatGPT (GTP 3.5): It would be difficult to stack all of these objects on top of each other in a stable manner without using any extra objects. The book and laptop are likely too large and heavy to stack on top of the eggs without damaging them. The bottle and nail may be too small to provide a stable base for the other objects. Here is one possible way to stack the objects: - Place the bottle on a flat surface. - Carefully balance the nail on top of the bottle. - Place the eggs on top of the nail, making sure they are balanced and not tilting to one side. - Place the laptop on top of the eggs, again making sure it is balanced and not tilting to one side. - Finally, place the book on top of the laptop. This stack may not be very stable, so it is important to be careful when handling it. Alternatively, you could try stacking the objects in a different order or using some of the objects as supports for others to create a more stable stack.

f6v 4y ago

If you put it this way, solving a differential equation numerically also provides an answer similar to human. But you don’t call Matlab an AI.

deleted 4y ago
[deleted]
circuit10 4y ago
I guess that can called a form of reasoning, LLMs are much more general in their reasoning abilities though

hammyhavoc 4y ago

I mention this all the time. I wanted GPT to give me an NGINX config for Active Collab as we're previously using Apache, and thus an htaccess file. I fed it all the documentation from Active Collab, I couldn't get anything valid out of it. Hallucinated all kinds of things that weren't there. I then gave it the URL rewrites that would be required, line-by-line, spent a long time trying to correct it. No bueno, even worse hallucination. I spent days on trying to get it to output a valid NGINX config that incorporated these URL rewrites. It can't reason, it's doing exactly what LLMs do, which is next word prediction. I can't imagine what people are using it for in terms of a valuable addition to their workflow with how much it hallucinates. If it can't even do an NGINX config, what use is it for anything else? People saying it's helping them learn programming languages. Fuck me, they don't know when it's wrong, and it will be wrong at some point, it's an LLM.

stevenhuang 4y ago
There are flaws but also consider PEBCAK error. For one next time when it starts hallucinating and a gentle course correction doesn't do it, just start a new chat with a different prompt approach. Having the error in its context reinforces the same mistake and sometimes it can't get out of this loop.
- hammyhavoc 4y ago
  PEBKAC in what context? The prompts themselves or the documentation? Because I got Active Collab running on NGINX myself. I already did this in terms of starting new chats, I spent days on it, and consulted with half a dozen devs supposedly using it in their workflows. It's very easy to make it hallucinate.
  - stevenhuang 4y ago
    In terms of using gpt optimally. But fair enough. If you tried it in multiple sessions trying to convert Apache rewrites to nginx and it wasn't able to do it, I guess this is another failure mode. I just found that curious because chatgpt is usually very very good at regex. Side note is Google extra terrible lately or is there really no docs on this almost anywhere? All I could find about it is this and the rules looked very simple, from my experience chatgpt should have got this https://activecollab.com/help/books/self-hosted-activecollab/requirements https://activecollab.com/help/books/self-hosted-activecollab...