3 ms·

Again its not reasoning. Reasoning would imply that it can figure out stuff without being trained in it. The chain of thought is basically just a more accurat

Again its not reasoning. Reasoning would imply that it can figure out stuff without being trained in it. The chain of thought is basically just a more accurate way to map input to output. But its still a map, i.e forward only. If an LLM coud reason, you should be able to ask it a question about how to make a bicycle frame from scratch with a small home cnc with limited work area and it should be able to iterate on an analysis of the best way to put it together, using internet to look up available parts and make decisions on optimization. No LLM can do that or even come close, because there are no real feedback loops, because nobody knows how to train a network like that.

strgcmc 2y ago
I think you need to re-calibrate your expectations... I'm not saying this is a solved problem by any means, but I just tried this out with Claude Sonnet 3.5, and these instructions seem quite reasonable and detailed to me (about what I might expect if I spoke to a human expert and they tried to explain the steps to me over the telephone, for example). Does this mean this LLM is able to "reason"? I don't know that I would make THAT bold of a claim, but I think your example is not sufficient to demonstrate something that LLMs are fundamentally incapable of... in other words, the distance between "normal LLM statistical tricks" vs "reasoning" keeps getting smaller and smaller. --- My base prompt: > Here is a hypothetical scenario, that I would like your help with: imagine you are trying to help a person create a bicycle frame, using their home workshop which includes a CNC machine, commonly available tools, a reasonable supply of raw metal and hardware, etc. Please provide a written set of instructions, that you would give to this person so that they can complete this task. First answer: https://claude.site/artifacts/f8af03ba-3f2c-497d-b564-a19baf2b6d4d https://claude.site/artifacts/f8af03ba-3f2c-497d-b564-a19baf... My follow-up, pressing for actual measurements: > Can you suggest some standard options for bike geometry, assuming an average sized human male? Answer including specific dimensions: https://claude.site/artifacts/2f5ea2f3-69d8-4a1b-a563-15d3343ad3be https://claude.site/artifacts/2f5ea2f3-69d8-4a1b-a563-15d334...
- ActorNightly 2y ago
  I don't have high expectations of this stuff. Im just saying what its doing is not reasoning.
  - strgcmc 2y ago
    And all I'm saying is that, you probably need a different example of what "reasoning" is, because the one you gave is something that Claude is seemingly able to do.
    - ActorNightly 2y ago
      What? Claude is definitely not able to do what I asked, not even close. Do you think thats an acceptable answer that shows reasoning?
crazygringo 2y ago
I think you might be confusing two concepts here. It's definitely reasoning. We can watch that in action, whatever the mechanism behind it is. But it's not doing long-term learning, it's not updating its model.
- gdhkgdhkvff 2y ago
  Even long term learning it does to some extent. Admittedly I’m not very familiar with what it’s doing, but it does create “memories” which appear to be personal details that it deems might be relevant in the future. Then I assume it uses some type of RAG to apply previously learned memories to future conversations. This makes me wonder if there is or could be some type of RAG for chains of thought…
- ActorNightly 2y ago
  >whatever the mechanism behind it is. The mechanism is that there is an additional model that basically outputs chain of thought for a particular problem, then runs the chain of thought through the core LLM. This is no different from just a complex forward map lookup. I mean, its incredibly useful, but its still just information search.
gdhkgdhkvff 2y ago
It’s like every single sentence you just wrote is incorrect? 1. You’re making up some weird goalposts here of what it means to reason. It’s not reasoning unless it can access the internet to search for parts? No. That has nothing to do with reasoning. You just think it would be cool if it could do that. 2. “Can figure out stuff without being trained on it” That’s exactly what it’s doing in the cypher example. It wasn’t trained to know that that input meant the corresponding output through the cypher. Emergent reasoning through autocomplete, sure, but that’s still reasoning. 3. “Forward only”. If that was the case then back and forth conversations with the llm would be pointless. It wouldn’t be able to improve upon previous answers it gave you when you give it new details. But that’s not how it works. If tell it one thing, then separately tell it another thing, it can change its original conclusion based on your new input. 4. Even desolate your convoluted test for reasoning, ChatGPT CAN do what you asked… even using the internet to look up parts it can either do out of the box or could do if given a plug-in to allow that.
- ActorNightly 2y ago
  Ill give you a more formal definition. A model can be said as reasoning, when it can use existing information to figure out new data that has not been in the training set. Here is a better example - lets say your input is 6 pictures of some object from each of the cardinal viewpoints, and you tell model these are the views and ask it how much it weighs. The model should basically figure out how to create a 3d shape and compute a camera view, and iterate until the camera view matches the pictures, then figure out that the shape can be hollow or solid, and to compute the weight you need the density, and that it should prompt the user for it if it cannot determine the true value for those from the information and its trained dataset. And it should do it without any specific training that this is the right way to do this, because it should be able to figure out this way through breaking the problem down into abstract representations of sub problems, and then figuring out how to solve those through basic logic, a.k.a reasoning. What that looks like, I don't know. If I did I would certainly have my own AI company. But i can tell you for certain we are not even close to figuring it out yet, because everyone is still stuck on transformers, like multiplying matricies together is some groundbreaking thing. In the cypher example, all its doing is basically using a separate model to break a particular model into chain of thought, and prompting that. And there is plenty in the training set of GPT about decrypting cyphers. >Forward only What I mean is that when its generate a response, the computation happens on a snapshot from input to output, trying to map a set of tokens, into a set of tokens. Model doesn't operate on a context larger than the window. Humans don't do this. We operate on a large context, with lots of previous information compressed, and furthermore, we don't just compute words, we compute complex abstract ideas that we then can translate into words. >even using the internet to look up parts it can either do out of the box or could do if given a plug-in to allow that. So apparently the way to AI is to manually code all the capability into LLMS? Give me a break. Just like with Chat GPT4, when people were screaming about how its the birth of true AI, give this model a year, it will find some niche use cases (depending on cost), and then nobody is going give a fuck about it, just like nobody is really doing anything groundbreaking with GPT4.

strgcmc 2y ago

I think you need to re-calibrate your expectations... I'm not saying this is a solved problem by any means, but I just tried this out with Claude Sonnet 3.5, and these instructions seem quite reasonable and detailed to me (about what I might expect if I spoke to a human expert and they tried to explain the steps to me over the telephone, for example). Does this mean this LLM is able to "reason"? I don't know that I would make THAT bold of a claim, but I think your example is not sufficient to demonstrate something that LLMs are fundamentally incapable of... in other words, the distance between "normal LLM statistical tricks" vs "reasoning" keeps getting smaller and smaller. --- My base prompt: > Here is a hypothetical scenario, that I would like your help with: imagine you are trying to help a person create a bicycle frame, using their home workshop which includes a CNC machine, commonly available tools, a reasonable supply of raw metal and hardware, etc. Please provide a written set of instructions, that you would give to this person so that they can complete this task. First answer: https://claude.site/artifacts/f8af03ba-3f2c-497d-b564-a19baf2b6d4d https://claude.site/artifacts/f8af03ba-3f2c-497d-b564-a19baf... My follow-up, pressing for actual measurements: > Can you suggest some standard options for bike geometry, assuming an average sized human male? Answer including specific dimensions: https://claude.site/artifacts/2f5ea2f3-69d8-4a1b-a563-15d3343ad3be https://claude.site/artifacts/2f5ea2f3-69d8-4a1b-a563-15d334...

ActorNightly 2y ago
I don't have high expectations of this stuff. Im just saying what its doing is not reasoning.
- strgcmc 2y ago
  And all I'm saying is that, you probably need a different example of what "reasoning" is, because the one you gave is something that Claude is seemingly able to do.
  - ActorNightly 2y ago
    What? Claude is definitely not able to do what I asked, not even close. Do you think thats an acceptable answer that shows reasoning?

crazygringo 2y ago

I think you might be confusing two concepts here. It's definitely reasoning. We can watch that in action, whatever the mechanism behind it is. But it's not doing long-term learning, it's not updating its model.

gdhkgdhkvff 2y ago
Even long term learning it does to some extent. Admittedly I’m not very familiar with what it’s doing, but it does create “memories” which appear to be personal details that it deems might be relevant in the future. Then I assume it uses some type of RAG to apply previously learned memories to future conversations. This makes me wonder if there is or could be some type of RAG for chains of thought…
ActorNightly 2y ago
>whatever the mechanism behind it is. The mechanism is that there is an additional model that basically outputs chain of thought for a particular problem, then runs the chain of thought through the core LLM. This is no different from just a complex forward map lookup. I mean, its incredibly useful, but its still just information search.

gdhkgdhkvff 2y ago

It’s like every single sentence you just wrote is incorrect? 1. You’re making up some weird goalposts here of what it means to reason. It’s not reasoning unless it can access the internet to search for parts? No. That has nothing to do with reasoning. You just think it would be cool if it could do that. 2. “Can figure out stuff without being trained on it” That’s exactly what it’s doing in the cypher example. It wasn’t trained to know that that input meant the corresponding output through the cypher. Emergent reasoning through autocomplete, sure, but that’s still reasoning. 3. “Forward only”. If that was the case then back and forth conversations with the llm would be pointless. It wouldn’t be able to improve upon previous answers it gave you when you give it new details. But that’s not how it works. If tell it one thing, then separately tell it another thing, it can change its original conclusion based on your new input. 4. Even desolate your convoluted test for reasoning, ChatGPT CAN do what you asked… even using the internet to look up parts it can either do out of the box or could do if given a plug-in to allow that.

ActorNightly 2y ago
Ill give you a more formal definition. A model can be said as reasoning, when it can use existing information to figure out new data that has not been in the training set. Here is a better example - lets say your input is 6 pictures of some object from each of the cardinal viewpoints, and you tell model these are the views and ask it how much it weighs. The model should basically figure out how to create a 3d shape and compute a camera view, and iterate until the camera view matches the pictures, then figure out that the shape can be hollow or solid, and to compute the weight you need the density, and that it should prompt the user for it if it cannot determine the true value for those from the information and its trained dataset. And it should do it without any specific training that this is the right way to do this, because it should be able to figure out this way through breaking the problem down into abstract representations of sub problems, and then figuring out how to solve those through basic logic, a.k.a reasoning. What that looks like, I don't know. If I did I would certainly have my own AI company. But i can tell you for certain we are not even close to figuring it out yet, because everyone is still stuck on transformers, like multiplying matricies together is some groundbreaking thing. In the cypher example, all its doing is basically using a separate model to break a particular model into chain of thought, and prompting that. And there is plenty in the training set of GPT about decrypting cyphers. >Forward only What I mean is that when its generate a response, the computation happens on a snapshot from input to output, trying to map a set of tokens, into a set of tokens. Model doesn't operate on a context larger than the window. Humans don't do this. We operate on a large context, with lots of previous information compressed, and furthermore, we don't just compute words, we compute complex abstract ideas that we then can translate into words. >even using the internet to look up parts it can either do out of the box or could do if given a plug-in to allow that. So apparently the way to AI is to manually code all the capability into LLMS? Give me a break. Just like with Chat GPT4, when people were screaming about how its the birth of true AI, give this model a year, it will find some niche use cases (depending on cost), and then nobody is going give a fuck about it, just like nobody is really doing anything groundbreaking with GPT4.