4 ms·

Seriously. I actually feel as impressed by the chain of thought, as I was when ChatGPT first came out. This isn't "just" autocompletion anymore, this is actual

Seriously. I actually feel as impressed by the chain of thought, as I was when ChatGPT first came out. This isn't "just" autocompletion anymore, this is actual step-by-step reasoning full of ideas and dead ends and refinement, just like humans do when solving problems. Even if it is still ultimately being powered by "autocompletion". But then it makes me wonder about human reasoning, and what if it's similar? Just following basic patterns of "thinking steps" that ultimately aren't any different from "English language grammar steps"? This is truly making me wonder if LLM's are actually far more powerful than we thought at first, and if it's just a matter of figuring out how to plug them together in the right configurations, like "making them think".

AndyKelley 2y ago
You ever see that scene from Westworld? (spoiler) https://www.youtube.com/watch?v=ZnxJRYit44k https://www.youtube.com/watch?v=ZnxJRYit44k
dsign 2y ago
You are just catching up to this idea, probably after hearing 2^n explanations about why we humans are superiors to <<fill in here our latest creation>>. I'm not the kind of scientist that can say how good an LLM is for human reasoning, but I know that we humans are very incentivized and kind of good at scaling, composing and perfecting things. If there is money to pay for human effort, we will play God no-problem, and maybe outdo the divine. Which makes me wonder, isn't there any other problem in our bucket list to dump ginormous amounts of effort at... maybe something more worth-while than engineering the thing that will replace Homo Sapiens?
Nadya 2y ago
When an AI makes a silly math mistake we say it is bad at math and laugh at how dumb it is. Some people extrapolate this to "they'll never get any better and will always be a dumb toy that gets things wrong". When I forget to carry a 1 when doing a math problem we call it "human error" even if I make that mistake an embarrassing number of times throughout my lifetime. Do I think LLM's are alive/close to ASI? No. Will they get there? If it's even at all possible - almost certainly one day. Do I think people severely underestimate AI's ability to solve problems while significantly overestimating their own? Absolutely 10,000%. If there is one thing I've learned from watching the AI discussion over the past 10-20 years its that people have overinflated egos and a crazy amount of hubris. "Today is the worst that it will ever be." applies to an awful large number of things that people work on creating and improving.
tsoj 2y ago
Yeah, humans are very similar. We have intuitive immediate-next-step-suggestions, and then we apply these intuitive next steps, until we find that it lead to a dead end, and then we backtrack. I always say, the way we used LLMs (so far) is basically like having a human write text only on gut reactions, and without backspace key.
exe34 2y ago
that's my assessment too. there's even a phenomenon I've observed both in others and myself, when thrust into a new field and given a task to complete, we do it to the best of our ability, which is often sod all. so we ape the things we've heard others say, roughly following the right chain of reasoning by luck, and then suddenly say something that in hind sight, with proper training, we realise was incredibly stupid. we autocomplete and then update with rlhf. we also have a ton of heuristics that trigger a closer look and loading of specific formal reasoning, but by and large, most of our thought process is just auto complete.
armchairhacker 2y ago
I think it's similar, although I think it would be more similar if the LLM did the steps in lower layers (not in English), and instead of the end being fed to the start, there would be a big mess of cycles throughout the neural net. That could be more efficient since the cycles are much smaller, but harder to train.
- vanviegen 2y ago
  It doesn't do the 'thinking' in English (inference is just math), but it does now verbalize intermediate thoughts in English (or whatever the input language is, presumably), just like humans tend to do.
  - konschubert 2y ago
    Agreed. It we never "just autocomplete" unless your definition of "autocomplete" includes "look at the whole body of text".
ActorNightly 2y ago
Again its not reasoning. Reasoning would imply that it can figure out stuff without being trained in it. The chain of thought is basically just a more accurate way to map input to output. But its still a map, i.e forward only. If an LLM coud reason, you should be able to ask it a question about how to make a bicycle frame from scratch with a small home cnc with limited work area and it should be able to iterate on an analysis of the best way to put it together, using internet to look up available parts and make decisions on optimization. No LLM can do that or even come close, because there are no real feedback loops, because nobody knows how to train a network like that.
- strgcmc 2y ago
  I think you need to re-calibrate your expectations... I'm not saying this is a solved problem by any means, but I just tried this out with Claude Sonnet 3.5, and these instructions seem quite reasonable and detailed to me (about what I might expect if I spoke to a human expert and they tried to explain the steps to me over the telephone, for example). Does this mean this LLM is able to "reason"? I don't know that I would make THAT bold of a claim, but I think your example is not sufficient to demonstrate something that LLMs are fundamentally incapable of... in other words, the distance between "normal LLM statistical tricks" vs "reasoning" keeps getting smaller and smaller. --- My base prompt: > Here is a hypothetical scenario, that I would like your help with: imagine you are trying to help a person create a bicycle frame, using their home workshop which includes a CNC machine, commonly available tools, a reasonable supply of raw metal and hardware, etc. Please provide a written set of instructions, that you would give to this person so that they can complete this task. First answer: https://claude.site/artifacts/f8af03ba-3f2c-497d-b564-a19baf2b6d4d https://claude.site/artifacts/f8af03ba-3f2c-497d-b564-a19baf... My follow-up, pressing for actual measurements: > Can you suggest some standard options for bike geometry, assuming an average sized human male? Answer including specific dimensions: https://claude.site/artifacts/2f5ea2f3-69d8-4a1b-a563-15d3343ad3be https://claude.site/artifacts/2f5ea2f3-69d8-4a1b-a563-15d334...
  - ActorNightly 2y ago
    I don't have high expectations of this stuff. Im just saying what its doing is not reasoning.
BriggyDwiggs42 2y ago
An exception i came up with was from a documentary on einstein that described how he did his thought experiments. He would, of course, imagine novel scenarios in his head, which led him to the insights he could rephrase into language. I worry language models will still lack that capacity for insights driven by imagination.

AndyKelley 2y ago

You ever see that scene from Westworld? (spoiler) https://www.youtube.com/watch?v=ZnxJRYit44k https://www.youtube.com/watch?v=ZnxJRYit44k

dsign 2y ago

You are just catching up to this idea, probably after hearing 2^n explanations about why we humans are superiors to <<fill in here our latest creation>>. I'm not the kind of scientist that can say how good an LLM is for human reasoning, but I know that we humans are very incentivized and kind of good at scaling, composing and perfecting things. If there is money to pay for human effort, we will play God no-problem, and maybe outdo the divine. Which makes me wonder, isn't there any other problem in our bucket list to dump ginormous amounts of effort at... maybe something more worth-while than engineering the thing that will replace Homo Sapiens?

Nadya 2y ago

When an AI makes a silly math mistake we say it is bad at math and laugh at how dumb it is. Some people extrapolate this to "they'll never get any better and will always be a dumb toy that gets things wrong". When I forget to carry a 1 when doing a math problem we call it "human error" even if I make that mistake an embarrassing number of times throughout my lifetime. Do I think LLM's are alive/close to ASI? No. Will they get there? If it's even at all possible - almost certainly one day. Do I think people severely underestimate AI's ability to solve problems while significantly overestimating their own? Absolutely 10,000%. If there is one thing I've learned from watching the AI discussion over the past 10-20 years its that people have overinflated egos and a crazy amount of hubris. "Today is the worst that it will ever be." applies to an awful large number of things that people work on creating and improving.

tsoj 2y ago

Yeah, humans are very similar. We have intuitive immediate-next-step-suggestions, and then we apply these intuitive next steps, until we find that it lead to a dead end, and then we backtrack. I always say, the way we used LLMs (so far) is basically like having a human write text only on gut reactions, and without backspace key.

exe34 2y ago

that's my assessment too. there's even a phenomenon I've observed both in others and myself, when thrust into a new field and given a task to complete, we do it to the best of our ability, which is often sod all. so we ape the things we've heard others say, roughly following the right chain of reasoning by luck, and then suddenly say something that in hind sight, with proper training, we realise was incredibly stupid. we autocomplete and then update with rlhf. we also have a ton of heuristics that trigger a closer look and loading of specific formal reasoning, but by and large, most of our thought process is just auto complete.

armchairhacker 2y ago

I think it's similar, although I think it would be more similar if the LLM did the steps in lower layers (not in English), and instead of the end being fed to the start, there would be a big mess of cycles throughout the neural net. That could be more efficient since the cycles are much smaller, but harder to train.

vanviegen 2y ago
It doesn't do the 'thinking' in English (inference is just math), but it does now verbalize intermediate thoughts in English (or whatever the input language is, presumably), just like humans tend to do.
- konschubert 2y ago
  Agreed. It we never "just autocomplete" unless your definition of "autocomplete" includes "look at the whole body of text".

ActorNightly 2y ago

Again its not reasoning. Reasoning would imply that it can figure out stuff without being trained in it. The chain of thought is basically just a more accurate way to map input to output. But its still a map, i.e forward only. If an LLM coud reason, you should be able to ask it a question about how to make a bicycle frame from scratch with a small home cnc with limited work area and it should be able to iterate on an analysis of the best way to put it together, using internet to look up available parts and make decisions on optimization. No LLM can do that or even come close, because there are no real feedback loops, because nobody knows how to train a network like that.

strgcmc 2y ago
I think you need to re-calibrate your expectations... I'm not saying this is a solved problem by any means, but I just tried this out with Claude Sonnet 3.5, and these instructions seem quite reasonable and detailed to me (about what I might expect if I spoke to a human expert and they tried to explain the steps to me over the telephone, for example). Does this mean this LLM is able to "reason"? I don't know that I would make THAT bold of a claim, but I think your example is not sufficient to demonstrate something that LLMs are fundamentally incapable of... in other words, the distance between "normal LLM statistical tricks" vs "reasoning" keeps getting smaller and smaller. --- My base prompt: > Here is a hypothetical scenario, that I would like your help with: imagine you are trying to help a person create a bicycle frame, using their home workshop which includes a CNC machine, commonly available tools, a reasonable supply of raw metal and hardware, etc. Please provide a written set of instructions, that you would give to this person so that they can complete this task. First answer: https://claude.site/artifacts/f8af03ba-3f2c-497d-b564-a19baf2b6d4d https://claude.site/artifacts/f8af03ba-3f2c-497d-b564-a19baf... My follow-up, pressing for actual measurements: > Can you suggest some standard options for bike geometry, assuming an average sized human male? Answer including specific dimensions: https://claude.site/artifacts/2f5ea2f3-69d8-4a1b-a563-15d3343ad3be https://claude.site/artifacts/2f5ea2f3-69d8-4a1b-a563-15d334...
- ActorNightly 2y ago
  I don't have high expectations of this stuff. Im just saying what its doing is not reasoning.

BriggyDwiggs42 2y ago

An exception i came up with was from a documentary on einstein that described how he did his thought experiments. He would, of course, imagine novel scenarios in his head, which led him to the insights he could rephrase into language. I worry language models will still lack that capacity for insights driven by imagination.