4 ms·

It can't form plans because it has no idea what a plan is or how to implement it. The ONLY thing these LLMs know how to do is predict the probability that their

It can't form plans because it has no idea what a plan is or how to implement it. The ONLY thing these LLMs know how to do is predict the probability that their next word will make a human satisfied. That is all they do. People get very impressed when they prompt these things to pretend like they are sentient or capable of planning, but that's literally the point, its guessing which string of meaningless (to it) characters will result in a user giving it a thumbs up on the chatgpt website. You could teach me how to phonetically sound out some of China's greatest poetry in Chinese perfectly, and lots of people would be impressed, but I would be no more capable of understanding what I said than an LLM is capable of understanding "a plan".

directevolve 2y ago
… but ChatGPT can make a plan if I ask it to. And it can use a plan to guide its future outputs. It can create code or terminal commands that I can trivially output to my terminal, letting it operate my computer. From my computer, it can send commands to operate physical machinery. What exactly is the hard fundamental barrier here, as opposed to a capability you speculate it is unlikely to realize in practice in the next year or two?
- Jerrrrrrry 2y ago
  you are asking for goalposts? as if they were stationary!
- sickofparadox 2y ago
  Brother, it is not operating your computer, YOU ARE!
  - esafak 2y ago
    Nothing is preventing bad actors from using them to operate computers.
    - sickofparadox 2y ago
      I mean nothing is preventing bad actors from writing their own code to do that either? This makes it easier (kind of) but the difference between a copilot written malware and a human one doesn't really change anything. Its a chat bot - it doesn't have agency.
willy_k 2y ago
A plan is a set of steps oriented towards a specific goal, not some magical artifact only achievable through true consciousness. If you ask it to make a plan, it will spit out a sequence of characters reasonably indistinguishable from a human-made plan. Sure, it isn’t “planning” in the strict sense of organizing things consciously (whatever that actually means), but it can produce sequences of text that convey a plan, and it can produce sequences of text that mimic reasoning about a plan. Going into the semantics is pointless, imo the artificial part of AI/AGI means that it should never be expected to follow the same process as biological consciousness, just arrive at the same results.
- alfonsodev 2y ago
  Yes, and what people miss is that it can be recursive, those steps can be passed to other instances that know how to sub task each step and choose best executor for the step. The power comes in the swarm organization of the whole thing, which I believe is what is behind o1-preview, specialization and orchestration, made transparent.
highfrequency 2y ago
Sure, but does this distinction matter? Is an advanced computer program that very convincingly imitates a super villain less worrisome than an actual super villain?
MrScruff 2y ago
If the multimodal model has embedded deep knowledge about words, concepts, moving images - sure it won’t have a humanlike understanding of what those ‘mean’, but it will have it’s own understanding that is required to allow it to make better predictions based on it’s training data. It’s true that understanding is quite primitive at the moment, and it will likely take further breakthroughs to crack long horizon problems, but even when we get there it will never understand things in the exact way a human does. But I don’t think that’s the point.
smus 2y ago
>the ONLY thing these LLMs know how to do is predict the probability that their next word This is super incorrect. The base model is trained to predict the distribution of next words (which obviously necessitates a ton of understanding about the language) Then there's the RLHF step, which teaches the model about what humans want to see But o1 (which is one of these LLMs) is trained entirely differently to do reinforcement learning on problem solving (we think), so it's a pretty different paradigm. I could see o1 planning very well

directevolve 2y ago

… but ChatGPT can make a plan if I ask it to. And it can use a plan to guide its future outputs. It can create code or terminal commands that I can trivially output to my terminal, letting it operate my computer. From my computer, it can send commands to operate physical machinery. What exactly is the hard fundamental barrier here, as opposed to a capability you speculate it is unlikely to realize in practice in the next year or two?

Jerrrrrrry 2y ago
you are asking for goalposts? as if they were stationary!
sickofparadox 2y ago
Brother, it is not operating your computer, YOU ARE!
- esafak 2y ago
  Nothing is preventing bad actors from using them to operate computers.
  - sickofparadox 2y ago
    I mean nothing is preventing bad actors from writing their own code to do that either? This makes it easier (kind of) but the difference between a copilot written malware and a human one doesn't really change anything. Its a chat bot - it doesn't have agency.

willy_k 2y ago

A plan is a set of steps oriented towards a specific goal, not some magical artifact only achievable through true consciousness. If you ask it to make a plan, it will spit out a sequence of characters reasonably indistinguishable from a human-made plan. Sure, it isn’t “planning” in the strict sense of organizing things consciously (whatever that actually means), but it can produce sequences of text that convey a plan, and it can produce sequences of text that mimic reasoning about a plan. Going into the semantics is pointless, imo the artificial part of AI/AGI means that it should never be expected to follow the same process as biological consciousness, just arrive at the same results.

alfonsodev 2y ago
Yes, and what people miss is that it can be recursive, those steps can be passed to other instances that know how to sub task each step and choose best executor for the step. The power comes in the swarm organization of the whole thing, which I believe is what is behind o1-preview, specialization and orchestration, made transparent.

highfrequency 2y ago

Sure, but does this distinction matter? Is an advanced computer program that very convincingly imitates a super villain less worrisome than an actual super villain?

MrScruff 2y ago

If the multimodal model has embedded deep knowledge about words, concepts, moving images - sure it won’t have a humanlike understanding of what those ‘mean’, but it will have it’s own understanding that is required to allow it to make better predictions based on it’s training data. It’s true that understanding is quite primitive at the moment, and it will likely take further breakthroughs to crack long horizon problems, but even when we get there it will never understand things in the exact way a human does. But I don’t think that’s the point.

smus 2y ago

>the ONLY thing these LLMs know how to do is predict the probability that their next word This is super incorrect. The base model is trained to predict the distribution of next words (which obviously necessitates a ton of understanding about the language) Then there's the RLHF step, which teaches the model about what humans want to see But o1 (which is one of these LLMs) is trained entirely differently to do reinforcement learning on problem solving (we think), so it's a pretty different paradigm. I could see o1 planning very well