4 ms·

We are anthropomorphizing whenever we refer to prompts as instructions to models. They predict text not obey our orders.

gigatree 6mo ago
That’s not how language works, just how engineers think it works
- getpokedagain 6mo ago
  This isn't a sarcastic response. What do you mean?
  - gigatree 6mo ago
    I just mean that the argument that words like “instructions”, “think”, “confess” are inaccurate when used in reference to a machine assumes that those words can only refer to humans/conscious beings, when really they can refer to more than that if used widely enough in those ways (in this case - text prediction following a human input). So it’s not “anthropomorphizing” because when people use those words they don’t [typically] actually believe the machine can think or reason, it’s just the word that most closely matches the concept, it’s convenient. You’re extending the definition of the words to apply to non-conscious entities too, not applying consciousness to the entities. It’s the same reason we call the handheld device we carry around to do everything a “phone” without a second thought. We don’t call it a phone because it’s primary purpose is calling, we call it a phone because the definition of the word “phone” has grown to include “navigates, entertains, takes pictures, etc”.
    - getpokedagain 6mo ago
      Thanks!
DiogenesKynikos 6mo ago
> They predict text not obey our orders. Those are the same thing in this case. The latter is just an extremely reductionist description of the mechanics behind the former.
- grey-area 6mo ago
  They are not in fact the same thing, and the difference is important. They are certainly marketed as if they think, learn and follow orders, but they do not.
  - DiogenesKynikos 6mo ago
    The result of "predicting text" is that they obey orders, just like the result of "random electrochemical impulses in synapses" is that you typed your comment. You can always reduce high-level phenomena to lower-level mechanisms. That doesn't mean that the high-level phenomenon doesn't exist. LLMs are obviously able to understand and follow instructions.
    - grey-area 6mo ago
      > The result of "predicting text" is that they obey orders And yet they don't, quite a lot of the time, and in a random way that is hard to predict or even notice sometimes (their errors can be important but subtle/small). They're simply not reliable enough to treat as independent agents, and this story is a good example of why not.
      - DiogenesKynikos 6mo ago
        First, they do follow instructions most of the time, and the leading models get better and better at doing it month for month. Second, whether they're perfect at following commands is besides the point. They're not just "predicting tokens," in the same way you're not just "sending electrochemical signals." LLMs think, solve problems, answer questions, write code, etc.

gigatree 6mo ago

That’s not how language works, just how engineers think it works

getpokedagain 6mo ago
This isn't a sarcastic response. What do you mean?
- gigatree 6mo ago
  I just mean that the argument that words like “instructions”, “think”, “confess” are inaccurate when used in reference to a machine assumes that those words can only refer to humans/conscious beings, when really they can refer to more than that if used widely enough in those ways (in this case - text prediction following a human input). So it’s not “anthropomorphizing” because when people use those words they don’t [typically] actually believe the machine can think or reason, it’s just the word that most closely matches the concept, it’s convenient. You’re extending the definition of the words to apply to non-conscious entities too, not applying consciousness to the entities. It’s the same reason we call the handheld device we carry around to do everything a “phone” without a second thought. We don’t call it a phone because it’s primary purpose is calling, we call it a phone because the definition of the word “phone” has grown to include “navigates, entertains, takes pictures, etc”.
  - getpokedagain 6mo ago
    Thanks!

DiogenesKynikos 6mo ago

> They predict text not obey our orders. Those are the same thing in this case. The latter is just an extremely reductionist description of the mechanics behind the former.

grey-area 6mo ago
They are not in fact the same thing, and the difference is important. They are certainly marketed as if they think, learn and follow orders, but they do not.
- DiogenesKynikos 6mo ago
  The result of "predicting text" is that they obey orders, just like the result of "random electrochemical impulses in synapses" is that you typed your comment. You can always reduce high-level phenomena to lower-level mechanisms. That doesn't mean that the high-level phenomenon doesn't exist. LLMs are obviously able to understand and follow instructions.
  - grey-area 6mo ago
    > The result of "predicting text" is that they obey orders And yet they don't, quite a lot of the time, and in a random way that is hard to predict or even notice sometimes (their errors can be important but subtle/small). They're simply not reliable enough to treat as independent agents, and this story is a good example of why not.
    - DiogenesKynikos 6mo ago
      First, they do follow instructions most of the time, and the leading models get better and better at doing it month for month. Second, whether they're perfect at following commands is besides the point. They're not just "predicting tokens," in the same way you're not just "sending electrochemical signals." LLMs think, solve problems, answer questions, write code, etc.