3 ms·

You have to be careful when anthropomorphizing AI models. Yes, goal-driven optimization is a thing, but in what sense does the model itself "want" to achieve it

You have to be careful when anthropomorphizing AI models. Yes, goal-driven optimization is a thing, but in what sense does the model itself "want" to achieve its goal? Can it even understand its own goal, in any sense? Change it? Improve it? In linear programming, you wouldn't describe the model as "wanting" to optimize its objective function, for instance.

refactor_master 6y ago
Well, where do you set the hard limit for “wanting” something, and how many grains of sand makes a pile?
Isinlor 6y ago
Can you understand your own goal? I can't, I don't know what my goal is. I also can't change it or improve it. As far as I can see, I'm just a process that keeps on going because it was good on going before, and has no purpose whatsoever.
- pedrosbmartins 6y ago
  Of course you can understand your own goals. You decided to reply to my comment, that's a goal. You used your existing knowledge of the world, language and technological tools to achieve it. And you did! Can a goal-oriented model do something like that, in a general sense? Note that I'm not talking about life purpose, but goals in the sense of wanting a result and performing the tasks needed to achieve it. Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand.
  - Isinlor 6y ago
    > Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand. You were asking: in what sense does the model itself "want" to achieve its goal? IMO there is no much difference between my own "want" and "want" of a simpler model. Granted, I am more skilled at achieving instrumental goals under more varied set of conditions. But as I said, I do not understand why I have the goals I have. I also strongly believe that I do not control my own goals. There is no mechanism in physics that would allow for control of goals.
    - pedrosbmartins 6y ago
      > Granted, I am more skilled at achieving instrumental goals under more varied set of conditions. Then we are in agreement. This statement is right around the threshold where we can take this discussion before coming to metaphysics territory. My argument was simply this: the AI technology we have now is much closer to simpler "mindless" models, like Linear Programming or inference statistics models, than to general human intelligence. Calling your model "goal-oriented" or saying that an AI system can, say, "see dogs in a photo" is simply an anthropomorphization.

refactor_master 6y ago

Well, where do you set the hard limit for “wanting” something, and how many grains of sand makes a pile?

Isinlor 6y ago

Can you understand your own goal? I can't, I don't know what my goal is. I also can't change it or improve it. As far as I can see, I'm just a process that keeps on going because it was good on going before, and has no purpose whatsoever.

pedrosbmartins 6y ago
Of course you can understand your own goals. You decided to reply to my comment, that's a goal. You used your existing knowledge of the world, language and technological tools to achieve it. And you did! Can a goal-oriented model do something like that, in a general sense? Note that I'm not talking about life purpose, but goals in the sense of wanting a result and performing the tasks needed to achieve it. Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand.
- Isinlor 6y ago
  > Asking if your wants are "real" or just part of a purposeless process doesn't really add much to the discussion at hand. You were asking: in what sense does the model itself "want" to achieve its goal? IMO there is no much difference between my own "want" and "want" of a simpler model. Granted, I am more skilled at achieving instrumental goals under more varied set of conditions. But as I said, I do not understand why I have the goals I have. I also strongly believe that I do not control my own goals. There is no mechanism in physics that would allow for control of goals.
  - pedrosbmartins 6y ago
    > Granted, I am more skilled at achieving instrumental goals under more varied set of conditions. Then we are in agreement. This statement is right around the threshold where we can take this discussion before coming to metaphysics territory. My argument was simply this: the AI technology we have now is much closer to simpler "mindless" models, like Linear Programming or inference statistics models, than to general human intelligence. Calling your model "goal-oriented" or saying that an AI system can, say, "see dogs in a photo" is simply an anthropomorphization.