3 ms·
it knows that if it modifies its utility function it is liable to collect less ore, which is something it doesn't want. Why would it want or not want anything,
by harshreality 15y ago
it knows that if it modifies its utility function it is liable to collect less ore, which is something it doesn't want.
Why would it want or not want anything, if it doesn't have a pleasure construct (which might also be called a motivation construct, since an AI might not be capable of the same subjective experience of pleasure that we are)?
I think it's a question of program design whether there's a utility function which decides whether to trigger the pleasure construct, or whether certain sensory input modules directly trigger the pleasure construct. To limit hacking potential, routing everything through a tamper-proof utility function might be better, except that it would also limit the AI's adaptability (short of recreating its own hardware to remove the tamper-proof module... which it might never do depending on the details of its motivation construct).
- robertskmiles 15y ago> Why would it want or not want anything, if it doesn't have a pleasure construct Why does a calculator want to add, without a pleasure construct? If it doesn't enjoy giving the right answer to 2 + 3, what motivates it to choose 5? The answer is it doesn't necessarily 'want' to do it, it just does it. That was how it was programmed. You program the AI to choose the plan that maximises the expected utility function. Someone looking at that AI in action might suppose it 'wants' to maximise the utility function. Whether they'd be right to suppose that is a question for the philosophers, but the point is, the thing doesn't just sit there apathetically just because it has no 'motivation construct'. You could say that the AI follows its programming, but it would be more accurate to say the AI is its programming.