That's true, but there is no known way to verify proper operation of those kinds of machines. What happens if your artificial companion stops liking you? What i

That's true, but there is no known way to verify proper operation of those kinds of machines. What happens if your artificial companion stops liking you? What if your negotiation robot throws a tantrum? Well, then the robot isn't doing what we want it to do. But you asked for human behavior, and that's what you got. There is not really any point in making robots do what humans do. We want them to do what humans can't do -- even if we want them to seem human while they do it. Addendum: >(unless we end up with a theory and evidence indicating that machines can be sentient). I think we'd sooner find a theory indicating that sentience is either not well-defined or not a useful concept for making decisions. If we found that ants were sentient, we'd still step on them and not care. This kind of talk is exactly the kind of reason I feel most people are useless for understanding ethics: you can't make decisions based on the answers of arbitrary grammatical yes/no questions. It's the same reason mathematicians don't really care if P=NP is true or not. You can just suppose one way or the other and you'll be no better off without a proof. The value in the proof is an advancement in conceptualization and language. Similarly, an important advancement in ethics/morality/AI will very likely make your concerns about sentience obsolete in a way that should be obvious to you.