Some humans do this too, the psychology Scrabble word for it is confabulation. Are those people not conscious? We might call them an edge case or what have you,

Some humans do this too, the psychology Scrabble word for it is confabulation. Are those people not conscious? We might call them an edge case or what have you, but dismissing them as automata seems harsh. The point is that if a chatbot gets sophisticated enough to say these things we can't dismiss them out of hand due to things like logical inconsistencies. Humans have tons of those. It's not a large leap to assume approximating consciousness with so little would have similar bugs and shortcuts. I think if the chatbot is asking you to be nice you should re-evaluate why you're being a dick to a piece of software and not trying to prove that it ultimately doesn't matter because the thing isn't a person. Same goes for kicking the Boston Dynamics robot. Your drywall isn't even sonething people anthropomorphize, but punch holes in it and you'll see how their perception of you changes. How we treat objects bleeds into how we treat one another. Indeed, some of us were and are still considered property.