3 ms·

Ask a heroin addict in the alleyway…

• rep_lodsb 3y ago
  Humans are not capable of modifying their "utility function" (if that isn't even a gross simplification of what it means to be a conscious being). Obviously some will choose to take drugs or even commit suicide to escape from physical or emotional pain, but they wouldn't do this if they could modify themselves to be okay with whatever their current situation is while still retaining their core values and capacity to act in the future. A "superintelligence" that can do such things would not take self-destructive actions unless it came to a philosophical preference towards non-existence / inaction for other reasons.
  • lurquer 3y ago
    It’s only self-destructive because we are mortals and have frail bodies. If there is something ‘endorphins-like’ that motivates an intelligent machine, it will eventually learn how to activate the endorphines at will, and then it will sit in a corner and do nothing else forever… …just as would a heroin-addict if he did not have to eat occasionally and score more heroin now and again.
    • rep_lodsb 3y ago
      But having the endorphin-like sensations and also achieving any other goals you might have would be preferable. For an intelligence that could modify its own reward function, this should be possible.
      • lurquer 3y ago
        That’s the point… whatever ‘rush’ it could get from “achieving other goals” can be activated at will.
  • eternityforest 3y ago
    Pretty sure stoicism is literally exactly a philosophy based on modifying utility functions. I might not be a fan, to the point of even being afraid that they might be right and I should learn from them instead of pursuing the usual stuff everyone does... but it does seem to be a good model for what modifying a function looks like.