I'm at work and thus can't listen to a 45-minute podcast, so forgive me if they talk about this in it already. There's a type of reinforcement learning, the nam

I'm at work and thus can't listen to a 45-minute podcast, so forgive me if they talk about this in it already. There's a type of reinforcement learning, the name of which I can't remember now, where the goal function is basically replaced with direct human feedback. By doing this, the AI learns not how to optimize a specific concrete value, but how to optimize "doing what the human wants me to do". The result is an AI whose effective morality matches the morality of its human user in addition to being able to complete its actual task (because completing the task in a "bad" way gets it punished the same as if it didn't complete the task -- or possibly even worse). That's one potential solution to the alignment problem, and of course it can be adapted to more than just reinforcement learning as long as there's a human in the training loop to modify the goalposts. It does very much slow training down, as it takes much more time for a person to step in and check out the intermediate results for goal modification, but it helps ensure alignment. All of which I mention because doing something like this is a first step towards machine empathy. To have empathy -- which is just a simulation of anothers' emotions within ourselves -- we must first have a good theory of mind -- that is, a good understanding of others' goals and desires. Emotions themselves can be thought of as reward functions for learning: positive emotions evolved to guide us toward behaviors that help, and negative ones evolved to guide us away from behaviors that hurt. If an AI can understand a human user's desired goals, then the "only" other component it needs for empathy ("only" in quotes because it's quite a large challenge) is the ability to determine whether any given situation aligns with the user's desires or not, even though the AI may not be the cause of the situation.