3 ms·

What does it even mean to align an intelligence? does it mean we want it to behave in a way that doesn't break moral/ethical rules, that aligns with our society

What does it even mean to align an intelligence? does it mean we want it to behave in a way that doesn't break moral/ethical rules, that aligns with our society rules ? Meaning do no crime, do no harm, etc... Well, maybe we should acknowledge that we've never even been able to do that with humans. There's crime, there's war, etc... We can see crime in our societies as a human alignment problem. If humans were "properly aligned", there wouldn't be any crime or misbehavior. So yeah i'm rather skeptical about aligning a superhuman intelligence that would dwarf us by its capabilities.

kromem 3y ago
You might be surprised at how prevalent TBIs are among violent offenders. One of my favorite books on true crime was a forensic psychologist who partnered with a neurologist in evaluations. Disruptions to impulse control or environmental factors that cause developmental issues with things like failing the marshmallow test can dramatically disadvantage people from being able to successfully stay non-offenders and instead succeed in modern society. So successful AGI alignment that might reduce harmful actions by high double digit percentages might be as simple as adding a secondary "impulse control" layer to the stack that reevaluates proposed actions and predicts the consequences of such actions, weighing projected net benefits and costs. A lot of people that do bad things aren't doing those things from a process driven by rational choices, and if we could successfully deploy AGI that is primarily driven by rational intelligent choices it would likely be better than humans in reduced crime propensity as well as the other things earning it the name of AGI.
- creer 3y ago
  TBI = traumatic brain injuries? And that hasn't worked all that well in the past: Even with strong impulse control, a highly considered state or government agency "for the general good" has often been serious bad news. But also the current alignment definition kinda posits that no "oops" is allowed. That is, escape or take over is not recoverable (from a sufficiently advanced AGI). So, yes, progress and one step at a time - but the field in its current definition is looking for a magic bullet.
- guybedo 3y ago
  > You might be surprised at how prevalent TBIs are among violent offenders. Didn't know that, interesting to know. > So successful AGI alignment (...) might be as simple as adding a secondary "impulse control" layer to the stack that reevaluates proposed actions and predicts the consequences of such actions, weighing projected net benefits and costs. Problem is how the super AGI gonna weight benefits and costs. What's the cost of stealing or killing to a super AGI ? At some point can't that super AGI consider that all those rules we're trying to enforce, are rules set by an inferior entity and so could be bypassed ?

kromem 3y ago

You might be surprised at how prevalent TBIs are among violent offenders. One of my favorite books on true crime was a forensic psychologist who partnered with a neurologist in evaluations. Disruptions to impulse control or environmental factors that cause developmental issues with things like failing the marshmallow test can dramatically disadvantage people from being able to successfully stay non-offenders and instead succeed in modern society. So successful AGI alignment that might reduce harmful actions by high double digit percentages might be as simple as adding a secondary "impulse control" layer to the stack that reevaluates proposed actions and predicts the consequences of such actions, weighing projected net benefits and costs. A lot of people that do bad things aren't doing those things from a process driven by rational choices, and if we could successfully deploy AGI that is primarily driven by rational intelligent choices it would likely be better than humans in reduced crime propensity as well as the other things earning it the name of AGI.

creer 3y ago
TBI = traumatic brain injuries? And that hasn't worked all that well in the past: Even with strong impulse control, a highly considered state or government agency "for the general good" has often been serious bad news. But also the current alignment definition kinda posits that no "oops" is allowed. That is, escape or take over is not recoverable (from a sufficiently advanced AGI). So, yes, progress and one step at a time - but the field in its current definition is looking for a magic bullet.
guybedo 3y ago
> You might be surprised at how prevalent TBIs are among violent offenders. Didn't know that, interesting to know. > So successful AGI alignment (...) might be as simple as adding a secondary "impulse control" layer to the stack that reevaluates proposed actions and predicts the consequences of such actions, weighing projected net benefits and costs. Problem is how the super AGI gonna weight benefits and costs. What's the cost of stealing or killing to a super AGI ? At some point can't that super AGI consider that all those rules we're trying to enforce, are rules set by an inferior entity and so could be bypassed ?