3 ms·

> The AI we create will have whatever motivational structure we choose for it. And surely we wouldn’t be so foolish as to create it in our image! I think this

by proc0 4y ago

> The AI we create will have whatever motivational structure we choose for it. And surely we wouldn’t be so foolish as to create it in our image! I think this is a common misconception. No one is really choosing the specific way the features are manifested in the final AI product. What is decided are the initial conditions, and then the decision to not use it, or limit its use, but the training is done in such a way the programmers have very little control over the resulting output of the training. This is why we've already seen AI's getting taken down for some unexpected behavior. It almost seems like the Halting Problem, but instead of not being able to predict a computer halting, it's not being able to predict if an AI will behave as expected. Maybe given enough scale and complexity, the unpredictability becomes inevitable. Our best bet would probably be something like a killswitch.

stcredzero 4y ago
Morality and politics are ultimately just mechanisms to allow us to live with each other efficiently. Has anyone formulated any kind of argument that shows that AIs created by training on data will have any politics and morality whatsoever? I think the instrumental goals which are the easiest to discover are the sociopathic ones. The behavior of children seems to indicate this.
- proc0 4y ago
  Right, we don't know enough about the brain and/or conscious experience to determine where and how morality arises. In a way this is part of the risk. We might stumble on emergent morality with a large enough language model and not know until it starts happening with real consequences.
- zmmmmm 4y ago
  > Has anyone formulated any kind of argument that shows that AIs created by training on data will have any politics I fully expect that it won't be long before the same pattern we see in other areas of tech plays out. That is, the right starts alleging left wing bias in all of these networks and even starts campaigning for laws to constrain them to represent right wing points of view. Of course, this is betraying my own bias which is roughly in line with Colbert's "It is a well known fact that reality has liberal bias". That is, right wing views are intrinsically rooted in principle and preconceived beliefs rather than observable facts and so there is always going to be a problem to get LLMs which are heavily constrained to train on what is observable to give even representation to things that aren't supported by observable facts.
  - stcredzero 4y ago
    I fully expect that it won't be long before the same pattern we see in other areas of tech plays out. That is, the right starts alleging left wing bias... Jeez. Left/right has nothing to do with this. Go and watch "The Fly" and ponder Brundle's comment about wanting to be the first insect politician.
- dragonwriter 4y ago
  > Has anyone formulated any kind of argument that shows that AIs created by training on data will have any politics and morality whatsoever? If no careful alignment work is put in, AIs will have political biases as an incidental result of the content of the training data. If careful alignment work is put in, AIs will have political biases selected as part of the alignment work. Both forms are readily observable in AI far short of AGI.
worrycue 4y ago
> No one is really choosing the specific way the features are manifested in the final AI product. For current algorithms, yes. If an algorithm has problematic tendencies we can just choose not to use it - or use it in a very controlled and limited fashion. We cannot rule out that future new AI algorithms might allow us to explicitly set the AI’s “primary directive”.
hinkley 4y ago
America's founding fathers worried about populism, and that's what we're at risk of getting in AI. Sure the creators may be smart enough to mark pages containing hate speech and filter it out, but the number of subtler policy issues are basically unbounded. You have to choose to filter all of that stuff out, unless you build some sort of AI that 'solves' human speech and verifies the assertions.

by proc0 4y ago

stcredzero 4y ago

Morality and politics are ultimately just mechanisms to allow us to live with each other efficiently. Has anyone formulated any kind of argument that shows that AIs created by training on data will have any politics and morality whatsoever? I think the instrumental goals which are the easiest to discover are the sociopathic ones. The behavior of children seems to indicate this.

proc0 4y ago
Right, we don't know enough about the brain and/or conscious experience to determine where and how morality arises. In a way this is part of the risk. We might stumble on emergent morality with a large enough language model and not know until it starts happening with real consequences.
zmmmmm 4y ago
> Has anyone formulated any kind of argument that shows that AIs created by training on data will have any politics I fully expect that it won't be long before the same pattern we see in other areas of tech plays out. That is, the right starts alleging left wing bias in all of these networks and even starts campaigning for laws to constrain them to represent right wing points of view. Of course, this is betraying my own bias which is roughly in line with Colbert's "It is a well known fact that reality has liberal bias". That is, right wing views are intrinsically rooted in principle and preconceived beliefs rather than observable facts and so there is always going to be a problem to get LLMs which are heavily constrained to train on what is observable to give even representation to things that aren't supported by observable facts.
- stcredzero 4y ago
  I fully expect that it won't be long before the same pattern we see in other areas of tech plays out. That is, the right starts alleging left wing bias... Jeez. Left/right has nothing to do with this. Go and watch "The Fly" and ponder Brundle's comment about wanting to be the first insect politician.
dragonwriter 4y ago
> Has anyone formulated any kind of argument that shows that AIs created by training on data will have any politics and morality whatsoever? If no careful alignment work is put in, AIs will have political biases as an incidental result of the content of the training data. If careful alignment work is put in, AIs will have political biases selected as part of the alignment work. Both forms are readily observable in AI far short of AGI.

worrycue 4y ago

> No one is really choosing the specific way the features are manifested in the final AI product. For current algorithms, yes. If an algorithm has problematic tendencies we can just choose not to use it - or use it in a very controlled and limited fashion. We cannot rule out that future new AI algorithms might allow us to explicitly set the AI’s “primary directive”.

hinkley 4y ago

America's founding fathers worried about populism, and that's what we're at risk of getting in AI. Sure the creators may be smart enough to mark pages containing hate speech and filter it out, but the number of subtler policy issues are basically unbounded. You have to choose to filter all of that stuff out, unless you build some sort of AI that 'solves' human speech and verifies the assertions.