9 ms·

> The problem is we have as yet struggled to clearly articulate consistently what our actual best interests are, in terms of goals we can train into our AIs. A

> The problem is we have as yet struggled to clearly articulate consistently what our actual best interests are, in terms of goals we can train into our AIs. And imho, we never will be able to do that, because as soon as there is more than one human, they are likely to disagree about something. Even things that should be no-brainers like "should we preserve our habitat or burn it down for profit", or questions like "is it a good idea to have loads of deadly assault weapons just float around in our society", seem to be too hard for our species to resolve. If we cannot even align with our fellow humans, how can we expect to do so with machines?

spacebanana7 3y ago
> Even things that should be no-brainers like "should we preserve our habitat or burn it down for profit", or questions like "is it a good idea to have loads of deadly assault weapons just float around in our society", seem to be too hard for our species to resolve. These are brain intensive questions because they require deep moral, political, economic and ecological context. It's very difficult to express such deep context to an AI agent with prompting, fine tuning or training. I suspect this deep context also makes it difficult to align humans on these issues with education, media or hacker news comments.
- usrbinbash 3y ago
  > These are brain intensive questions They really are not. Even rodents with brains the size of a peanut manage to not willfully destroy the environment they are living in, despite already having all the resources they need, out of sheer greed.
cyanydeez 3y ago
Agreement is predictability. Predictability is vulnerability. Vulnerability is susceptibility. Evolution in identity either creates super organisms like ants or infinitely recursive "self" that necessarily requires unpredictability to ensure theatre not susceptible to coercion towards vulnerability. So, if you want self identity, you can't be agreeable and if you want survival, you can't be vulnerable.

spacebanana7 3y ago

> Even things that should be no-brainers like "should we preserve our habitat or burn it down for profit", or questions like "is it a good idea to have loads of deadly assault weapons just float around in our society", seem to be too hard for our species to resolve. These are brain intensive questions because they require deep moral, political, economic and ecological context. It's very difficult to express such deep context to an AI agent with prompting, fine tuning or training. I suspect this deep context also makes it difficult to align humans on these issues with education, media or hacker news comments.

usrbinbash 3y ago
> These are brain intensive questions They really are not. Even rodents with brains the size of a peanut manage to not willfully destroy the environment they are living in, despite already having all the resources they need, out of sheer greed.

cyanydeez 3y ago

Agreement is predictability. Predictability is vulnerability. Vulnerability is susceptibility. Evolution in identity either creates super organisms like ants or infinitely recursive "self" that necessarily requires unpredictability to ensure theatre not susceptible to coercion towards vulnerability. So, if you want self identity, you can't be agreeable and if you want survival, you can't be vulnerable.