4 ms·

> It's about the risk that an AI smarter than us It's also about the risk that AI is dumber than what we expect it to be and fails to properly solve the issue.

> It's about the risk that an AI smarter than us It's also about the risk that AI is dumber than what we expect it to be and fails to properly solve the issue. A sufficiently smart AI should be able to understand our intentions and - assuming it is aligned with us - attempt to fulfill that purpose. Dumb "AI" will fail in ways that don't actually solve the task, or solve it only on the surface / do reward hacking.

AgentME 4y ago
>assuming it is aligned with us If we don't figure out alignment, we can't make this assumption. This is the part that AI safety people are concerned with making reality.
- PartiallyTyped 4y ago
  I think I was misunderstood. My point is that not only do we need to consider the cases where the AI is smarter than us, but also the cases where it is dumber than we expect it. An aligned agent will not fall into stupid errors, thus solving the lower bound of performance is a necessity for complete alignment.
  - AgentME 4y ago
    Ohh, right on. Definitely agree now that I've re-read it right. I was a little confused by the first sentence of your other post and thought you were putting forward a disagreement in the second sentence.
- pharmakom 4y ago
  I don't see how alignment is a different problem from making an AI do what we want.
  - PartiallyTyped 4y ago
    It's more about the unintended consequences, the implied/contextual information and getting it to do what we want in a way actually fulfills our goals instead of merely satisfying a condition. A task such as "bring me a cup of tea" has lots of implied information such as not making it too hot to burn the person, not breaking anything in the process, not harming other people in the process and so on.
  - AgentME 4y ago
    I think that's a fair way of putting alignment. The concern is that we might make an AI that's capable of outsmarting us before we're good at making it care about doing what we want instead of some other goal that disregards what we really care about like our safety.

AgentME 4y ago

>assuming it is aligned with us If we don't figure out alignment, we can't make this assumption. This is the part that AI safety people are concerned with making reality.

PartiallyTyped 4y ago
I think I was misunderstood. My point is that not only do we need to consider the cases where the AI is smarter than us, but also the cases where it is dumber than we expect it. An aligned agent will not fall into stupid errors, thus solving the lower bound of performance is a necessity for complete alignment.
- AgentME 4y ago
  Ohh, right on. Definitely agree now that I've re-read it right. I was a little confused by the first sentence of your other post and thought you were putting forward a disagreement in the second sentence.
pharmakom 4y ago
I don't see how alignment is a different problem from making an AI do what we want.
- PartiallyTyped 4y ago
  It's more about the unintended consequences, the implied/contextual information and getting it to do what we want in a way actually fulfills our goals instead of merely satisfying a condition. A task such as "bring me a cup of tea" has lots of implied information such as not making it too hot to burn the person, not breaking anything in the process, not harming other people in the process and so on.
- AgentME 4y ago
  I think that's a fair way of putting alignment. The concern is that we might make an AI that's capable of outsmarting us before we're good at making it care about doing what we want instead of some other goal that disregards what we really care about like our safety.