4 ms·

What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Automobile manufacturers and their tier 1 su

What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Automobile manufacturers and their tier 1 suppliers are the world leaders in automobile safety, after all.

jsgo 8y ago
haven't read a ton into it, but whatever Google is doing with the Pentagon probably runs counter to what OpenAI claims to be after.
- sanxiyn 8y ago
  Not necessarily. Project Maven probably has no intention of developing AGI and therefore probably is not a concern to value alignment problem.
s1dechnl 8y ago
This is what always amazed me about the AI safety groups... and mission statements for funding therein : There is nothing to suggest the minds and efforts capable of producing AGI will not have solved this at a fundamental level. Absolutely nothing... and after-all something of high intelligence is within controlled and reasoned states. If someone declared they created AGI and it was behind a door and when that door was opened you saw a robot manically trashing the place, you'd clearly laugh.
Analemma_ 8y ago
> What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Competitive pressure, and the "if we don't, someone else will" effect (or Moloch, if you like). AGI- particularly recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from getting it (if it doesn't, y'know, kill us). This strongly encourages treating all concerns other than "get there first" as secondary. > Automobile manufacturers and their tier 1 suppliers are the world leaders in automobile safety, after all. Not by choice they aren't. They are forced to be the way they are by government regulations, which they always bitterly opposed at the time of creation. In fact, capitalism has such a reliable record of "not giving a shit about safety until they are forced to" that I'm perplexed you think AGI would be any different.
- dsacco 8y ago
  > recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from getting it How? This doesn't seem axiomatic.
  - nunya213 8y ago
    Seems clear that you could instruct the AGI to do anything it could to interfere with other organizations efforts to build an AGI. Obviously Nation States would have strong reasons for pursuing such a course of action and unscrupulous corporations would likewise have strong capitalistic motivations to do so.
    - p1esk 8y ago
      Seems very unclear if you could "instruct" true AGI to do something.
      - nunya213 8y ago
        Maybe you assume that an AGI would be totally uncontrollable but in this highly speculative exercise I don't think you should assume your position is the only valid one.
        p1esk 8y ago
        I don't think you should assume your position is the only valid one. I don't, that's why I said "very unclear" (to me). If something is "clear" to you about AGI, then it seems like it's you who makes that assumption.
        nunya213 8y ago
        Seems very unclear if you could "instruct" true AGI to do something. So you are claiming that by making this statement you are not assuming that it is impossible to control an AGI?
letlambda 8y ago
They're also the world leaders in automobile safety-defects.
rojobuffalo 8y ago
Attempting safety and achieving safety are not the same thing. The default way new things get invented involves iterative attempts that gradually reduce failure cases. The fear is that AGI failure modes might be catastrophic and non-recoverable.

jsgo 8y ago

haven't read a ton into it, but whatever Google is doing with the Pentagon probably runs counter to what OpenAI claims to be after.

sanxiyn 8y ago
Not necessarily. Project Maven probably has no intention of developing AGI and therefore probably is not a concern to value alignment problem.

s1dechnl 8y ago

This is what always amazed me about the AI safety groups... and mission statements for funding therein : There is nothing to suggest the minds and efforts capable of producing AGI will not have solved this at a fundamental level. Absolutely nothing... and after-all something of high intelligence is within controlled and reasoned states. If someone declared they created AGI and it was behind a door and when that door was opened you saw a robot manically trashing the place, you'd clearly laugh.

Analemma_ 8y ago

> What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Competitive pressure, and the "if we don't, someone else will" effect (or Moloch, if you like). AGI- particularly recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from getting it (if it doesn't, y'know, kill us). This strongly encourages treating all concerns other than "get there first" as secondary. > Automobile manufacturers and their tier 1 suppliers are the world leaders in automobile safety, after all. Not by choice they aren't. They are forced to be the way they are by government regulations, which they always bitterly opposed at the time of creation. In fact, capitalism has such a reliable record of "not giving a shit about safety until they are forced to" that I'm perplexed you think AGI would be any different.

dsacco 8y ago
> recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from getting it How? This doesn't seem axiomatic.
- nunya213 8y ago
  Seems clear that you could instruct the AGI to do anything it could to interfere with other organizations efforts to build an AGI. Obviously Nation States would have strong reasons for pursuing such a course of action and unscrupulous corporations would likewise have strong capitalistic motivations to do so.
  - p1esk 8y ago
    Seems very unclear if you could "instruct" true AGI to do something.
    - nunya213 8y ago
      Maybe you assume that an AGI would be totally uncontrollable but in this highly speculative exercise I don't think you should assume your position is the only valid one.
      - p1esk 8y ago
        I don't think you should assume your position is the only valid one. I don't, that's why I said "very unclear" (to me). If something is "clear" to you about AGI, then it seems like it's you who makes that assumption.
        nunya213 8y ago
        Seems very unclear if you could "instruct" true AGI to do something. So you are claiming that by making this statement you are not assuming that it is impossible to control an AGI?

letlambda 8y ago

They're also the world leaders in automobile safety-defects.

rojobuffalo 8y ago

Attempting safety and achieving safety are not the same thing. The default way new things get invented involves iterative attempts that gradually reduce failure cases. The fear is that AGI failure modes might be catastrophic and non-recoverable.