3 ms·

You are ignoring Sam's objectively excellent performance at YCombinator. I have one big problem with Altman which sullies my otherwise incredibly high opinion

You are ignoring Sam's objectively excellent performance at YCombinator. I have one big problem with Altman which sullies my otherwise incredibly high opinion of him: he explicitly denies the orthogonality thesis and has created a culture at OpenAI that, to analogize scaling modals with nuclear energy, feels like one that knows something cool will happen when you collide large enough enriched uranium lumps together at high speed, but does not believe nuclear explosions are possible. How he can deny the is/ought distinction when his own organization published Faulty Reward Functions in the Wild is beyond me. And now many of those most concerned with safety, such as Paul Christiano, appear to be leaving OpenAI. To accuse him of being incapable of building and scaling is incredibly obtuse. How much did YCombinator grow under his stewardship? I am more worried that he is too good at that.

deleted 6y ago
[deleted]
alecst 6y ago
I'll be honest, I have no idea what you're saying. Seems like you have some interesting points to make though.
- marvin 6y ago
  The orthogonality thesis says that advanced artificial intelligence (yet to be built) can be both vastly powerful and have arbitrary goals -- in particular, goals that are catastrophic for humanity. (More specifically: capability and morality for AIs are not correlated). GP says that Altman explicitly denies this theory and fosters a culture that might end up creating dangerous AI. Implied here is that there is a risk this might cause a truly catastrophic accident. The parts about "safety" refers to acknowledging this danger and attempting to mitigate it. I don't know enough about Altman or OpenAI to make any judgements on their approach, but that's a brief explanation.
rytill 6y ago
I am a safety advocate, and your analogy to nuclear explosions is pretty interesting. However, simplified analogies may be doing more harm than good because they fuel the straw-men people create of arguments for AI safety. I am not sure what the proper technique to get someone already entrenched in their perspective to broaden their possibility distribution is, though. Do you have any tips on that, as someone that appears to be engaged in the space and appears to advocate similarly?

deleted 6y ago

[deleted]

alecst 6y ago

I'll be honest, I have no idea what you're saying. Seems like you have some interesting points to make though.

marvin 6y ago
The orthogonality thesis says that advanced artificial intelligence (yet to be built) can be both vastly powerful and have arbitrary goals -- in particular, goals that are catastrophic for humanity. (More specifically: capability and morality for AIs are not correlated). GP says that Altman explicitly denies this theory and fosters a culture that might end up creating dangerous AI. Implied here is that there is a risk this might cause a truly catastrophic accident. The parts about "safety" refers to acknowledging this danger and attempting to mitigate it. I don't know enough about Altman or OpenAI to make any judgements on their approach, but that's a brief explanation.

rytill 6y ago

I am a safety advocate, and your analogy to nuclear explosions is pretty interesting. However, simplified analogies may be doing more harm than good because they fuel the straw-men people create of arguments for AI safety. I am not sure what the proper technique to get someone already entrenched in their perspective to broaden their possibility distribution is, though. Do you have any tips on that, as someone that appears to be engaged in the space and appears to advocate similarly?