5 ms·
> We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, saf
by iooi 9y ago
> We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project.
Wouldn't it be much more likely that a non-value-aligned project comes close first? Wouldn't the Google/Apple/Microsofts of the world have insanely more resources to dedicate to this, and thus get there first?
- throwawayjava 9y agoWhat, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Automobile manufacturers and their tier 1 suppliers are the world leaders in automobile safety, after all.
- jsgo 9y agohaven't read a ton into it, but whatever Google is doing with the Pentagon probably runs counter to what OpenAI claims to be after.
- sanxiyn 9y agoNot necessarily. Project Maven probably has no intention of developing AGI and therefore probably is not a concern to value alignment problem.
- s1dechnl 9y agoThis is what always amazed me about the AI safety groups... and mission statements for funding therein : There is nothing to suggest the minds and efforts capable of producing AGI will not have solved this at a fundamental level. Absolutely nothing... and after-all something of high intelligence is within controlled and reasoned states. If someone declared they created AGI and it was behind a door and when that door was opened you saw a robot manically trashing the place, you'd clearly laugh.
- Analemma_ 9y ago> What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Competitive pressure, and the "if we don't, someone else will" effect (or Moloch, if you like). AGI- particularly recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from getting it (if it doesn't, y'know, kill us). This strongly encourages treating all concerns other than "get there first" as secondary. > Automobile manufacturers and their tier 1 suppliers are the world leaders in automobile safety, after all. Not by choice they aren't. They are forced to be the way they are by government regulations, which they always bitterly opposed at the time of creation. In fact, capitalism has such a reliable record of "not giving a shit about safety until they are forced to" that I'm perplexed you think AGI would be any different.
- dsacco 9y ago> recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from getting it How? This doesn't seem axiomatic.
- nunya213 9y agoSeems clear that you could instruct the AGI to do anything it could to interfere with other organizations efforts to build an AGI. Obviously Nation States would have strong reasons for pursuing such a course of action and unscrupulous corporations would likewise have strong capitalistic motivations to do so.
- letlambda 9y agoThey're also the world leaders in automobile safety-defects.
- rojobuffalo 9y agoAttempting safety and achieving safety are not the same thing. The default way new things get invented involves iterative attempts that gradually reduce failure cases. The fear is that AGI failure modes might be catastrophic and non-recoverable.
- dsacco 9y agoYes, it's pretty hard to take that line seriously. At minimum it's very charitable framing, considering there is absolutely no indication whatsoever that OpenAI would credibly reach this (vague, underspecified) goal before any of the other serious contenders. Nor would competitors have any requirement to include OpenAI if and when they were getting close.
- s1dechnl 9y agoAgreed but it is such lines that bring in the $$$ in the valley. At some point, humanity has to cut the fluff and games and get to the fundamentals so we can all rise much higher heights of our collective intelligence. It's very hard to do so when you're constantly bombarded with crafty agenda based language on a day-to-day basis. > there is absolutely no indication whatsoever that OpenAI would credibly reach this (vague, underspecified) goal before any of the other serious contenders. Nor would competitors have any requirement to include OpenAI if and when they were getting close. This is the clear reality. So why do certain human beings pretend otherwise? Why does this pretentious game garner the most funding?Why do human beings spend so much time manipulating and hyping things into over-stated valuations when it ultimately results in wasted resources, time, and potential for the collective?
- dsacco 9y agoNo offense intended, but I don't think you and I are actually agreeing on anything. I can't really take your comments in this thread seriously.
- s1dechnl 9y agoThat's fine. I don't hide who I am. I've discussed things on various accounts/mediums over the years and more increasingly have no concern about restricting my identity. Maybe you can make accurate assumptions as to why this is the case. In any event, speaking freely and openly in this manner helps put my at ease knowing I at least got the information in its raw form out in the open. Whether or not you believe my framing, arguments, commentary is up to you. Whether or not anyone inquires is up to them. The information was put out there and that clears my conscience in a manner about the times ahead.
- samirm 9y agoSure, but that has nothing to do with the fact that they're saying they won't compete, but assist if that wasn't the case.