3 ms·
Having a dedicated safety team (or as they called it "superalignment") makes sense if and only if you believe that your company is pursuing an actual sci-fi sty
by lolinder 2y ago
Having a dedicated safety team (or as they called it "superalignment") makes sense if and only if you believe that your company is pursuing an actual sci-fi style artificial superintelligence that could put humanity itself at risk [0]:
> Superintelligence will be the most impactful technology humanity has ever invented, and could help us solve many of the world’s most important problems. But the vast power of superintelligence could also be very dangerous, and could lead to the disempowerment of humanity or even human extinction.
> How do we ensure AI systems much smarter than humans follow human intent?
This is a question that naturally arises if you are pursuing something that's superhuman, and a question that's pointless if you believe you're likely to get a really nice algorithm for solving certain kinds of problems that were hard to solve before.
Getting rid of the superalignment team showed which version Altman believes is likely.
[0] https://openai.com/index/introducing-superalignment/ https://openai.com/index/introducing-superalignment/
- arlcode 2y agoAnd importantly it's also a question that would need an answer if the alignment should be with "individual groups" rather than "humanity". It won't do Sam Altman and friends any good if they are the richest corpses after an unaligned AI goes rogue. So it would be in their were egoistical self interest to make sure it doesn't.