3 ms·
> AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion. I don’t think that’s really true. Rather,
by dragonwriter 4y ago
> AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion.
I don’t think that’s really true. Rather, the faction that sees alignment as a subordinate concern within ethics hates the faction that sees ethics as a subordinate concern within alignment.
> The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks.
Not really; the alignment-priority branch has no problem recognizing (certain subsets of) the bias and other issues that are the more central focus of the ethics branch as real and significant, but they view alignment as solving them and view narrow centralized control as a mitigation (for both “alignment” and “ethics” types of issues) until it is solved. They also view progress on AI as a general priority, for a mix of commercial, ideological, and other [0] reasons, and their own progress on AI in particular as essential (because they view AI from others – either because potentially unaligned, or for other reasons – as a critical and potentially existential threat.)
> The ethics branch (responsible for all the media attention about racist AIs/bias etc) view alignment people as pie in the sky researchers whose results are irrelevant to society.
No, they view them as an enormous threat [EDIT: 3], because they are working in a way that not only insufficiently mitigates in basic approach because of a lack of priority, but also magnifies by its focus on closely concentrated control, the issues of paramount concern to the ethics-focused.
> It seems increasingly that the concerns from the ethics branch have not come to pass at all.
They have in fact come to pass in deployments of AI in all kinds of socially-significant roles. E.g, the facial recognition in the CBP One app newly mandated for asylum seekers doesn’t work well for dark-skinned faces (racial bias in facial image recognition is literally one of the earliest publicly recognized AI bias problems that motivated the AI ethics movement.) They’ve also manifested in systems deployed in welfare management [1], hiring [2], and all kinds of other areas.
[0] e.g., Roko’s Basilisk
[1] https://www.wired.com/story/welfare-algorithms-discrimination/ https://www.wired.com/story/welfare-algorithms-discriminatio...
[2] https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G https://www.reuters.com/article/us-amazon-com-jobs-automatio...
[3] EDIT: It’s very hard to emphasize how much this is true. Here's, though, an illustration I came across on Twitter after first posting this message (its someone from the ehics side laying out their view of the threat from certain people on the alignment side, though that may not be obvious because the interconnected set of disagreements goes way deeper than the alignment vs ethics thing, which is just a surface manifestation): https://twitter.com/xriskology/status/1635313838508883968?t=0r4p-WAANnph8FFRF_BCIw&s=19 https://twitter.com/xriskology/status/1635313838508883968?t=...
- FeepingCreature 4y agoNote that if you see someone bring up Roko, it's overwhelmingly somebody arguing against alignment researchers; its use is primarily as "look at this crazy thing these people believe."
- hgsgm 4y agoThat's because the rationalist/alignment people's belief system leads to a logical conclusion that talking about Roko's Basilisk harms the person you tell, and even rationalists have a human heart interfering with their extreme ultilitarianism.
- FeepingCreature 4y agoI mean, it's also because no rationalists take Roko's Basilisk intellectually seriously nowadays. There was a period of maybe a few months in 2010 where people were genuinely worried about it, and then they gradually figured out the logical holes in it. Any spread after that has been from outsiders who were fascinated with the idea. Nowadays it mostly persists as a trap for people with schizoaffective disorders. (Disclaimer: my view is of the forum itself; I don't know how long it persisted in SF circles.)