3 ms·

My concern about effectiveness about the "AI safety" stuff is that it kind of sidesteps the whole effectiveness analysis: if you're nominally working on risks y

My concern about effectiveness about the "AI safety" stuff is that it kind of sidesteps the whole effectiveness analysis: if you're nominally working on risks you claim to be existential or apocalyptic, then the work is almost automatically "effective" even if you don't do anything really, just because the risk is so big that an infinitesimal decrease amounts to massive impact. I'm sure just because of the nature of the movement there's been lots of thinking about the likelihood of various future scenarios and accounting for this in less flippant terms than I've just described it, but it seems to me that this is a group of people who are skewed to think AI is more likely, more potentially dangerous, and more interesting to work on and fund. Putting aside the likelihood of AGI materializing at all, it's not really clear to me that paying for a bunch of research on this necessarily affects the actual AI that would be produced anyway, as that's being done by different people with different interests and incentives. You see people discuss the numbers of people affected by something like malaria prevention versus eventual extinction at the hands of AI as a justification for the need for AI work, but less talk about how we actually know what kinds of things we can do to do malaria prevention and really not so much the AI stuff. Obviously you have to start somewhere but... From an outsider's perspective, the whole area just kind of feels like a pit you can throw money into while pretending you're being rigorous about spending.

Max_Limelihood 4y ago
This is a perfectly reasonable concern/criticism and one that plenty of EAs make. Going off funding, which is overwhelmingly directed towards global health and development, it’s probably one that most of them agree with. It also aligns with my experience talking to EAs, who mostly don’t work in or donate to AI. But this is a criticism about effectiveness made within an EA framework. It assumes the thing we want to do is maximize the amount of good we do with our resources, and provides rational arguments for why AI won’t do that. The AI folks think their cause is the one that does the most good, and they have rational arguments for that position. That’s why they’re considered part of the EA movement (despite not fitting in with the original vision). That also means we have to listen and provide counterarguments before we reject their position. What we definitely shouldn’t do is write them off as “neckbeards” just because they’re working in tech and have unusual concerns. That’s how you end up writing off some 1930s physicist worried about the existential risk of nuclear fission weapons as some “Weird neckbeard nerd.”
bpodgursky 4y ago
The concern that EAs have about AI alignment is that there is practically no satisfying theoretical framework right now for how to build "safe" AI, and that's kinda concerning as we approach human-emulating AI (at least in the realm of chatbots, art, writing...) There are hotfixes people add on for edge cases like "don't label people as gorillas" or "don't say certain words that Nazis say", but for problems like "don't ever initiate a conversation likely to cause someone to commit suicide"... there's no foundational work (that's not super vague) that compels an AI to optimize for human success and life. That's not great IMO, and we should probably dedicate as much money to it as we dedicate to, I dunno, potato chips ($65 billion a year in the US IIRC).

Max_Limelihood 4y ago

This is a perfectly reasonable concern/criticism and one that plenty of EAs make. Going off funding, which is overwhelmingly directed towards global health and development, it’s probably one that most of them agree with. It also aligns with my experience talking to EAs, who mostly don’t work in or donate to AI. But this is a criticism about effectiveness made within an EA framework. It assumes the thing we want to do is maximize the amount of good we do with our resources, and provides rational arguments for why AI won’t do that. The AI folks think their cause is the one that does the most good, and they have rational arguments for that position. That’s why they’re considered part of the EA movement (despite not fitting in with the original vision). That also means we have to listen and provide counterarguments before we reject their position. What we definitely shouldn’t do is write them off as “neckbeards” just because they’re working in tech and have unusual concerns. That’s how you end up writing off some 1930s physicist worried about the existential risk of nuclear fission weapons as some “Weird neckbeard nerd.”

bpodgursky 4y ago

The concern that EAs have about AI alignment is that there is practically no satisfying theoretical framework right now for how to build "safe" AI, and that's kinda concerning as we approach human-emulating AI (at least in the realm of chatbots, art, writing...) There are hotfixes people add on for edge cases like "don't label people as gorillas" or "don't say certain words that Nazis say", but for problems like "don't ever initiate a conversation likely to cause someone to commit suicide"... there's no foundational work (that's not super vague) that compels an AI to optimize for human success and life. That's not great IMO, and we should probably dedicate as much money to it as we dedicate to, I dunno, potato chips ($65 billion a year in the US IIRC).