4 ms·

> they're actually doing is researching the dangers of AI and other neckbeard concerns that have nothing to do with the problems and sufferings of actual human

> they're actually doing is researching the dangers of AI and other neckbeard concerns that have nothing to do with the problems and sufferings of actual human beings today. This is really the kind of comment I'd like not to see on HN. "Neckbeard concern" implies that... what, people worrying about AI risk are worrying for fun? Because they're socially awkward nerds disconnected from reality? This reminds me of https://xkcd.com/743/ https://xkcd.com/743/ Worrying about the future consequences of incoming technological developments always looks like "neckbeard concerns" until it's too late to do anything about it. And thus he we are, worrying about climate change and supply chain sovereignty and corporate control of information and wondering why nobody took sensible measures 20 years ago. > And there's no mention of anything this org has actually done to help anyone, other than getting people to donate Note that this is an internal post from an individual suggesting organizational changes, not an official public-facing communication. As sibling comments pointed out, they do have posts listing success stories.

zerocrates 4y ago
My concern about effectiveness about the "AI safety" stuff is that it kind of sidesteps the whole effectiveness analysis: if you're nominally working on risks you claim to be existential or apocalyptic, then the work is almost automatically "effective" even if you don't do anything really, just because the risk is so big that an infinitesimal decrease amounts to massive impact. I'm sure just because of the nature of the movement there's been lots of thinking about the likelihood of various future scenarios and accounting for this in less flippant terms than I've just described it, but it seems to me that this is a group of people who are skewed to think AI is more likely, more potentially dangerous, and more interesting to work on and fund. Putting aside the likelihood of AGI materializing at all, it's not really clear to me that paying for a bunch of research on this necessarily affects the actual AI that would be produced anyway, as that's being done by different people with different interests and incentives. You see people discuss the numbers of people affected by something like malaria prevention versus eventual extinction at the hands of AI as a justification for the need for AI work, but less talk about how we actually know what kinds of things we can do to do malaria prevention and really not so much the AI stuff. Obviously you have to start somewhere but... From an outsider's perspective, the whole area just kind of feels like a pit you can throw money into while pretending you're being rigorous about spending.
- Max_Limelihood 4y ago
  This is a perfectly reasonable concern/criticism and one that plenty of EAs make. Going off funding, which is overwhelmingly directed towards global health and development, it’s probably one that most of them agree with. It also aligns with my experience talking to EAs, who mostly don’t work in or donate to AI. But this is a criticism about effectiveness made within an EA framework. It assumes the thing we want to do is maximize the amount of good we do with our resources, and provides rational arguments for why AI won’t do that. The AI folks think their cause is the one that does the most good, and they have rational arguments for that position. That’s why they’re considered part of the EA movement (despite not fitting in with the original vision). That also means we have to listen and provide counterarguments before we reject their position. What we definitely shouldn’t do is write them off as “neckbeards” just because they’re working in tech and have unusual concerns. That’s how you end up writing off some 1930s physicist worried about the existential risk of nuclear fission weapons as some “Weird neckbeard nerd.”
- bpodgursky 4y ago
  The concern that EAs have about AI alignment is that there is practically no satisfying theoretical framework right now for how to build "safe" AI, and that's kinda concerning as we approach human-emulating AI (at least in the realm of chatbots, art, writing...) There are hotfixes people add on for edge cases like "don't label people as gorillas" or "don't say certain words that Nazis say", but for problems like "don't ever initiate a conversation likely to cause someone to commit suicide"... there's no foundational work (that's not super vague) that compels an AI to optimize for human success and life. That's not great IMO, and we should probably dedicate as much money to it as we dedicate to, I dunno, potato chips ($65 billion a year in the US IIRC).

zerocrates 4y ago

My concern about effectiveness about the "AI safety" stuff is that it kind of sidesteps the whole effectiveness analysis: if you're nominally working on risks you claim to be existential or apocalyptic, then the work is almost automatically "effective" even if you don't do anything really, just because the risk is so big that an infinitesimal decrease amounts to massive impact. I'm sure just because of the nature of the movement there's been lots of thinking about the likelihood of various future scenarios and accounting for this in less flippant terms than I've just described it, but it seems to me that this is a group of people who are skewed to think AI is more likely, more potentially dangerous, and more interesting to work on and fund. Putting aside the likelihood of AGI materializing at all, it's not really clear to me that paying for a bunch of research on this necessarily affects the actual AI that would be produced anyway, as that's being done by different people with different interests and incentives. You see people discuss the numbers of people affected by something like malaria prevention versus eventual extinction at the hands of AI as a justification for the need for AI work, but less talk about how we actually know what kinds of things we can do to do malaria prevention and really not so much the AI stuff. Obviously you have to start somewhere but... From an outsider's perspective, the whole area just kind of feels like a pit you can throw money into while pretending you're being rigorous about spending.

Max_Limelihood 4y ago
This is a perfectly reasonable concern/criticism and one that plenty of EAs make. Going off funding, which is overwhelmingly directed towards global health and development, it’s probably one that most of them agree with. It also aligns with my experience talking to EAs, who mostly don’t work in or donate to AI. But this is a criticism about effectiveness made within an EA framework. It assumes the thing we want to do is maximize the amount of good we do with our resources, and provides rational arguments for why AI won’t do that. The AI folks think their cause is the one that does the most good, and they have rational arguments for that position. That’s why they’re considered part of the EA movement (despite not fitting in with the original vision). That also means we have to listen and provide counterarguments before we reject their position. What we definitely shouldn’t do is write them off as “neckbeards” just because they’re working in tech and have unusual concerns. That’s how you end up writing off some 1930s physicist worried about the existential risk of nuclear fission weapons as some “Weird neckbeard nerd.”
bpodgursky 4y ago
The concern that EAs have about AI alignment is that there is practically no satisfying theoretical framework right now for how to build "safe" AI, and that's kinda concerning as we approach human-emulating AI (at least in the realm of chatbots, art, writing...) There are hotfixes people add on for edge cases like "don't label people as gorillas" or "don't say certain words that Nazis say", but for problems like "don't ever initiate a conversation likely to cause someone to commit suicide"... there's no foundational work (that's not super vague) that compels an AI to optimize for human success and life. That's not great IMO, and we should probably dedicate as much money to it as we dedicate to, I dunno, potato chips ($65 billion a year in the US IIRC).