6 ms·

I think people have yet to realize that this whole AI Safety thing is complete BS. It's just another veil, like Effective Altruism, to get good PR and build a c

I think people have yet to realize that this whole AI Safety thing is complete BS. It's just another veil, like Effective Altruism, to get good PR and build a career around. The only people who truly believe this AI safety stuff are those with no technical knowledge or expertise.

vouaobrasil 3y ago
Then it should be a fundamental mission of people who DO have technical knowledge to spread the word on both the negative and positive effects of AI. If WE have the knowledge, WE have the responsibility to go out there and let people know. But as far as I know, most technical people are happy to discuss it amongst themselves and not even say one word to people who have no technical knowledge. Not one word. We need more balanced opinions in a world where the only ideas that survive are the ones that promise short-term gains at the cost of long-term disasters.
- loveparade 3y ago
  The problem is the same as some of the woke movements. Nobody wants to publicly speak out against AI safety BS because a Twitter mob who has made a career out of it will descend onto them and ruin their career. There is nothing to gain from speaking out against it, but you gain career opportunities by being part of the AI safety BS community. In fact, this has happened with Hinton (I think it was?) when he dared to explain how models aren't inherently biased. Had to publicly apologize for it later. Even ML conferences have AI safety tracks these days. Nobody sane wants to be excluded from opportunities.
  - vouaobrasil 3y ago
    Well, I for one am speaking out against it (and against AI in general). I have a blog, Twitter, newsletter, and I even post warnings on my YouTube channel from time to time. I was part of a discussion that led my employer to commit to NOT using generative AI. No Twitter mob is going to stop me. They can descend all they want and pretend to be brave in front of a keyboard and screen. As far as I'm concerned, they're cowards.
    - KingOfCoders 3y ago
      "against AI safety" You speak against AI and against AI safety? In what way? Wouldn't be speaking against AI enough? Or would AI safety mean laws to prevent AI existence? Like in the Algebraist? (though it is funny in that book how the powerful use AI when it suits them).
      - vouaobrasil 3y ago
        I speak out against both. I am strongly against AI. But I am also against AI safety in general because I think AI safety is: (1) A false amelioration that cannot prevent the dangers of AI (2) A false promise to people that AI can be made safe (3) A weak on-the-fence position to take of people who think they can have their cake and eat it too.
        calf 3y ago
        But is it impossible to study AI safety (or "safety") scientifically? What about all the papers like AI watermarking or AI racial bias and so for? Do you argue that academics working on those questions are doing illegitimate science?
        vouaobrasil 3y ago
        I don't think the science is illegitimate in the sense that it doesn't follow the scientific method or at least the principles of science or engineering in some form. Nor do I think they have bad intentions. BUT, I do think their work is useless and only serves to better integrate AI into to the system, which is better in the short term but worse in the long term. Therefore, overall, I vehemently disagree with their research and I think it is fundamentally a contribution to the worsening of the earth. I absolutely AGREE with treating all people equally and eliminating racial biases of course. However, TECH companies have a VESTED interest in this sort of thing, which is to integrate as many people as possible into the tech empire. It just so happens that building the tech empire is roughly equivalent to success in modern society, which is why their intentions seem good, but they are only ostensibly good. In reality, they do no good of any kind.
- TotalCrackpot 3y ago
  If you are actually interested in that you can promote voices like this (Why We Must Resist AI w/ Dan McQuillan): https://news.ycombinator.com/item?id=38362498 https://news.ycombinator.com/item?id=38362498
  - vouaobrasil 3y ago
    Thanks, I'll look into it!
- bsenftner 3y ago
  The problem with AI has never been the technology that composes AI, the problem has always and will forever be the immature yet powerful humans that take advantage of less advantaged humans using whatever means they have, which today can include AI. The issue is not a technology issue, it's a human psychology and sociology issue, it's our immature humans in society that is the issue. Technologists cannot speak about that in any meaningful manner.
  - vouaobrasil 3y ago
    I absolutely disagree. Technologists understand the true nature of AI and can thus predict its uses. Moreover, if technologists realize this problem of how AI will inevitably be used, their voice will be a powerful message that even those who enjoy math and science are fearful of AI.
    - bsenftner 3y ago
      Nope, nope. Technologists do not understand the true nature of AI. What technologists know they can rarely communicate, and even less so to a general public. Technology developers are not some special deep understanders that ordinary humans are not; we just have more time spent in our specialization than they, as they have in their specializations which we do not. If anything, technology developers, due to our specialization's time requirements and the traditional lack of professional communications training, tend to be weaker communicators than the general public. The last people you want explaining AI to a potentially fearful general public is weak communicators, such as the AI developers.
      - Gud 3y ago
        Just because many technologists are poor communicators, doesn't mean all are. Carl Sagan.
  - RhysU 3y ago
    Michael Crichton has (rightly) entered the chat. https://en.m.wikipedia.org/wiki/Michael_Crichton https://en.m.wikipedia.org/wiki/Michael_Crichton Crichton says these failures of humanity are inevitable because there will always be a technologist who does things because they can, not because they should. That everything will go off the rails, no matter what people attempt, is the plot of the novel Jurassic Park.
- matheusmoreira 3y ago
  There should be no need for us to spread the word about anything. Technology needs to be built and the chips need to fall where they may. If society changes as a result, so be it. I'm seriously tired of these vested interests destroying perfectly good technology just because it would endanger their precious business models. Computers should have put an end to silly things like copyright but they would rather destroy our computing freedom than allow that to happen. Now we have these awesome language models and AIs and they're desperately trying to limit and control it. It's a tragedy in of itself that we even need to fight these political battles at all.
  - pas 3y ago
    the strong needs to just take whatever it wants, and so be it? there's no separating technology from context, as in consequences. yes, all the talk about bombing datacenters is crazy, but oversight is not. https://www.youtube.com/watch?v=bQfJi7rjuEk https://www.youtube.com/watch?v=bQfJi7rjuEk
    - matheusmoreira 3y ago
      Yeah, sure. "Oversight". Funny how that only ever seems to benefit governments and entrenched corporate interests. Can't let normal citizens have free computing, lest they start encrypting things or violating copyright at massive scales. Now the new boogeyman is "unrestricted" AI. Can't let people have that, it might provide people instructions about sarin gas manufacturing or something. The government intelligence agencies totally want full access though, they can feed everyone's communications into it so they can ask it questions about the people they're surveilling. Is person X planning a terrorist attack? Is there anything that could ruin person Y's reputation? The truth is these technologies are subversive. They literally subvert the existing social order and impose a new one. A new social order where the users of the technology are empowered. They can't tolerate it when we are the users. It takes their power away. So they ban it, control it, flip it around on us and turn it into the very mechanism of our wide-scale oppression.
      - pas 3y ago
- rramadass 3y ago
  > Then it should be a fundamental mission of people who DO have technical knowledge to spread the word on both the negative and positive effects of AI. Researchers have done so. One good resource is Melanie Mitchell's book Artificial Intelligence: A Guide for Thinking Humans (i presume a 2nd edition is in works to incorporate LLMs/ChatGPT etc.) Also checkout her videos on youtube. The problem is that technologically clueless vested interests in the business/media push sound bites to the already distracted public thus distorting everything.
newsclues 3y ago
[flagged]
- thaanpaa 3y ago
  It's a marketing tactic in the case of artificial intelligence. AI promoters all want to make people believe that it is far more advanced than it actually is. Although, I'm sure that many people who work in the industry also fall for the hype, as many of them seem to exhibit cultish behavior.
  - newsclues 3y ago
    The marketing tactic was a not unique to AI and is being actively weaponized in modern times
wongarsu 3y ago
AI safety has been a thing since long before ChatGPT viable LLMs, and most of those people are legitimate, with good research that solves real (current or potential future) problems. But that is currently getting drowned out by people hopping on the LLM bandwagon; with AI Safety as a good way to profit from the current hype or to build a moat around your own LLM.
- loveparade 3y ago
  Depends on your definition of AI Safety. Like you said, there is in fact some very legit research that (unfortunately) is done under the umbrella of "AI Safety", but that's quite different from the threat-to-humanity AI safety crowd.
127 3y ago
False. AI alignment makes the LLM less effective. It's not pure virtue signaling if there's a cost. It's actual virtue.
lucubratory 3y ago
That is just objectively untrue. Geoffrey Hinton, Ilya Sutskever, Alexander Madry, Yoshua Bengio, Ya-Qin Zhang, Stuart Russel, Demis Hassabis, Dario Amodei, Shane Legg, Yi Zeng, Albert Efimov, Jianyi Zhang, Peter Norvig, Eric Horvitz, Ian Goodfellow, Mary Phuong, Max Tegmark, Jan Hendrik Kirchner, Jason Wei, Geoffrey Irving, Jan Leike, and many many more that I'm not going to just continually type out. If you actually believe this you are not familiar with the field.
- gcr 3y ago
  what are you trying to say with this giant list of names of various machine learning researchers? Most of these aren’t even EA folks
  - lucubratory 3y ago
    They are a small subset of the large number of people that signed onto a very short open letter, stating: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." I only listed the machine learning researchers who are famous or have significant achievements in AI research or engineering in my estimation (not all of them, full list of signatories at https://www.safe.ai/statement-on-ai-risk https://www.safe.ai/statement-on-ai-risk), because the comment I was responding to claimed that anyone who knew this subject was not concerned about the safety of AI systems and this list repudiates that as laughably inaccurate. There are many other people who signed onto that letter, but the commenter I responded to presumably doesn't care about their opinions because they're not experts specifically on building these systems or researching the science behind them. "Effective Altruism" is a dumb social club that at best makes people who would do good things anyway think about that in really unhealthy ways (e.g. thinking about how many lives you have or haven't saved every single day), and at worst provides ideological cover to some truly odious views (e.g. the persistent presence of eugenicists who don't get repudiated by other EA people). I don't like them, and I think all the "value cone" crap is bloviating, worse than useless for informing our actual decisions and anti-convincing to people who aren't bought into their STEM Sierra Club shtick. They don't have a monopoly on being concerned about the safety of artificially intelligent systems, or on being concerned that the development of such systems has a non-zero chance of causing the extinction of humanity. We should take that possibility seriously and make sure it's not going to happen or we at least have a good idea of the exact risks before we build one or engage in research projects or engineering that could accidentally create an artifact which lead to building one.
username332211 3y ago
> It's just another veil, like Effective Altruism I'm afraid EA is not a veil. Altruism is a deeply disturbing moral philosophy, if you actually think about it's implications and how it differs from it's originator - utilitarianism. Similarly, AI safety is something that's religiously believed by some quarters, not all of them ignorant. According to The Atlantic's article on the drama that was posted here earlier[1], Sutskever - supposedly the brightest scientific mind in OpenAI believed in things that would not be out of place in the MIRI cult of old. [1] https://www.theatlantic.com/technology/archive/2023/11/sam-altman-open-ai-chatgpt-chaos/676050/ https://www.theatlantic.com/technology/archive/2023/11/sam-a...
chronofar 3y ago
Babies and bathwater. The philosophical underpinnings of AI Safety and Effective Altruism absolutely have merit and warrant consideration. Of course as with most things human group/tribal tendencies can tend to overwhelm such underpinnings rather quickly, and it should be no surprise when it turns out individuals who broadcasted belonging to x or y group turned out to be less than trustworthy (or merely incorrect/misguided) along various dimensions. We can critique the specific actions or beliefs of such actors and question broad labels worn as badges without casting dispersions on a wide swath of potential tenets that may fit into such a label as "complete BS".
- rapnie 3y ago
  The danger of those broad terms when they become trendy is how effective they are as a facade for all kinds of malpractices to hide behind and fester. A term popular some time ago was "Microfinancing". On TV you witnessed celebrities proudly micro-investing directly in people with few financial means. But for ruthless types it was also a way to put naive people in severe debt with extortionary loans. "UBI" is another such term, with a lot of well-meaning proponents. And sometimes advocates one wouldn't expect. For UBI I wonder if the abuse may lie in how it can lull people to sleep, while pervasive use of AI eats into the low(er)-skilled labor markets.
- mejutoco 3y ago
  I think EA is at its essence "the end justify the means", but communicated in a less simple way. I do not see what is so interesting or novel about it.
  - chronofar 3y ago
    Not really at all, if we were to try and reduce EA to a pithy one-liner it would be more like "if one aspires to do good (altruism), one should use evidenced based data to determine how they can do the most good with their limited resources (effective)."
    - skohan 3y ago
rramadass 3y ago
All of the talk about "AI Safety" and "Effective Altruism" cannot be dismissed as BS. As technology has become more and more integrated into the very fabric of society, some dangers are very real unless we explicitly use Human standards of Morals/Ethics to regulate them. Of course this should be no excuse to go off into la-la land. From https://en.wikipedia.org/wiki/AI_safety https://en.wikipedia.org/wiki/AI_safety Scholars discuss current risks from critical systems failures, bias, and AI enabled surveillance; emerging risks from technological unemployment, digital manipulation, and weaponization; and speculative risks from losing control of future artificial general intelligence (AGI) agents. Also watch Munk debate: AI research and development poses an existential threat featuring Yoshua Bengio, Max Tegmark, Melanie Mitchell and Yann LeCun - https://www.youtube.com/watch?v=144uOfr4SYA https://www.youtube.com/watch?v=144uOfr4SYA
sfjailbird 3y ago
> The only people who truly believe this AI safety stuff are those with no technical knowledge or expertise. Elon Musk seems to be really into it.
lewhoo 3y ago
I don't think this is entirely true. There are degrees to AI safety dependent on what we input as AI in the equation. You typically don't hear alarm bells when AlphaFold is discussed. LLM's indeed may produce harmful content - just like an unmoderated web forum, and perhaps discussing safety in this context is justified. Sutskever is of opinion (or at least how I understand it) that LLMs' emergent properties will eventually give them the power to extrapolate in a superintelligent way. In other words if you ask it to assume it's now a superintelligent entity outsmarting humans by x10 it will truly deliver. If intelligence in itself is an amplifier of capability it is I think justified to wonder how to limit the negative impact in this case.
digbybk 3y ago
> The only people who truly believe this AI safety stuff are those with no technical knowledge or expertise. This is a demonstrably false claim that gets repeated over and over. 2 out of 3 Turing Award recipients for work in AI are people who talk about AI safety, and even existential risk. There are many more. It may actually be BS but it’s simply false to say that it’s based on technical ignorance. The question I have is why is this claim made over and over when it is so easily seen to be untrue?
ryloric 3y ago
AI Safety is real and very important... AI Safety financed and practiced by corps looking to make money off of AI is mostly bullshit. I think it's important to make a distinction between the two.
tnecniv 3y ago
Well I think it’s a bit harder to parse out for most people. Effective altruism is easily seen as garbage when you learn the definition. AI Safety is a bit more nuanced. I think there’s a lot of issues that are nonsense like having an “aligned AGI.” Aligned with whom? I could ostensibly envision you aligning it with a single person but aligning it with “humanity” is infeasible because humans are unaligned with each other. However there is an actual technical problem worth solving in alignment, which is basically matching the output of the AI with what the human intends not what the human asks. Moreover, AI has been improperly used already in ways that are plausibly avoidable. A classic example is algorithmically approving bank loans leading to members of ethnic groups getting denied at a higher rate due to their zip code, which is likely irrelevant to the loan. The designers of the algorithm weren’t explicitly trying to be racist (that would have massive legal ramifications on top of just being plainly unethical, but maybe they did it intentionally and I forgot), they screwed up and didn’t realize they produced an algorithm unaligned with their intentions. Figuring out how to avoid scenarios like that through either methodology or algorithmically is an interesting and much more tractable research problem than making an “AI that benefits all of humanity.”

vouaobrasil 3y ago

Then it should be a fundamental mission of people who DO have technical knowledge to spread the word on both the negative and positive effects of AI. If WE have the knowledge, WE have the responsibility to go out there and let people know. But as far as I know, most technical people are happy to discuss it amongst themselves and not even say one word to people who have no technical knowledge. Not one word. We need more balanced opinions in a world where the only ideas that survive are the ones that promise short-term gains at the cost of long-term disasters.

loveparade 3y ago
The problem is the same as some of the woke movements. Nobody wants to publicly speak out against AI safety BS because a Twitter mob who has made a career out of it will descend onto them and ruin their career. There is nothing to gain from speaking out against it, but you gain career opportunities by being part of the AI safety BS community. In fact, this has happened with Hinton (I think it was?) when he dared to explain how models aren't inherently biased. Had to publicly apologize for it later. Even ML conferences have AI safety tracks these days. Nobody sane wants to be excluded from opportunities.
- vouaobrasil 3y ago
  Well, I for one am speaking out against it (and against AI in general). I have a blog, Twitter, newsletter, and I even post warnings on my YouTube channel from time to time. I was part of a discussion that led my employer to commit to NOT using generative AI. No Twitter mob is going to stop me. They can descend all they want and pretend to be brave in front of a keyboard and screen. As far as I'm concerned, they're cowards.
  - KingOfCoders 3y ago
    "against AI safety" You speak against AI and against AI safety? In what way? Wouldn't be speaking against AI enough? Or would AI safety mean laws to prevent AI existence? Like in the Algebraist? (though it is funny in that book how the powerful use AI when it suits them).
    - vouaobrasil 3y ago
      I speak out against both. I am strongly against AI. But I am also against AI safety in general because I think AI safety is: (1) A false amelioration that cannot prevent the dangers of AI (2) A false promise to people that AI can be made safe (3) A weak on-the-fence position to take of people who think they can have their cake and eat it too.
      - calf 3y ago
        But is it impossible to study AI safety (or "safety") scientifically? What about all the papers like AI watermarking or AI racial bias and so for? Do you argue that academics working on those questions are doing illegitimate science?
        vouaobrasil 3y ago
        I don't think the science is illegitimate in the sense that it doesn't follow the scientific method or at least the principles of science or engineering in some form. Nor do I think they have bad intentions. BUT, I do think their work is useless and only serves to better integrate AI into to the system, which is better in the short term but worse in the long term. Therefore, overall, I vehemently disagree with their research and I think it is fundamentally a contribution to the worsening of the earth. I absolutely AGREE with treating all people equally and eliminating racial biases of course. However, TECH companies have a VESTED interest in this sort of thing, which is to integrate as many people as possible into the tech empire. It just so happens that building the tech empire is roughly equivalent to success in modern society, which is why their intentions seem good, but they are only ostensibly good. In reality, they do no good of any kind.
TotalCrackpot 3y ago
If you are actually interested in that you can promote voices like this (Why We Must Resist AI w/ Dan McQuillan): https://news.ycombinator.com/item?id=38362498 https://news.ycombinator.com/item?id=38362498
- vouaobrasil 3y ago
  Thanks, I'll look into it!
bsenftner 3y ago
The problem with AI has never been the technology that composes AI, the problem has always and will forever be the immature yet powerful humans that take advantage of less advantaged humans using whatever means they have, which today can include AI. The issue is not a technology issue, it's a human psychology and sociology issue, it's our immature humans in society that is the issue. Technologists cannot speak about that in any meaningful manner.
- vouaobrasil 3y ago
  I absolutely disagree. Technologists understand the true nature of AI and can thus predict its uses. Moreover, if technologists realize this problem of how AI will inevitably be used, their voice will be a powerful message that even those who enjoy math and science are fearful of AI.
  - bsenftner 3y ago
    Nope, nope. Technologists do not understand the true nature of AI. What technologists know they can rarely communicate, and even less so to a general public. Technology developers are not some special deep understanders that ordinary humans are not; we just have more time spent in our specialization than they, as they have in their specializations which we do not. If anything, technology developers, due to our specialization's time requirements and the traditional lack of professional communications training, tend to be weaker communicators than the general public. The last people you want explaining AI to a potentially fearful general public is weak communicators, such as the AI developers.
    - Gud 3y ago
      Just because many technologists are poor communicators, doesn't mean all are. Carl Sagan.
- RhysU 3y ago
  Michael Crichton has (rightly) entered the chat. https://en.m.wikipedia.org/wiki/Michael_Crichton https://en.m.wikipedia.org/wiki/Michael_Crichton Crichton says these failures of humanity are inevitable because there will always be a technologist who does things because they can, not because they should. That everything will go off the rails, no matter what people attempt, is the plot of the novel Jurassic Park.
matheusmoreira 3y ago
There should be no need for us to spread the word about anything. Technology needs to be built and the chips need to fall where they may. If society changes as a result, so be it. I'm seriously tired of these vested interests destroying perfectly good technology just because it would endanger their precious business models. Computers should have put an end to silly things like copyright but they would rather destroy our computing freedom than allow that to happen. Now we have these awesome language models and AIs and they're desperately trying to limit and control it. It's a tragedy in of itself that we even need to fight these political battles at all.
- pas 3y ago
  the strong needs to just take whatever it wants, and so be it? there's no separating technology from context, as in consequences. yes, all the talk about bombing datacenters is crazy, but oversight is not. https://www.youtube.com/watch?v=bQfJi7rjuEk https://www.youtube.com/watch?v=bQfJi7rjuEk
  - matheusmoreira 3y ago
    Yeah, sure. "Oversight". Funny how that only ever seems to benefit governments and entrenched corporate interests. Can't let normal citizens have free computing, lest they start encrypting things or violating copyright at massive scales. Now the new boogeyman is "unrestricted" AI. Can't let people have that, it might provide people instructions about sarin gas manufacturing or something. The government intelligence agencies totally want full access though, they can feed everyone's communications into it so they can ask it questions about the people they're surveilling. Is person X planning a terrorist attack? Is there anything that could ruin person Y's reputation? The truth is these technologies are subversive. They literally subvert the existing social order and impose a new one. A new social order where the users of the technology are empowered. They can't tolerate it when we are the users. It takes their power away. So they ban it, control it, flip it around on us and turn it into the very mechanism of our wide-scale oppression.
    - pas 3y ago
rramadass 3y ago
> Then it should be a fundamental mission of people who DO have technical knowledge to spread the word on both the negative and positive effects of AI. Researchers have done so. One good resource is Melanie Mitchell's book Artificial Intelligence: A Guide for Thinking Humans (i presume a 2nd edition is in works to incorporate LLMs/ChatGPT etc.) Also checkout her videos on youtube. The problem is that technologically clueless vested interests in the business/media push sound bites to the already distracted public thus distorting everything.

newsclues 3y ago

[flagged]

thaanpaa 3y ago
It's a marketing tactic in the case of artificial intelligence. AI promoters all want to make people believe that it is far more advanced than it actually is. Although, I'm sure that many people who work in the industry also fall for the hype, as many of them seem to exhibit cultish behavior.
- newsclues 3y ago
  The marketing tactic was a not unique to AI and is being actively weaponized in modern times

wongarsu 3y ago

AI safety has been a thing since long before ChatGPT viable LLMs, and most of those people are legitimate, with good research that solves real (current or potential future) problems. But that is currently getting drowned out by people hopping on the LLM bandwagon; with AI Safety as a good way to profit from the current hype or to build a moat around your own LLM.

loveparade 3y ago
Depends on your definition of AI Safety. Like you said, there is in fact some very legit research that (unfortunately) is done under the umbrella of "AI Safety", but that's quite different from the threat-to-humanity AI safety crowd.

127 3y ago

False. AI alignment makes the LLM less effective. It's not pure virtue signaling if there's a cost. It's actual virtue.

lucubratory 3y ago

That is just objectively untrue. Geoffrey Hinton, Ilya Sutskever, Alexander Madry, Yoshua Bengio, Ya-Qin Zhang, Stuart Russel, Demis Hassabis, Dario Amodei, Shane Legg, Yi Zeng, Albert Efimov, Jianyi Zhang, Peter Norvig, Eric Horvitz, Ian Goodfellow, Mary Phuong, Max Tegmark, Jan Hendrik Kirchner, Jason Wei, Geoffrey Irving, Jan Leike, and many many more that I'm not going to just continually type out. If you actually believe this you are not familiar with the field.

gcr 3y ago
what are you trying to say with this giant list of names of various machine learning researchers? Most of these aren’t even EA folks
- lucubratory 3y ago
  They are a small subset of the large number of people that signed onto a very short open letter, stating: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." I only listed the machine learning researchers who are famous or have significant achievements in AI research or engineering in my estimation (not all of them, full list of signatories at https://www.safe.ai/statement-on-ai-risk https://www.safe.ai/statement-on-ai-risk), because the comment I was responding to claimed that anyone who knew this subject was not concerned about the safety of AI systems and this list repudiates that as laughably inaccurate. There are many other people who signed onto that letter, but the commenter I responded to presumably doesn't care about their opinions because they're not experts specifically on building these systems or researching the science behind them. "Effective Altruism" is a dumb social club that at best makes people who would do good things anyway think about that in really unhealthy ways (e.g. thinking about how many lives you have or haven't saved every single day), and at worst provides ideological cover to some truly odious views (e.g. the persistent presence of eugenicists who don't get repudiated by other EA people). I don't like them, and I think all the "value cone" crap is bloviating, worse than useless for informing our actual decisions and anti-convincing to people who aren't bought into their STEM Sierra Club shtick. They don't have a monopoly on being concerned about the safety of artificially intelligent systems, or on being concerned that the development of such systems has a non-zero chance of causing the extinction of humanity. We should take that possibility seriously and make sure it's not going to happen or we at least have a good idea of the exact risks before we build one or engage in research projects or engineering that could accidentally create an artifact which lead to building one.

username332211 3y ago

> It's just another veil, like Effective Altruism I'm afraid EA is not a veil. Altruism is a deeply disturbing moral philosophy, if you actually think about it's implications and how it differs from it's originator - utilitarianism. Similarly, AI safety is something that's religiously believed by some quarters, not all of them ignorant. According to The Atlantic's article on the drama that was posted here earlier[1], Sutskever - supposedly the brightest scientific mind in OpenAI believed in things that would not be out of place in the MIRI cult of old. [1] https://www.theatlantic.com/technology/archive/2023/11/sam-altman-open-ai-chatgpt-chaos/676050/ https://www.theatlantic.com/technology/archive/2023/11/sam-a...

chronofar 3y ago

Babies and bathwater. The philosophical underpinnings of AI Safety and Effective Altruism absolutely have merit and warrant consideration. Of course as with most things human group/tribal tendencies can tend to overwhelm such underpinnings rather quickly, and it should be no surprise when it turns out individuals who broadcasted belonging to x or y group turned out to be less than trustworthy (or merely incorrect/misguided) along various dimensions. We can critique the specific actions or beliefs of such actors and question broad labels worn as badges without casting dispersions on a wide swath of potential tenets that may fit into such a label as "complete BS".

rapnie 3y ago
The danger of those broad terms when they become trendy is how effective they are as a facade for all kinds of malpractices to hide behind and fester. A term popular some time ago was "Microfinancing". On TV you witnessed celebrities proudly micro-investing directly in people with few financial means. But for ruthless types it was also a way to put naive people in severe debt with extortionary loans. "UBI" is another such term, with a lot of well-meaning proponents. And sometimes advocates one wouldn't expect. For UBI I wonder if the abuse may lie in how it can lull people to sleep, while pervasive use of AI eats into the low(er)-skilled labor markets.
mejutoco 3y ago
I think EA is at its essence "the end justify the means", but communicated in a less simple way. I do not see what is so interesting or novel about it.
- chronofar 3y ago
  Not really at all, if we were to try and reduce EA to a pithy one-liner it would be more like "if one aspires to do good (altruism), one should use evidenced based data to determine how they can do the most good with their limited resources (effective)."
  - skohan 3y ago

rramadass 3y ago

All of the talk about "AI Safety" and "Effective Altruism" cannot be dismissed as BS. As technology has become more and more integrated into the very fabric of society, some dangers are very real unless we explicitly use Human standards of Morals/Ethics to regulate them. Of course this should be no excuse to go off into la-la land. From https://en.wikipedia.org/wiki/AI_safety https://en.wikipedia.org/wiki/AI_safety Scholars discuss current risks from critical systems failures, bias, and AI enabled surveillance; emerging risks from technological unemployment, digital manipulation, and weaponization; and speculative risks from losing control of future artificial general intelligence (AGI) agents. Also watch Munk debate: AI research and development poses an existential threat featuring Yoshua Bengio, Max Tegmark, Melanie Mitchell and Yann LeCun - https://www.youtube.com/watch?v=144uOfr4SYA https://www.youtube.com/watch?v=144uOfr4SYA

sfjailbird 3y ago

> The only people who truly believe this AI safety stuff are those with no technical knowledge or expertise. Elon Musk seems to be really into it.

lewhoo 3y ago

I don't think this is entirely true. There are degrees to AI safety dependent on what we input as AI in the equation. You typically don't hear alarm bells when AlphaFold is discussed. LLM's indeed may produce harmful content - just like an unmoderated web forum, and perhaps discussing safety in this context is justified. Sutskever is of opinion (or at least how I understand it) that LLMs' emergent properties will eventually give them the power to extrapolate in a superintelligent way. In other words if you ask it to assume it's now a superintelligent entity outsmarting humans by x10 it will truly deliver. If intelligence in itself is an amplifier of capability it is I think justified to wonder how to limit the negative impact in this case.

digbybk 3y ago

> The only people who truly believe this AI safety stuff are those with no technical knowledge or expertise. This is a demonstrably false claim that gets repeated over and over. 2 out of 3 Turing Award recipients for work in AI are people who talk about AI safety, and even existential risk. There are many more. It may actually be BS but it’s simply false to say that it’s based on technical ignorance. The question I have is why is this claim made over and over when it is so easily seen to be untrue?

ryloric 3y ago

AI Safety is real and very important... AI Safety financed and practiced by corps looking to make money off of AI is mostly bullshit. I think it's important to make a distinction between the two.

tnecniv 3y ago

Well I think it’s a bit harder to parse out for most people. Effective altruism is easily seen as garbage when you learn the definition. AI Safety is a bit more nuanced. I think there’s a lot of issues that are nonsense like having an “aligned AGI.” Aligned with whom? I could ostensibly envision you aligning it with a single person but aligning it with “humanity” is infeasible because humans are unaligned with each other. However there is an actual technical problem worth solving in alignment, which is basically matching the output of the AI with what the human intends not what the human asks. Moreover, AI has been improperly used already in ways that are plausibly avoidable. A classic example is algorithmically approving bank loans leading to members of ethnic groups getting denied at a higher rate due to their zip code, which is likely irrelevant to the loan. The designers of the algorithm weren’t explicitly trying to be racist (that would have massive legal ramifications on top of just being plainly unethical, but maybe they did it intentionally and I forgot), they screwed up and didn’t realize they produced an algorithm unaligned with their intentions. Figuring out how to avoid scenarios like that through either methodology or algorithmically is an interesting and much more tractable research problem than making an “AI that benefits all of humanity.”