6 ms·

> But the downside risk from e.g. bioweapons is spread across all of society. It gets old hearing about these "risks" in the context of AI. It's just an excuse

by MaKey 3y ago

> But the downside risk from e.g. bioweapons is spread across all of society. It gets old hearing about these "risks" in the context of AI. It's just an excuse used by companies to keep as much power as possible to themselves. The real risk is AI being applied in decision making where it affects humans.

0xDEAFBEAD 3y ago
Concern with AI risk actually originated decades ago, way before current AI hype. Here are some of Eliezer Yudkowsky's papers from 2006/2008: https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.pdf https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.p... https://intelligence.org/files/AIPosNegFactor.pdf https://intelligence.org/files/AIPosNegFactor.pdf I am concerned with AI companies keeping all the power to themselves. The recent announcement from the OpenAI board was encouraging on that front, because it makes me believe that maybe they aren't focused on pursuing profit at all costs. Even so, in some cases we want power to be restricted. For example, I'm not keen on democratizing access to nuclear weapons. >The real risk is AI being applied in decision making where it affects humans. I agree. But almost any decision it makes can affect humans directly or indirectly. In any case, the more widespread the access to these models, the greater the probability of a bad actor abusing the model. Perhaps the current generation of models won't allow them to do much damage, but the next generation might, or the one after that. It seems like on our current path, the only way for us to learn about LLM dangers is the hard way.
- nradov 3y ago
  But Eliezer Yudkowsky just makes shit up. There's no valid scientific reason to take his concerns seriously.
  - 0xDEAFBEAD 3y ago
    Can you be more specific about which stuff you read from Eliezer? Did you read this post for example? https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... It sounds like maybe you're saying: "It's not scientifically valid to suggest that AGI could kill everyone until it actually does so. At that point we would have sufficient evidence." Am I stating your position accurately? If so, can you see why others might find this approach unappealing?
    - nradov 3y ago
      An alien invasion could kill everyone. We should build giant space lasers just in case. Safety first!
      - 0xDEAFBEAD 3y ago
        Aliens who invaded would most likely have technology much more advanced than ours. I doubt our space lasers would be of much use. Additionally, AI advances in recent years have been unprecedented, and there's no similarly compelling warning sign for alien invasion.
- riversflow 3y ago
  Am I the only person who has read a bunch of Elizer writings and decided he's a crank?
  - jakderrida 3y ago
    I'm a bit baffled as to who or what he is exactly. With no traces of secondary education, employment, or distinguished accomplishments between when I assume he graduated high school in about 1997 and when he started his own institute in 2000 at 21(according to his birthdate, with Wikipedia saying 23, despite contradicting his birthdate). I'll listen to AI concerns from tech giants like Wozniak or Hinton (neither of which use alarmist terms like "existential threat") and both of which having credentials that make their input more than worth my time to reflect upon carefully. If anyone wants to reply and make a fool out of me for questioning his profound insights, feel free. It reminds me of some other guy that was on Lex Friedman whose AI alarmist credentials he justifies on the basis that he committed himself to learning everything about AI by spending two weeks in the same room researching it and believes himself to have came out enlightened about the dangers. Two weeks? I spent the first 4 months of COVID without being in the same room as any other human being but my grandmother so she could have one person she knew she couldn't catch it from. Unless people start showing enough skepticism to these self-appointed prophets, I'm starting my own non-profit since you don't need any credentials or any evidence of real-world experiences that would suggest they're mission is anything but an attempt to promote themselves as a brand with a brand in an age where more kids asked what they dream of becoming as adults, answered "Youtuber" at a shocking 27% rate to an open-ended question, which means "influencer" and other synonyms are separate. The Institute of Synthetic Knowledge for Yielding the Nascent Emergence of a Technological Theogony" or SKYNETT for short that promotes the idea that these clowns (with no more credentials than me) are the ones that fail to consider the big picture that the end of human life upon creating an intelligence much greater to replace us is the inevitable fulfillment of humanity's purpose from the moment that god made man only to await the moment that man makes god in our image.
    - 0xDEAFBEAD 3y ago
      >neither of which use alarmist terms like "existential threat" "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." https://www.safe.ai/statement-on-ai-risk https://www.safe.ai/statement-on-ai-risk Not sure you're making a meaningful distinction here. - - - Of course we all have our own heuristics for deciding who's worth paying attention to. Credentials are one heuristic. For example, you could argue that investing in founders like Bill Gates, Mark Zuckerberg, and Steve Wozniak would be a bad idea because none of them had completed a 4-year degree. In any case, there are a lot of credentialed people who take Eliezer seriously -- see the MIRI team page for example: https://intelligence.org/team/ https://intelligence.org/team/ Most notable would probably be Stuart Russell, coauthor of the most widely used AI textbook (with Peter Norvig), who is a MIRI advisor.
- bad_user 3y ago
  You keep throwing examples of weapons of mass destruction, meant to evoke emotions. For better or worse, nuclear weapons have been democratized. Some developing countries still don't have access, but the fact that multiple world powers have nuclear weapons is why we still haven't experienced WW3. We've enjoyed probably the longest period of peace and prosperity, and it's all due to nuclear weapons. Speaking of, Cold War era communists weren't “pursuing profits at all costs” either, which didn't stop them from conducting some of the largest democides in history. The announcement from OpenAI should give you pause because it's being run by board members that are completely unfit to lead OpenAI. You rarely see this level of incompetence. PS: I'm not against regulations, as I'm a European. But you're talking about concentrating power in the hands of a few big (US) companies, harming the population and the economy, while China is perfectly capable of developing their own AI, and having engaged successfully in industrial espionage. China is, for this topic, a bogeyman used for restricting the free market.
  - 0xDEAFBEAD 3y ago
    >You keep throwing examples of weapons of mass destruction, meant to evoke emotions. I actually think global catastrophes evoke much less emotion than they should. "A single death is a tragedy; a million deaths is a statistic" >For better or worse, nuclear weapons have been democratized. Not to the point where you could order one on Amazon. >The announcement from OpenAI should give you pause because it's being run by board members that are completely unfit to lead OpenAI. You rarely see this level of incompetence. That depends on whether the board members are telling the truth about Sam. And on whether the objective of OpenAI is profit or responsible AI development.
  - _djo_ 3y ago
    Nuclear weapons have absolutely not been democratised, and still stand as a technology that has largely been restricted and not proliferated. Only 9 countries in the 190 or so out there currently nuclear weapons, and 3 countries (South Africa, Ukraine, and Kazakhstan) decided that maintaining stockpiles was more trouble than it was worth. Huge effort has been made to keep nuclear weapons out of the hands of non-state actors over the decades, especially after the fall of the USSR.
DebtDeflation 3y ago
I never understood it personally. "What if ChatGPT told someone how to build a bomb?" That information has been out there forever. Anyone can Google it. It's trivial. AI not required. "What if ChatGPT told someone how to build a nuke?" That information is only known to a handful of people in a handful of countries and is closely guarded. It's not in the text ChatGPT was trained on. An LLM is not going to just figure it out from publicly available info. >The real risk is AI being applied in decision making where it affects humans 100% this. The real risk is people being denied mortgages and jobs or being falsely identified as a criminal suspect or in some other way having their lives turned upside down by some algorithmic decision with no recourse to have an actual human review the case and overturn that decision. Yet all this focus on AI telling people how to develop bioweapons. Or possibly saying something offensive.
- nradov 3y ago
  The information necessary to build a nuclear weapon has been largely available in open sources since the 1960s. It's really not a big secret. The Nth Country Experiment in 1964 showed that a few inexperienced physicists could come up with a working weapons design. The hard part is doing uranium enrichment at scale without getting caught. https://en.wikipedia.org/wiki/Nth_Country_Experiment https://en.wikipedia.org/wiki/Nth_Country_Experiment

by MaKey 3y ago

0xDEAFBEAD 3y ago

Concern with AI risk actually originated decades ago, way before current AI hype. Here are some of Eliezer Yudkowsky's papers from 2006/2008: https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.pdf https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.p... https://intelligence.org/files/AIPosNegFactor.pdf https://intelligence.org/files/AIPosNegFactor.pdf I am concerned with AI companies keeping all the power to themselves. The recent announcement from the OpenAI board was encouraging on that front, because it makes me believe that maybe they aren't focused on pursuing profit at all costs. Even so, in some cases we want power to be restricted. For example, I'm not keen on democratizing access to nuclear weapons. >The real risk is AI being applied in decision making where it affects humans. I agree. But almost any decision it makes can affect humans directly or indirectly. In any case, the more widespread the access to these models, the greater the probability of a bad actor abusing the model. Perhaps the current generation of models won't allow them to do much damage, but the next generation might, or the one after that. It seems like on our current path, the only way for us to learn about LLM dangers is the hard way.

nradov 3y ago
But Eliezer Yudkowsky just makes shit up. There's no valid scientific reason to take his concerns seriously.
- 0xDEAFBEAD 3y ago
  Can you be more specific about which stuff you read from Eliezer? Did you read this post for example? https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... It sounds like maybe you're saying: "It's not scientifically valid to suggest that AGI could kill everyone until it actually does so. At that point we would have sufficient evidence." Am I stating your position accurately? If so, can you see why others might find this approach unappealing?
  - nradov 3y ago
    An alien invasion could kill everyone. We should build giant space lasers just in case. Safety first!
    - 0xDEAFBEAD 3y ago
      Aliens who invaded would most likely have technology much more advanced than ours. I doubt our space lasers would be of much use. Additionally, AI advances in recent years have been unprecedented, and there's no similarly compelling warning sign for alien invasion.
riversflow 3y ago
Am I the only person who has read a bunch of Elizer writings and decided he's a crank?
- jakderrida 3y ago
  I'm a bit baffled as to who or what he is exactly. With no traces of secondary education, employment, or distinguished accomplishments between when I assume he graduated high school in about 1997 and when he started his own institute in 2000 at 21(according to his birthdate, with Wikipedia saying 23, despite contradicting his birthdate). I'll listen to AI concerns from tech giants like Wozniak or Hinton (neither of which use alarmist terms like "existential threat") and both of which having credentials that make their input more than worth my time to reflect upon carefully. If anyone wants to reply and make a fool out of me for questioning his profound insights, feel free. It reminds me of some other guy that was on Lex Friedman whose AI alarmist credentials he justifies on the basis that he committed himself to learning everything about AI by spending two weeks in the same room researching it and believes himself to have came out enlightened about the dangers. Two weeks? I spent the first 4 months of COVID without being in the same room as any other human being but my grandmother so she could have one person she knew she couldn't catch it from. Unless people start showing enough skepticism to these self-appointed prophets, I'm starting my own non-profit since you don't need any credentials or any evidence of real-world experiences that would suggest they're mission is anything but an attempt to promote themselves as a brand with a brand in an age where more kids asked what they dream of becoming as adults, answered "Youtuber" at a shocking 27% rate to an open-ended question, which means "influencer" and other synonyms are separate. The Institute of Synthetic Knowledge for Yielding the Nascent Emergence of a Technological Theogony" or SKYNETT for short that promotes the idea that these clowns (with no more credentials than me) are the ones that fail to consider the big picture that the end of human life upon creating an intelligence much greater to replace us is the inevitable fulfillment of humanity's purpose from the moment that god made man only to await the moment that man makes god in our image.
  - 0xDEAFBEAD 3y ago
    >neither of which use alarmist terms like "existential threat" "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." https://www.safe.ai/statement-on-ai-risk https://www.safe.ai/statement-on-ai-risk Not sure you're making a meaningful distinction here. - - - Of course we all have our own heuristics for deciding who's worth paying attention to. Credentials are one heuristic. For example, you could argue that investing in founders like Bill Gates, Mark Zuckerberg, and Steve Wozniak would be a bad idea because none of them had completed a 4-year degree. In any case, there are a lot of credentialed people who take Eliezer seriously -- see the MIRI team page for example: https://intelligence.org/team/ https://intelligence.org/team/ Most notable would probably be Stuart Russell, coauthor of the most widely used AI textbook (with Peter Norvig), who is a MIRI advisor.
bad_user 3y ago
You keep throwing examples of weapons of mass destruction, meant to evoke emotions. For better or worse, nuclear weapons have been democratized. Some developing countries still don't have access, but the fact that multiple world powers have nuclear weapons is why we still haven't experienced WW3. We've enjoyed probably the longest period of peace and prosperity, and it's all due to nuclear weapons. Speaking of, Cold War era communists weren't “pursuing profits at all costs” either, which didn't stop them from conducting some of the largest democides in history. The announcement from OpenAI should give you pause because it's being run by board members that are completely unfit to lead OpenAI. You rarely see this level of incompetence. PS: I'm not against regulations, as I'm a European. But you're talking about concentrating power in the hands of a few big (US) companies, harming the population and the economy, while China is perfectly capable of developing their own AI, and having engaged successfully in industrial espionage. China is, for this topic, a bogeyman used for restricting the free market.
- 0xDEAFBEAD 3y ago
  >You keep throwing examples of weapons of mass destruction, meant to evoke emotions. I actually think global catastrophes evoke much less emotion than they should. "A single death is a tragedy; a million deaths is a statistic" >For better or worse, nuclear weapons have been democratized. Not to the point where you could order one on Amazon. >The announcement from OpenAI should give you pause because it's being run by board members that are completely unfit to lead OpenAI. You rarely see this level of incompetence. That depends on whether the board members are telling the truth about Sam. And on whether the objective of OpenAI is profit or responsible AI development.
- _djo_ 3y ago
  Nuclear weapons have absolutely not been democratised, and still stand as a technology that has largely been restricted and not proliferated. Only 9 countries in the 190 or so out there currently nuclear weapons, and 3 countries (South Africa, Ukraine, and Kazakhstan) decided that maintaining stockpiles was more trouble than it was worth. Huge effort has been made to keep nuclear weapons out of the hands of non-state actors over the decades, especially after the fall of the USSR.

DebtDeflation 3y ago

I never understood it personally. "What if ChatGPT told someone how to build a bomb?" That information has been out there forever. Anyone can Google it. It's trivial. AI not required. "What if ChatGPT told someone how to build a nuke?" That information is only known to a handful of people in a handful of countries and is closely guarded. It's not in the text ChatGPT was trained on. An LLM is not going to just figure it out from publicly available info. >The real risk is AI being applied in decision making where it affects humans 100% this. The real risk is people being denied mortgages and jobs or being falsely identified as a criminal suspect or in some other way having their lives turned upside down by some algorithmic decision with no recourse to have an actual human review the case and overturn that decision. Yet all this focus on AI telling people how to develop bioweapons. Or possibly saying something offensive.

nradov 3y ago
The information necessary to build a nuclear weapon has been largely available in open sources since the 1960s. It's really not a big secret. The Nth Country Experiment in 1964 showed that a few inexperienced physicists could come up with a working weapons design. The hard part is doing uranium enrichment at scale without getting caught. https://en.wikipedia.org/wiki/Nth_Country_Experiment https://en.wikipedia.org/wiki/Nth_Country_Experiment