4 ms·

> The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the

> The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. How sure are you that they don't think the risk is real? I have only third hand accounts, but at least some of the low level people working for OpenAI seemed to think the risk is worth taking seriously, its not just a cynical CEO ploy. Sam Altman knows and has talked to Yudkowsky plenty of times. To me the simplest explanation is that he thinks figuring out alignment via low powered models is the best way of solving the problem, so pushed to get those working and then trying to reduce the rate of progress for a bit while they figure it out. (I think it's a plan that doesn't have good odds, but nothing seems to give amazing odds atm)

nerdponx 3y ago
The best lies contain a kernel of truth.
randomdata 3y ago
> but at least some of the low level people working for OpenAI seemed to think the risk is worth taking seriously Everyone has an opinion, but the experts seem strangely silent on this matter. I mean experts in human risk, not experts in AI. The latter love to ramble on, but are unlikely to know anything about social and humanitarian consequences – at least no more than the stock boy at Walmart. Dedicating one's life to becoming an expert in AI doesn't leave much time to become an expert in anything else. There is only so much time in the day.
- danenania 3y ago
  Conversely, why should we expect an expert in "human risk" with no understanding of AI to have anything productive to add to the conversation? We should be looking for people who are well-grounded in both areas, if any exist.
  - randomdata 3y ago
    What additional information would understanding AI actually bring to the table? Worst case you just assume the AI can become human-like. Certainly all the vocal AI experts can come up with is things that humans already do to each other already, only noting that AI enables it at larger scale. Clearly there is no benefit in understanding the AI technology with respect to this matter when the only thing you can point to is scale. The concept of scalability doesn't require an understanding of AI.
    - danenania 3y ago
      That's like saying that you can regulate nuclear materials without input from nuclear physicists or regulate high risk virology without input from virologists. The goal is (or should be) to determine how we can get the benefits of the new technology (nuclear energy, vaccines, AI productivity boom) while minimizing civilizational risk (nuclear war/terrorism, bioweapons/man-made pandemics, anti-human AI applications). There's no way this can be achieved if you don't understand the actual capabilities or trajectory of the technology. You will either over-regulate and throw out the baby with the bathwater, stopping innovation completely or ensuring it only happens under governments that don't care about human rights, or you will miss massive areas of risk because you don't know how the new technology works, what it's capable of, or where it's heading.
      - randomdata 3y ago
        Experts are not lawmakers. We aren't looking for them to craft regulation, we're looking to hear from them about the various realistic scenarios that could play out. But we're not hearing anything. ...probably because there isn't much to hear. Like, Hinton's big warning is that AI will be used to steal identities. What does that tell us? We already know that identity isn't reliable. We've known that for centuries. Realistically, we've likely known that throughout the entirety of human existence. AI doesn't change anything on that front.
        danenania 3y ago
danenania 3y ago
I agree. I think the concern is genuine. If you follow Sam, he has been talking about it for a long time--long before OpenAI had much success to speak of. It doesn't mean he's right or all his ideas for dealing with it are right, but the idea that he is secretly motivated by greed or power isn't consistent with his past statements or actions imo.

nerdponx 3y ago

The best lies contain a kernel of truth.

randomdata 3y ago

> but at least some of the low level people working for OpenAI seemed to think the risk is worth taking seriously Everyone has an opinion, but the experts seem strangely silent on this matter. I mean experts in human risk, not experts in AI. The latter love to ramble on, but are unlikely to know anything about social and humanitarian consequences – at least no more than the stock boy at Walmart. Dedicating one's life to becoming an expert in AI doesn't leave much time to become an expert in anything else. There is only so much time in the day.

danenania 3y ago
Conversely, why should we expect an expert in "human risk" with no understanding of AI to have anything productive to add to the conversation? We should be looking for people who are well-grounded in both areas, if any exist.
- randomdata 3y ago
  What additional information would understanding AI actually bring to the table? Worst case you just assume the AI can become human-like. Certainly all the vocal AI experts can come up with is things that humans already do to each other already, only noting that AI enables it at larger scale. Clearly there is no benefit in understanding the AI technology with respect to this matter when the only thing you can point to is scale. The concept of scalability doesn't require an understanding of AI.
  - danenania 3y ago
    That's like saying that you can regulate nuclear materials without input from nuclear physicists or regulate high risk virology without input from virologists. The goal is (or should be) to determine how we can get the benefits of the new technology (nuclear energy, vaccines, AI productivity boom) while minimizing civilizational risk (nuclear war/terrorism, bioweapons/man-made pandemics, anti-human AI applications). There's no way this can be achieved if you don't understand the actual capabilities or trajectory of the technology. You will either over-regulate and throw out the baby with the bathwater, stopping innovation completely or ensuring it only happens under governments that don't care about human rights, or you will miss massive areas of risk because you don't know how the new technology works, what it's capable of, or where it's heading.
    - randomdata 3y ago
      Experts are not lawmakers. We aren't looking for them to craft regulation, we're looking to hear from them about the various realistic scenarios that could play out. But we're not hearing anything. ...probably because there isn't much to hear. Like, Hinton's big warning is that AI will be used to steal identities. What does that tell us? We already know that identity isn't reliable. We've known that for centuries. Realistically, we've likely known that throughout the entirety of human existence. AI doesn't change anything on that front.
      - danenania 3y ago

danenania 3y ago

I agree. I think the concern is genuine. If you follow Sam, he has been talking about it for a long time--long before OpenAI had much success to speak of. It doesn't mean he's right or all his ideas for dealing with it are right, but the idea that he is secretly motivated by greed or power isn't consistent with his past statements or actions imo.