4 ms·

>Nobody has been killed by AI Some of us worry that billions of people will be killed by AI in the future -- possibly without anything that you or the average

>Nobody has been killed by AI Some of us worry that billions of people will be killed by AI in the future -- possibly without anything that you or the average decision-maker might regard as a warning. (They're likely to be killed all at the same time.) I.e., it is more like a large asteroid slamming into the Earth than a stream of deaths over time such as produced by the deployment in society of the automobile (except that the asteroid does not have the capability of noticing that it's first plan failed to kill a group of human over there, then to devise a second plan for killing those).

echelon 3y ago
> My worry is that billions of people will be killed (probably all at the same time) by AI in the future Science fiction.
- hollerith 3y ago
  The idea that something with greater cognitive capabilities than us might be dangerous to us occurs to many people: sci-fi writers in large numbers to be sure, but also Alan Turing and a large fraction of currently-living senior AI researchers. What really gets me concerned is the quality of the writing on the subject of how can we design an AI so that it will not want to hurt us (just as we design bridges so that we know from first principles they won't fall down). Most leaders of AI labs have by now written about the topic, but the writings are shockingly bad: everyone has some explanation as to why the AI will turn out to be safe, but there are dozens of orthogonal explanations, some very simplistic, none of which I want to bet my life on or the lives of my younger relatives. Those who do write well about the topic, particularly Eliezer Yudkowsky and Nate Soares of the Machine Intelligence Research Institute, say that it is probably not currently within the capabilities of any living human or group of humans to design an AI to be safe (to humans) the way we design bridges to be safe, and that our best hope is the hope that over the next centuries humankind will become cognitively capable enough to do and that in the meantime people stop trying to create AIs that might turn out to be dangerously capable -- which (because outside of actually doing the training run, we have no way of predicting the effects on capability of the next architectural improvement or the next increase in computing resources devoted to training) basically means stopping all AI research now worldwide and for good measure stopping progress in GPU technology. Eliezer has been full-time employed for over 20 years to work on the issue (and Nate has been for about 15 years) and they've had enough funding to employ at least a dozen researchers and researcher-apprentices over that time to bounce ideas off of in the office.
- sanxiyn 3y ago
  How do you know? If we can agree something about AI, it is that we are ignorant about AI. We were similarly ignorant about recombinant DNA, so Asilomar was very cautious about it. Now we know more, we are less cautious. I still think it was good to be cautious and not to dismiss recombinant DNA concerns as "science fiction".
freedomben 3y ago
Safety and alignment stopped being about preventing AI from killing all humans a while ago. Unless you think that "don't say anything potentially offensive" is in-scope with "don't kill humans and don't take over the world by any means necessary to carry out your prompt."
- hollerith 3y ago
  I agree, but since neither I nor the person I responded to have used either word, what is the relevance?
  - freedomben 3y ago
    Very fair point/question, I should have explicitly drawn this link because my comment was quite ambiguous and making (bad) assumptions on shared context. The relevance is IMHO this bill is largely an ossification at the government level of the safety and alignment philosophy of the big corps. I'm guessing they mainly wrote this bill. It's not the specific words "safey and alignment" that matter, it's the philosophy. If the bill were only covering AI killing machines I'd (probably) be in agreement with it, but it seems significantly more overreaching than that.
    - hollerith 3y ago
      >my comment was quite ambiguous and . . . No harm done! >If the bill were only covering AI killing machines I'd (probably) be in agreement with it, but it seems significantly more overreaching than that. Just to make sure we are on the same page: my main worry is the projects ("deployments"?) that aren't intended to kill anybody, but one of those project ends up killing billions of people anyways. It probably kills absolutely everyone. That one project might be trying to cure cancer. The only way of not incurring this risk of extinction (and of mass death) that I know of is to shut down all AI research now, which I'm guessing you would consider "overreaching". It would be great if there were a way to derive the profound benefits of continuing to do AI research without incurring the extinction risk. If you think you have a way to do that, please let me know. If I agree that your approach is promising, I'll drop everything to make sure you get a high-paying job to develop your approach. There are lots of people who would do that (and lots of high-net-worth people and organizations who would pay you the money). The Machine Intelligence Research Institute for example has a lot of money that was donated to them by cryptocurrency entrepreneurs that they've been holding on to year after year because they cannot think of any good ways to spend it to reduce extinction risk. They'd be eager to give money to anyone that can convince them that they have an approach with even a 1% probability of success.

echelon 3y ago

> My worry is that billions of people will be killed (probably all at the same time) by AI in the future Science fiction.

hollerith 3y ago
The idea that something with greater cognitive capabilities than us might be dangerous to us occurs to many people: sci-fi writers in large numbers to be sure, but also Alan Turing and a large fraction of currently-living senior AI researchers. What really gets me concerned is the quality of the writing on the subject of how can we design an AI so that it will not want to hurt us (just as we design bridges so that we know from first principles they won't fall down). Most leaders of AI labs have by now written about the topic, but the writings are shockingly bad: everyone has some explanation as to why the AI will turn out to be safe, but there are dozens of orthogonal explanations, some very simplistic, none of which I want to bet my life on or the lives of my younger relatives. Those who do write well about the topic, particularly Eliezer Yudkowsky and Nate Soares of the Machine Intelligence Research Institute, say that it is probably not currently within the capabilities of any living human or group of humans to design an AI to be safe (to humans) the way we design bridges to be safe, and that our best hope is the hope that over the next centuries humankind will become cognitively capable enough to do and that in the meantime people stop trying to create AIs that might turn out to be dangerously capable -- which (because outside of actually doing the training run, we have no way of predicting the effects on capability of the next architectural improvement or the next increase in computing resources devoted to training) basically means stopping all AI research now worldwide and for good measure stopping progress in GPU technology. Eliezer has been full-time employed for over 20 years to work on the issue (and Nate has been for about 15 years) and they've had enough funding to employ at least a dozen researchers and researcher-apprentices over that time to bounce ideas off of in the office.
sanxiyn 3y ago
How do you know? If we can agree something about AI, it is that we are ignorant about AI. We were similarly ignorant about recombinant DNA, so Asilomar was very cautious about it. Now we know more, we are less cautious. I still think it was good to be cautious and not to dismiss recombinant DNA concerns as "science fiction".

freedomben 3y ago

Safety and alignment stopped being about preventing AI from killing all humans a while ago. Unless you think that "don't say anything potentially offensive" is in-scope with "don't kill humans and don't take over the world by any means necessary to carry out your prompt."

hollerith 3y ago
I agree, but since neither I nor the person I responded to have used either word, what is the relevance?
- freedomben 3y ago
  Very fair point/question, I should have explicitly drawn this link because my comment was quite ambiguous and making (bad) assumptions on shared context. The relevance is IMHO this bill is largely an ossification at the government level of the safety and alignment philosophy of the big corps. I'm guessing they mainly wrote this bill. It's not the specific words "safey and alignment" that matter, it's the philosophy. If the bill were only covering AI killing machines I'd (probably) be in agreement with it, but it seems significantly more overreaching than that.
  - hollerith 3y ago
    >my comment was quite ambiguous and . . . No harm done! >If the bill were only covering AI killing machines I'd (probably) be in agreement with it, but it seems significantly more overreaching than that. Just to make sure we are on the same page: my main worry is the projects ("deployments"?) that aren't intended to kill anybody, but one of those project ends up killing billions of people anyways. It probably kills absolutely everyone. That one project might be trying to cure cancer. The only way of not incurring this risk of extinction (and of mass death) that I know of is to shut down all AI research now, which I'm guessing you would consider "overreaching". It would be great if there were a way to derive the profound benefits of continuing to do AI research without incurring the extinction risk. If you think you have a way to do that, please let me know. If I agree that your approach is promising, I'll drop everything to make sure you get a high-paying job to develop your approach. There are lots of people who would do that (and lots of high-net-worth people and organizations who would pay you the money). The Machine Intelligence Research Institute for example has a lot of money that was donated to them by cryptocurrency entrepreneurs that they've been holding on to year after year because they cannot think of any good ways to spend it to reduce extinction risk. They'd be eager to give money to anyone that can convince them that they have an approach with even a 1% probability of success.