6 ms·

I think you're looking at this through the wrong lens. There are a variety of issues that the Less Wrong community gets ironically wrong. But the agenda here i

I think you're looking at this through the wrong lens. There are a variety of issues that the Less Wrong community gets ironically wrong. But the agenda here is very simple: make sure AI doesn't kill us. They believe that earnestly. That is the end goal. Everything is in service of that not the other way around. And with how quickly AI is progressing I'm pretty onboard with that.

gcr 4y ago
Let's flip this around a little. "The agenda of Christianity here is very simple: make sure the devil doesn't kill us. Christians believe that earnestly. That's the end goal. Everything is in service of that and not the other way around. And with how quickly the world is turning to shit, I'm pretty onboard with that." There's a blind devotion to a group's chosen x-risk event, alongside assigning infinite utility to that one goal at the exclusion of all other useful goals. That's an important hallmark of fundamentalism. I sincerely don't think it's a good thing.
- altacc 4y ago
  Except gods and devils have never and will never exist but AI might. Although I think it incredibly unlikely that AI will ever be in a position to create a human exterminating Skynet totality and the discussions should be more about how humanity is affected by machine learning, as it has been by previous technological advances. The LessWrong community try to address bias but seem quite biased here by sci-fi writers.
  - gcr 4y ago
    Christians genuinely have the faith that gods and devils exist. It's a genuine belief, they're convinced by whatever evidence they see, even though outsiders don't necessarily understand. Maybe they're not 100% there, but that's okay - they plan for this low probability but important outcome by going to church, helping their neighbor, performing outreach, converting nonbelievers, discriminating against queers and bitching about homeless people, pretending to read bible, etc. You have faith that AGI will both exist and have similar power. That's your genuine belief, you've been convinced by the evidence that you see. Maybe you're not 100% there but that's okay - you plan for this low-probability but important event by reading up on machine learning, contributing to your community discussions, studying alignment, performing outreach, converting folks who don't believe in AGI, pretending to read Norvig textbook, etc. My point is that these two beliefs are related.
    - allturtles 4y ago
      Your analogy keeps getting more strained, IMO. Should we believe that unicorns and dark matter are equally plausible? Neither has been directly observed, after all. I'm not sure this line of argument is getting you anywhere. It would be more productive, I think, to directly engage with the argument and say why you think concern about AGI is implausible.
      - ElevenLathe 4y ago
        > Should we believe that unicorns and dark matter are equally plausible? Yes. > Neither has been directly observed, after all. Exactly, they are both just ideas people have that help make sense of the world as they see it.
    - dwohnitmok 4y ago
      > You have faith that X. That's your genuine belief, you've been convinced by the evidence that you see. Maybe you're not 100% there but that's okay - you plan for this low-probability but important event by doing Y. Is there any low-probability but important event that doesn't follow this template?
- dwohnitmok 4y ago
  > one goal at the exclusion of all other useful goals. I don't believe this to be true. Nor do I believe the AI research community broadly believes this. Other goals can certainly coexist. But when the median AI researcher believes there's a 10% (!) chance that what they're working on will destroy the world, and the pace of that work is only quickening, that seems like a big issue to worry about. To quote casebash elsewhere https://news.ycombinator.com/item?id=35195102 https://news.ycombinator.com/item?id=35195102 > Just because people decide to focus their time on one issue (say AI), doesn't mean that they don't appreciate people working on other issues.
  - AnIdiotOnTheNet 4y ago
    > But when the median AI researcher believes there's a 10% (!) chance that what they're working on will destroy the world, and the pace of that work is only quickening, that seems like a big issue to worry about. Yet they will not quit their job. What that suggests to me is that they don't really believe it, otherwise they would sacrifice their paycheck to stop it. Seems to me that what they really believe is that they should get more attention and more funding.
    - dwohnitmok 4y ago
      > Yet they will not quit their job. What that suggests to me is that they don't really believe it, otherwise they would sacrifice their paycheck to stop it. I've talked to quite a few researchers. They generally do earnestly believe that AI has huge damage potential. The answers to why they keep developing AI falls in several different buckets (to which I have varying degrees of agreement) 1. We're in a prisoner's dilemma. If everyone could unilaterally agree to slow down AI research I'd be happy to quit, but until that happens if I don't do it someone else will and with even fewer scruples. 2. The only way to develop AGI safely is to push things to the brink. It's impossible to make progress on how to make sure that AGI is safe is to almost make AGI because not until then do you have a clear enough picture of the exact form AGI will take to do productive safety research. 3. Slowing down AI research has its own risks. If you slow down AI research, but other fields of technology continue to progress, you can end up in a scenario where everyone's personal computer is capable not only of running models, but of doing cutting-edge ML training. Right now to coordinate on developing AGI safely, there are maybe ~10 top teams with the infrastructure possible to do the training (not just running the model!) needed to advance the state of the art you have to get cooperation among. Hard, but not impossible. If technology progresses to the point where there tens of thousands of teams all capable of meaningfully advancing the state of the art then coordination is absolutely impossible. 4. If AI can, then AI should replace humans entirely. (the most chilling response I've heard. Luckily few in number, but it shows up even on HN)
- allturtles 4y ago
  That's a very strange analogy. What about Christian doctrine includes "making sure the devil doesn't kill us"? "Making sure the devil doesn't tempt us into evil" maybe? But Christianity kind of assumes we will fall into temptation and be forgiven anyway.
matthewdgreen 4y ago
As other commenters point out, the end is not the same as the means. Simple agendas, when followed with sufficient vigor, can sometimes cause enormous amounts of harm. I’ve recently encountered a number of posts from this community that are filled with despair about the speed at which AI is progressing, and predict the end of humanity on a short timeline. Some are written by very well-known figures in the community. Other adjacent figures have mooted the idea that a police state might be the only way to control certain technologies. It’s reasonable to ask whether existential terror might lead some in this community to make rash decisions. It’s also very concerning because these existential fears can distract people from thinking about other, but perhaps more relevant concerns. Imagine that AI doesn’t resist human orders, but the people giving the orders use it to radically restructure our society in ways that make our current economic system incompatible with human flourishing: is this community concerned about that (perhaps very realistic) outcome, or do their interests end when AI is compliant with its masters?
- deleted 4y ago
  [deleted]
- dwohnitmok 4y ago
  > is this community concerned about that (perhaps very realistic) outcome, or do their interests end when AI is compliant with its masters? The short of it is yes. As far as I'm aware the long-term hope in the Less Wrong community or more generally the AI safety community is that AI becomes available for everyone and under the control of everyone. But the first, paramount step is to make sure it doesn't destroy everything and recognize that such a guarantee doesn't come for free, but rather requires an enormous amount of work and coordination.
  - wahnfrieden 4y ago
    my original question was what realm of solutions are they advocating I don't take lightly to suggestions that we enact police states, highly authoritarian government controls, and other state or capital extremes that I've seen that community / the alignment folks advocate for as you can find when you look past their concern and fears

gcr 4y ago

Let's flip this around a little. "The agenda of Christianity here is very simple: make sure the devil doesn't kill us. Christians believe that earnestly. That's the end goal. Everything is in service of that and not the other way around. And with how quickly the world is turning to shit, I'm pretty onboard with that." There's a blind devotion to a group's chosen x-risk event, alongside assigning infinite utility to that one goal at the exclusion of all other useful goals. That's an important hallmark of fundamentalism. I sincerely don't think it's a good thing.

altacc 4y ago
Except gods and devils have never and will never exist but AI might. Although I think it incredibly unlikely that AI will ever be in a position to create a human exterminating Skynet totality and the discussions should be more about how humanity is affected by machine learning, as it has been by previous technological advances. The LessWrong community try to address bias but seem quite biased here by sci-fi writers.
- gcr 4y ago
  Christians genuinely have the faith that gods and devils exist. It's a genuine belief, they're convinced by whatever evidence they see, even though outsiders don't necessarily understand. Maybe they're not 100% there, but that's okay - they plan for this low probability but important outcome by going to church, helping their neighbor, performing outreach, converting nonbelievers, discriminating against queers and bitching about homeless people, pretending to read bible, etc. You have faith that AGI will both exist and have similar power. That's your genuine belief, you've been convinced by the evidence that you see. Maybe you're not 100% there but that's okay - you plan for this low-probability but important event by reading up on machine learning, contributing to your community discussions, studying alignment, performing outreach, converting folks who don't believe in AGI, pretending to read Norvig textbook, etc. My point is that these two beliefs are related.
  - allturtles 4y ago
    Your analogy keeps getting more strained, IMO. Should we believe that unicorns and dark matter are equally plausible? Neither has been directly observed, after all. I'm not sure this line of argument is getting you anywhere. It would be more productive, I think, to directly engage with the argument and say why you think concern about AGI is implausible.
    - ElevenLathe 4y ago
      > Should we believe that unicorns and dark matter are equally plausible? Yes. > Neither has been directly observed, after all. Exactly, they are both just ideas people have that help make sense of the world as they see it.
  - dwohnitmok 4y ago
    > You have faith that X. That's your genuine belief, you've been convinced by the evidence that you see. Maybe you're not 100% there but that's okay - you plan for this low-probability but important event by doing Y. Is there any low-probability but important event that doesn't follow this template?
dwohnitmok 4y ago
> one goal at the exclusion of all other useful goals. I don't believe this to be true. Nor do I believe the AI research community broadly believes this. Other goals can certainly coexist. But when the median AI researcher believes there's a 10% (!) chance that what they're working on will destroy the world, and the pace of that work is only quickening, that seems like a big issue to worry about. To quote casebash elsewhere https://news.ycombinator.com/item?id=35195102 https://news.ycombinator.com/item?id=35195102 > Just because people decide to focus their time on one issue (say AI), doesn't mean that they don't appreciate people working on other issues.
- AnIdiotOnTheNet 4y ago
  > But when the median AI researcher believes there's a 10% (!) chance that what they're working on will destroy the world, and the pace of that work is only quickening, that seems like a big issue to worry about. Yet they will not quit their job. What that suggests to me is that they don't really believe it, otherwise they would sacrifice their paycheck to stop it. Seems to me that what they really believe is that they should get more attention and more funding.
  - dwohnitmok 4y ago
    > Yet they will not quit their job. What that suggests to me is that they don't really believe it, otherwise they would sacrifice their paycheck to stop it. I've talked to quite a few researchers. They generally do earnestly believe that AI has huge damage potential. The answers to why they keep developing AI falls in several different buckets (to which I have varying degrees of agreement) 1. We're in a prisoner's dilemma. If everyone could unilaterally agree to slow down AI research I'd be happy to quit, but until that happens if I don't do it someone else will and with even fewer scruples. 2. The only way to develop AGI safely is to push things to the brink. It's impossible to make progress on how to make sure that AGI is safe is to almost make AGI because not until then do you have a clear enough picture of the exact form AGI will take to do productive safety research. 3. Slowing down AI research has its own risks. If you slow down AI research, but other fields of technology continue to progress, you can end up in a scenario where everyone's personal computer is capable not only of running models, but of doing cutting-edge ML training. Right now to coordinate on developing AGI safely, there are maybe ~10 top teams with the infrastructure possible to do the training (not just running the model!) needed to advance the state of the art you have to get cooperation among. Hard, but not impossible. If technology progresses to the point where there tens of thousands of teams all capable of meaningfully advancing the state of the art then coordination is absolutely impossible. 4. If AI can, then AI should replace humans entirely. (the most chilling response I've heard. Luckily few in number, but it shows up even on HN)
allturtles 4y ago
That's a very strange analogy. What about Christian doctrine includes "making sure the devil doesn't kill us"? "Making sure the devil doesn't tempt us into evil" maybe? But Christianity kind of assumes we will fall into temptation and be forgiven anyway.

matthewdgreen 4y ago

As other commenters point out, the end is not the same as the means. Simple agendas, when followed with sufficient vigor, can sometimes cause enormous amounts of harm. I’ve recently encountered a number of posts from this community that are filled with despair about the speed at which AI is progressing, and predict the end of humanity on a short timeline. Some are written by very well-known figures in the community. Other adjacent figures have mooted the idea that a police state might be the only way to control certain technologies. It’s reasonable to ask whether existential terror might lead some in this community to make rash decisions. It’s also very concerning because these existential fears can distract people from thinking about other, but perhaps more relevant concerns. Imagine that AI doesn’t resist human orders, but the people giving the orders use it to radically restructure our society in ways that make our current economic system incompatible with human flourishing: is this community concerned about that (perhaps very realistic) outcome, or do their interests end when AI is compliant with its masters?

deleted 4y ago
[deleted]
dwohnitmok 4y ago
> is this community concerned about that (perhaps very realistic) outcome, or do their interests end when AI is compliant with its masters? The short of it is yes. As far as I'm aware the long-term hope in the Less Wrong community or more generally the AI safety community is that AI becomes available for everyone and under the control of everyone. But the first, paramount step is to make sure it doesn't destroy everything and recognize that such a guarantee doesn't come for free, but rather requires an enormous amount of work and coordination.
- wahnfrieden 4y ago
  my original question was what realm of solutions are they advocating I don't take lightly to suggestions that we enact police states, highly authoritarian government controls, and other state or capital extremes that I've seen that community / the alignment folks advocate for as you can find when you look past their concern and fears