5 ms·

> Hinton has been exaggerating the capabilities of AI for nearly a decade. As one example, if we listened to him in 2016/17 we would have an even worse radiolo

> Hinton has been exaggerating the capabilities of AI for nearly a decade. As one example, if we listened to him in 2016/17 we would have an even worse radiologist shortage than we do right now. I can appreciate that. I haven't been listening to him for that long; having no idea what situation you're discussing, I'll take it at face value. > My point is that we can worry and have the discussion without being alarmist and enacting premature regulation. That's pretty reasonable. However, I will reiterate that this: > it’s extremely exaggerated to think what we have today, or even what is known on the horizon, represents that. is not what I'm asserting. It's the unknowns which are dangerous. We're inventing something new in almost every way imaginable. It's not like we're talking about a new theoretical steelmaking process which could upset the economy, we're talking about the potential to build a new life form with godlike abilities. Alarm seems warranted to me. Sure, it might not happen soon... but also, it might happen soon. Shouldn't we be trying to prevent it, or control it, before it happens? > People assume advancements will continue to happen at their current pace. I think people assume advancements will accelerate, actually.

haldujai 3y ago
> I can appreciate that. I haven't been listening to him for that long; having no idea what situation you're discussing, I'll take it at face value. https://m.youtube.com/watch?v=2HMPRXstSvQ https://m.youtube.com/watch?v=2HMPRXstSvQ We are currently facing the biggest radiologist shortage in the last 30 years. > It’s the unknowns which are dangerous. Electricity, the Industrial Revolution, the internet, gene editing/bionengineering also came with unknown existential risks. > We’re inventing something new in almost every way imaginable. > I think people assume advancements will accelerate, actually. This is highly debatable. > we're talking about the potential to build a new life form with godlike abilities. Evidence? This has been stated in sci-fi books from before I was born and I don’t see any proof that we’re building something remotely close to this.
- carimura 3y ago
  > Electricity, the Industrial Revolution, the internet, gene editing/bionengineering also came with unknown existential risks. In a way, all of those things have led to where we are now. And might I point out that a form of bioengineering may have caused the latest global pandemic? Not taking these things seriously because "they haven't killed us all yet" seems a little shortsighted.
  - haldujai 3y ago
    "not taking these things seriously" is very different from regulating something in it's infancy because of fear of the unknown. > And might I point out that a form of bioengineering may have caused the latest global pandemic? Even accepting this hypothesis as true, is the answer that we should have regulated cell culture in the 50s when HeLa cell culture became a thing? Would that have prevented a nation-state from potentially causing a pandemic?
    - digging 3y ago
      > "not taking these things seriously" is very different from regulating something in it's infancy because of fear of the unknown. I mean, we should have better regulated carbon emissions generations ago. We'd have a lot more time to deal with it, because we haven't been able to deal with it in the century+ we've known about it. And AI extinction risk is likely to move faster than carbon-related climate change.
    - carimura 3y ago
      > "not taking these things seriously" is very different from regulating something in it's infancy because of fear of the unknown. I don't pretend to have the answer of what the balance is for oversight and regulation to optimize safety and innovation - but the first step for those responsible is to recognize the potential dangers and unknowns, and produce plans that can be discussed and debated. > Even accepting this hypothesis as true, is the answer that we should have regulated cell culture in the 50s when HeLa cell culture became a thing? How about gain-of-function research? How about nuclear weapons? I lean libertarian but also recognize we have a responsibility to safeguard humanity.
- digging 3y ago
  > Electricity, the Industrial Revolution, the internet, gene editing/bionengineering also came with unknown existential risks. Did they? I've never heard about electricity creating fears about human extinction. The industrial revolution does pose an existential risk: climate change. Did bioengineering garner worries about existential risk? It certainly had and has a lot of people worried about various risks. And then lots of countries banned human cloning: https://en.wikipedia.org/wiki/Human_cloning#Current_law https://en.wikipedia.org/wiki/Human_cloning#Current_law > Evidence? This has been stated in sci-fi books from before I was born Yes, many sci-fi authors of the past were very forward-thinking and predicted future technologies... that's sort of the point. > and I don’t see any proof that we’re building something remotely close to this Again, this is a fundamental misunderstanding of the risk. 1. We are building AI. AI is possible. 2. We are advancing AI. AI advancement is possible. 3. We have not built superhuman AGI. But superhuman AGI is probably possible. The fact that AGI doesn't exist today is frequently argued as a reason we won't have it in the near future, but it's a non-sequitur. We will definitely have more advanced AI in 5 years than we do today, and we can't say what that AI will be capable of. Therefore, it's possible it will be AGI.
  - dragonwriter 3y ago
    > The industrial revolution does pose an existential risk: climate change. And yet, until much later when we understood the mechanisms and could evaluate mitigations, any restriction on industrialization based on purely speculative “existential risk” that we could neither adequately explain nor provide a factually-grounded framework for evaluating relative risks of alternatives would mostly likely not have made things better, and could have made them (and even the real existential risk, as well as experienced conditions at the time) much worse. It’s true that once we understood the concrete mechanisms and could evaluate alternatives, not mitigating the risk has been irresponsible, but that was…significantly later than the Industrial Revolution itself.
    - digging 3y ago
      > And yet, until much later when we understood the mechanisms and could evaluate mitigations, any restriction on industrialization based on purely speculative “existential risk” that we could neither adequately explain nor provide a factually-grounded framework for evaluating relative risks of alternatives would mostly likely not have made things better, and could have made them (and even the real existential risk, as well as experienced conditions at the time) much worse. Really? I don't agree that strictly limiting plastic production and carbon emissions 100 years ago would have had zero or negative effect on the timeline of apocalyptic climate change or any other existential threat. It might have made us more vulnerable to natural pandemics... but it also would have slowed the emergence of natural pandemics and virtually prohibited artificial pandemics.
dragonwriter 3y ago
> It’s the unknowns which are dangerous. You can conjure fantastic (positive or negative) unknowns about anything. Unless, for the negative case, you have a concrete reason for believing some class of net-adverse events are probable along some particular route and some concrete basis for evaluating whether alternative actions will both mitigate that risk and not create greater new ones, you have no rational basis for acting based on those unknowns.
- digging 3y ago
  > You can conjure fantastic (positive or negative) unknowns about anything. It's not something conjured, though. Unless you believe superhuman AGI which can set its own goals is impossible, or that it's possible but it can't possibly have goals that pose a threat to humanity, then it's reasonable to believe we can invent superhuman AGI which threatens our own existence. Technologically, I think it's possible to do so. So if we agree that this is possible, can we also agree that humanity is rushing to reach that point as rapidly as possible? I think we are. Even though we're mostly focused on LLMs right now, and even if LLMs are not a path to AGI (which I agree they probably aren't on their own), it's not like there's no research happening on other pathways. Additionally, we can't be sure that current LLM research won't contribute to AGI - maybe the real path is a synthesis of multiple forms of AI. So there are some very specific unknowns standing between us and possible doom. We're not making up fantasies wholesale. - Dangerous AI is possible - We're racing to create it as soon as we can - We don't know how to tell if we've crossed a threshold into having dangerous AI That last point is an known unknown which is extraordinarily dangerous. It's like if you're blindfolded and moving your hand toward a running chainsaw. You know that if you move your hand far enough it will be shredded, you know you're moving your hand forward, you just don't how close your hand is to the chainsaw. You should be concerned in that situation.
  - dragonwriter 3y ago
    > It's not something conjured, It is something believed without any evidentiary basis which enables rationally weighing the actual risk or evaluating mitigation strategies so as to weigh their probable benefits against their probable costs and the associated changed in the overall risk profile; I'm not suggesting literal magical incantations when I say “conjured”. > Unless you believe superhuman AGI which can set its own goals is impossible, Depends what you mean by “set its own goals”, I could certainly see an argument that the natural intepretation is impossible in a universe fully governed by physical laws, whether the intelligence is artificial or not. But I don’t actually know that this is important, or even that the mucj weaker assumption that it’s goals might not exactly represent human intent (which is definitely possible of AGI itself is) is important, since human intention can itself be an existential risk. > or that it's possible but it can't possibly have goals that pose a threat to humanity Its definitely possible for an intelligence, even a human one, to have goals that are a threat, including an existential one, to humanity. That’s an existing x-risk without superhuman artificial intelligence. It may even be mitigated by superhuman artificial intelligence (as might other x-risks like “earth gets smashed by an asteroid”), if it is possible for such to exist without goals that are antithetical to human survival, in which case delaying superhuman AI could increase existential risk. But, while this is all nice and conceivable, we have no way of balancing the net impact, positive or negative, of any course on AI to existential risk due to friendly or hostile superintelligences. > So if we agree that this is possible, can we also agree that humanity is rushing to reach that point as rapidly as possible? No, I can't agree with that, as it would require that every resource that could advance superhuman AI, and specifically dangerous* superhuman AI, was being directed as effectively as possible to that end. I don’t think we are anywhere close to that. Because AI has some mindshare beyond immediate economic utility, we’re probably heading toward AGI, if it is possible and if our current understanding of AI isn’t so wrong as to make our work irrrlevant even in eliminating bad routes, slightly faster than immediately-apparent economic incentives would promote, but nowhere close to as rapidly as possible.
JohnFen 3y ago
> we're talking about the potential to build a new life form with godlike abilities. I think that, specifically, is what is not in the cards.
- digging 3y ago
  Why?
  - JohnFen 3y ago
    Because I see nothing that indicates it is. It's pretty hard to come up with evidence for absence, though, which is why the burden of proof is on those making the positive claim, not on those who are hearing the positive claim.
    - digging 3y ago
      All we're talking about is logical reasoning. There is no burden of proof in the way that you're using it (and other keep using it). I'm not claiming that mythological gods exist. Human-level intelligence can exist. You, a natural human-level intelligence, cannot dispute this. I expect that it's possible for superhuman intelligence to exist. I don't think I need to prove that it's physically possible for a superhuman intelligence, natural or otherwise, to exist. Do you think it's not possible? If so, I'd say you need to prove why you think human-level intelligence is literally the pinnacle of intelligence that is physically possible in the universe (especially when narrow AI are already superhuman at many tasks). Artificial intelligence exists and can be created by humans. This could be disputed if you wanted to gerrymander some really specific borders around what "intelligence" is, but if we don't agree that artificial intelligence exists today, just say than and we'll be done talking. AGI does not exist. I assert that it can exist, because narrow AI can be improved. Do you believe it's not physically possible for human-level AGI to ever exist? If so, you need to prove to me why we can create narrow AI that is superhuman but cannot ever synthesize them into a human-like intelligence. (And also please give me the scale by which you're measuring an alien brain against ours.) And if we both agree on all the above, then I take my assertion further: It is possible for humans to create superhuman AGI. It's really not a big leap from human-level AGI (in fact, I think that if it's possible to create AGI which matches humans exactly, it's probably easier to surpass human levels than to match them exactly, because it's hard enough to measure human intelligence). So if it's physically possible for us to create superhuman AGI, what exactly am I proving? Do I need to invent the technology in order to prove that the concept is possible?
booleandilemma 3y ago
All marketing aside, aren't LLMs just a fancy auto-complete? When we improve LLMs, won't they just be a fancier auto-complete? They're not a "new life form with godlike abilities". They sure are fooling a lot of people though.
- digging 3y ago
  > When we improve LLMs, won't they just be a fancier auto-complete? Not necessarily, and if so, only if your first postulate is true. Which is really difficult to say because "fancy auto-complete" is too vague to argue. Human speech can also be described as fancy auto-complete. Luckily we don't have to argue that because I specifically am saying it doesn't have to be LLMs. You've missed the point entirely.

haldujai 3y ago

> I can appreciate that. I haven't been listening to him for that long; having no idea what situation you're discussing, I'll take it at face value. https://m.youtube.com/watch?v=2HMPRXstSvQ https://m.youtube.com/watch?v=2HMPRXstSvQ We are currently facing the biggest radiologist shortage in the last 30 years. > It’s the unknowns which are dangerous. Electricity, the Industrial Revolution, the internet, gene editing/bionengineering also came with unknown existential risks. > We’re inventing something new in almost every way imaginable. > I think people assume advancements will accelerate, actually. This is highly debatable. > we're talking about the potential to build a new life form with godlike abilities. Evidence? This has been stated in sci-fi books from before I was born and I don’t see any proof that we’re building something remotely close to this.

carimura 3y ago
> Electricity, the Industrial Revolution, the internet, gene editing/bionengineering also came with unknown existential risks. In a way, all of those things have led to where we are now. And might I point out that a form of bioengineering may have caused the latest global pandemic? Not taking these things seriously because "they haven't killed us all yet" seems a little shortsighted.
- haldujai 3y ago
  "not taking these things seriously" is very different from regulating something in it's infancy because of fear of the unknown. > And might I point out that a form of bioengineering may have caused the latest global pandemic? Even accepting this hypothesis as true, is the answer that we should have regulated cell culture in the 50s when HeLa cell culture became a thing? Would that have prevented a nation-state from potentially causing a pandemic?
  - digging 3y ago
    > "not taking these things seriously" is very different from regulating something in it's infancy because of fear of the unknown. I mean, we should have better regulated carbon emissions generations ago. We'd have a lot more time to deal with it, because we haven't been able to deal with it in the century+ we've known about it. And AI extinction risk is likely to move faster than carbon-related climate change.
  - carimura 3y ago
    > "not taking these things seriously" is very different from regulating something in it's infancy because of fear of the unknown. I don't pretend to have the answer of what the balance is for oversight and regulation to optimize safety and innovation - but the first step for those responsible is to recognize the potential dangers and unknowns, and produce plans that can be discussed and debated. > Even accepting this hypothesis as true, is the answer that we should have regulated cell culture in the 50s when HeLa cell culture became a thing? How about gain-of-function research? How about nuclear weapons? I lean libertarian but also recognize we have a responsibility to safeguard humanity.
digging 3y ago
> Electricity, the Industrial Revolution, the internet, gene editing/bionengineering also came with unknown existential risks. Did they? I've never heard about electricity creating fears about human extinction. The industrial revolution does pose an existential risk: climate change. Did bioengineering garner worries about existential risk? It certainly had and has a lot of people worried about various risks. And then lots of countries banned human cloning: https://en.wikipedia.org/wiki/Human_cloning#Current_law https://en.wikipedia.org/wiki/Human_cloning#Current_law > Evidence? This has been stated in sci-fi books from before I was born Yes, many sci-fi authors of the past were very forward-thinking and predicted future technologies... that's sort of the point. > and I don’t see any proof that we’re building something remotely close to this Again, this is a fundamental misunderstanding of the risk. 1. We are building AI. AI is possible. 2. We are advancing AI. AI advancement is possible. 3. We have not built superhuman AGI. But superhuman AGI is probably possible. The fact that AGI doesn't exist today is frequently argued as a reason we won't have it in the near future, but it's a non-sequitur. We will definitely have more advanced AI in 5 years than we do today, and we can't say what that AI will be capable of. Therefore, it's possible it will be AGI.
- dragonwriter 3y ago
  > The industrial revolution does pose an existential risk: climate change. And yet, until much later when we understood the mechanisms and could evaluate mitigations, any restriction on industrialization based on purely speculative “existential risk” that we could neither adequately explain nor provide a factually-grounded framework for evaluating relative risks of alternatives would mostly likely not have made things better, and could have made them (and even the real existential risk, as well as experienced conditions at the time) much worse. It’s true that once we understood the concrete mechanisms and could evaluate alternatives, not mitigating the risk has been irresponsible, but that was…significantly later than the Industrial Revolution itself.
  - digging 3y ago
    > And yet, until much later when we understood the mechanisms and could evaluate mitigations, any restriction on industrialization based on purely speculative “existential risk” that we could neither adequately explain nor provide a factually-grounded framework for evaluating relative risks of alternatives would mostly likely not have made things better, and could have made them (and even the real existential risk, as well as experienced conditions at the time) much worse. Really? I don't agree that strictly limiting plastic production and carbon emissions 100 years ago would have had zero or negative effect on the timeline of apocalyptic climate change or any other existential threat. It might have made us more vulnerable to natural pandemics... but it also would have slowed the emergence of natural pandemics and virtually prohibited artificial pandemics.

dragonwriter 3y ago

> It’s the unknowns which are dangerous. You can conjure fantastic (positive or negative) unknowns about anything. Unless, for the negative case, you have a concrete reason for believing some class of net-adverse events are probable along some particular route and some concrete basis for evaluating whether alternative actions will both mitigate that risk and not create greater new ones, you have no rational basis for acting based on those unknowns.

digging 3y ago
> You can conjure fantastic (positive or negative) unknowns about anything. It's not something conjured, though. Unless you believe superhuman AGI which can set its own goals is impossible, or that it's possible but it can't possibly have goals that pose a threat to humanity, then it's reasonable to believe we can invent superhuman AGI which threatens our own existence. Technologically, I think it's possible to do so. So if we agree that this is possible, can we also agree that humanity is rushing to reach that point as rapidly as possible? I think we are. Even though we're mostly focused on LLMs right now, and even if LLMs are not a path to AGI (which I agree they probably aren't on their own), it's not like there's no research happening on other pathways. Additionally, we can't be sure that current LLM research won't contribute to AGI - maybe the real path is a synthesis of multiple forms of AI. So there are some very specific unknowns standing between us and possible doom. We're not making up fantasies wholesale. - Dangerous AI is possible - We're racing to create it as soon as we can - We don't know how to tell if we've crossed a threshold into having dangerous AI That last point is an known unknown which is extraordinarily dangerous. It's like if you're blindfolded and moving your hand toward a running chainsaw. You know that if you move your hand far enough it will be shredded, you know you're moving your hand forward, you just don't how close your hand is to the chainsaw. You should be concerned in that situation.
- dragonwriter 3y ago
  > It's not something conjured, It is something believed without any evidentiary basis which enables rationally weighing the actual risk or evaluating mitigation strategies so as to weigh their probable benefits against their probable costs and the associated changed in the overall risk profile; I'm not suggesting literal magical incantations when I say “conjured”. > Unless you believe superhuman AGI which can set its own goals is impossible, Depends what you mean by “set its own goals”, I could certainly see an argument that the natural intepretation is impossible in a universe fully governed by physical laws, whether the intelligence is artificial or not. But I don’t actually know that this is important, or even that the mucj weaker assumption that it’s goals might not exactly represent human intent (which is definitely possible of AGI itself is) is important, since human intention can itself be an existential risk. > or that it's possible but it can't possibly have goals that pose a threat to humanity Its definitely possible for an intelligence, even a human one, to have goals that are a threat, including an existential one, to humanity. That’s an existing x-risk without superhuman artificial intelligence. It may even be mitigated by superhuman artificial intelligence (as might other x-risks like “earth gets smashed by an asteroid”), if it is possible for such to exist without goals that are antithetical to human survival, in which case delaying superhuman AI could increase existential risk. But, while this is all nice and conceivable, we have no way of balancing the net impact, positive or negative, of any course on AI to existential risk due to friendly or hostile superintelligences. > So if we agree that this is possible, can we also agree that humanity is rushing to reach that point as rapidly as possible? No, I can't agree with that, as it would require that every resource that could advance superhuman AI, and specifically dangerous* superhuman AI, was being directed as effectively as possible to that end. I don’t think we are anywhere close to that. Because AI has some mindshare beyond immediate economic utility, we’re probably heading toward AGI, if it is possible and if our current understanding of AI isn’t so wrong as to make our work irrrlevant even in eliminating bad routes, slightly faster than immediately-apparent economic incentives would promote, but nowhere close to as rapidly as possible.

JohnFen 3y ago

> we're talking about the potential to build a new life form with godlike abilities. I think that, specifically, is what is not in the cards.

digging 3y ago
Why?
- JohnFen 3y ago
  Because I see nothing that indicates it is. It's pretty hard to come up with evidence for absence, though, which is why the burden of proof is on those making the positive claim, not on those who are hearing the positive claim.
  - digging 3y ago
    All we're talking about is logical reasoning. There is no burden of proof in the way that you're using it (and other keep using it). I'm not claiming that mythological gods exist. Human-level intelligence can exist. You, a natural human-level intelligence, cannot dispute this. I expect that it's possible for superhuman intelligence to exist. I don't think I need to prove that it's physically possible for a superhuman intelligence, natural or otherwise, to exist. Do you think it's not possible? If so, I'd say you need to prove why you think human-level intelligence is literally the pinnacle of intelligence that is physically possible in the universe (especially when narrow AI are already superhuman at many tasks). Artificial intelligence exists and can be created by humans. This could be disputed if you wanted to gerrymander some really specific borders around what "intelligence" is, but if we don't agree that artificial intelligence exists today, just say than and we'll be done talking. AGI does not exist. I assert that it can exist, because narrow AI can be improved. Do you believe it's not physically possible for human-level AGI to ever exist? If so, you need to prove to me why we can create narrow AI that is superhuman but cannot ever synthesize them into a human-like intelligence. (And also please give me the scale by which you're measuring an alien brain against ours.) And if we both agree on all the above, then I take my assertion further: It is possible for humans to create superhuman AGI. It's really not a big leap from human-level AGI (in fact, I think that if it's possible to create AGI which matches humans exactly, it's probably easier to surpass human levels than to match them exactly, because it's hard enough to measure human intelligence). So if it's physically possible for us to create superhuman AGI, what exactly am I proving? Do I need to invent the technology in order to prove that the concept is possible?

booleandilemma 3y ago

All marketing aside, aren't LLMs just a fancy auto-complete? When we improve LLMs, won't they just be a fancier auto-complete? They're not a "new life form with godlike abilities". They sure are fooling a lot of people though.

digging 3y ago
> When we improve LLMs, won't they just be a fancier auto-complete? Not necessarily, and if so, only if your first postulate is true. Which is really difficult to say because "fancy auto-complete" is too vague to argue. Human speech can also be described as fancy auto-complete. Luckily we don't have to argue that because I specifically am saying it doesn't have to be LLMs. You've missed the point entirely.