10 ms·

What is the probability of an asteroid hitting the earth and wiping out humanity? I've seen numbers like 0.000001%, yet NASA is spending real money on mitigatin

by thurn 3y ago

What is the probability of an asteroid hitting the earth and wiping out humanity? I've seen numbers like 0.000001%, yet NASA is spending real money on mitigating this risk. Is the risk of fast takeoff AI dramatically lower than that?

jvanderbot 3y ago
Asteroid impacts have been demonstrated over and over to have obvious and widespread consequences. I'd liken AI to alien invasion. We can imagine bad scenarios, but have absolutely zero data, experience, or sound theories predicting anything. It's all scifi.
- MVissers 3y ago
  We have only our history, where we dominated and decimated other close species (Homo Neanderthals) thanks to our intelligence, and literally caused the next extinction event on this planet where >95% of all species already disappeared. On top of that, think about how we treat animals where we have billions of intelligent animals (cows, pigs) that we breed to butcher and eat. Just because they are tasty, and we tell ourselves they are less worthy because of their lesser intelligence and consciousness. I sure hope that AI doesn't behave like us. I have no idea what will happen and what the odds are, but I do think it's the most dangerous experiment we have ever run.
  - hammyhavoc 3y ago
    Our own history does not include any AGI. Likening AI to predators in an ecosystem is bizarre and irrelevant.
    - haswell 3y ago
      This is not about likening AI to predators, it’s about the fact that AI is and will be created by and in the image of the most dominant predator in the history of biological life. To whatever degree AI is embedded with the properties of our species, it carries the risks of repeating our worst behaviors, and has the potential to do so far more potently than we ever could due to the absence of biological constraints.
      - hammyhavoc 3y ago
        Will it? So when this hypothetical AGI with sentience emerges, it's going to think like a human being? Why would it with so much knowledge? Most negative aspects of humanity come from a lack of knowledge.
        haswell 3y ago
        No, I’m not claiming it will think like a human being, but that human thinking (despite the blind spots we know we have) is what we’ll try to replicate, and this will lead us to create something dangerous. What you’re hinting at presupposes that an AGI will interpret knowledge in a way that is compatible with human values, and that we understand human values well enough to codify them into whatever it is that we manage to build. I’m sure we’ll try. But it is this assumption that is at the center of the issue IMO, because assuming the machines will think like us is to assume we understand how we think. What is far more likely is that it will appear to have human properties because we baked the appearance of human-ness into the code (see LLMs), but like the hallucinations and “lies” these LLMs are happy to spit out, an apparently human AGI will just get some things fundamentally wrong. And when getting things wrong can have consequences in the physical world, they’re no longer just relatively harmless hallucinations. Imagine an AGI with the temperament of Microsoft Tay.
        hammyhavoc 3y ago
        IMO, we are more likely to stumble across a completely new form of intelligence than we are to successfully recreate something akin to human intelligence. However, I think actual brain cells in a petri dish learning and playing Pong is a far greater concern than a hypothetical AGI. Well, they don't hallucinate or lie, that's a human-centric concept that we've misapplied. An LLM is just a next word prediction engine. It's more accurately described as incorrect in its output based on the data it has available to it, skewing the probability calculation. It didn't lie nor hallucinate. Lying assumes intent and planning, it is incapable of either, it's an LLM. But calling it "wrong" kills the hype cycle, and you're on HN, a Y Combinator platform. A lot of folks here are invested in LLMs or selling something utilizing them as it's hot right now.
    - lukevp 3y ago
      I think you missed their point. From a cow’s perspective, they are imprisoned, enslaved, and killed for their flesh. They live a horrific existence and probably would think we are all Nazis if they had enough reasoning to realize what was happening to them and their families. It doesn’t really matter to them that a human out there is consuming them and has some higher purpose and goals (besides just eating them to survive). An AGI could have any number of goals that are not related to dominating, enslaving and slaughtering us, where it could still be easier and more convenient to do so, so they do it. The side effects on the world caused by intelligent life with agency are unpredictable. If they become smarter than us, and have robotics, what if they decide animal life isn’t necessary anymore? Or that our pollution causes solar energy capture to be affected, and the quickest way to stop it is to kill everyone? Maybe it’s better to demolish a city for a solar farm because it’s already got so much concrete and electrical infrastructure and is cheaper/faster than building new. Who knows what their goals will be and how they’ll seek to accomplish them? All we know is that we don’t have much consideration for the “lesser” beings on our planet, and how things are from their perspective.
      - hammyhavoc 3y ago
        I understood the point just fine, but the hypothetical AGI does not exist, and any hypothetical AGI is unlike anything else in history. An AGI has never existed before in history. Why would a hypothetical AGI behave anything like a human? Complete sci-fi.
        DennisP 3y ago
        An AI that doesn't behave at all like a human could well be worse. Humans tend to place at least some value on human life; an AI might just see humans as weird chemical reactions that are less useful than whatever else the AI wants to do with the raw material.
        hammyhavoc 3y ago
        This is assuming a hypothetical AGI even understands what a human being is. If it has no body, can it ever understand what the world is? Even if you put it into a robot, it cannot understand mortality because it is effectively immortal and can be brought back online at any time. Erasure is mitigated by duplication of the data set, ensuring it can always repair itself. In the same breath, if a hypothetical omniscient ever-present god were to exist, how would we as physical lifeforms ever truly understand that? We can know of it, but we cannot truly understand it because we haven't had that experience of another plain of existence. All compelling ideas, but mercifully well within the realms of sci-fi.
    - broast 3y ago
      Our cells built our bodies and brain as an AI to navigate the macro realm!
  - Lacerda69 3y ago
    >we dominated and decimated other close species (Homo Neanderthals) thanks to our intelligence it's a nice homo sapiens supremacy narrative but the truth is much more complicated. we existed at least 4/5 thousand years together and a lot of mixing and mingling has been going on.
    - haswell 3y ago
      > we existed at least 4/5 thousand years together and a lot of mixing and mingling has been going on. This sounds strikingly similar to how I imagine our first encounters with advanced AI will unfold.
  - jvanderbot 3y ago
    It's anthropomorphic to the extreme to assume that computer programs will assume all of our evolutionary, biological characteristics
- sgregnt 3y ago
  Why spend resources now? We can wait 1k years and utilize future technology to solve the asteroid problem.
jl6 3y ago
We don't know the probability of either, because we haven't mapped every asteroid, and we don't know what AI breakthroughs are reachable.
AbrahamParangi 3y ago
What is the risk of Santa going rogue and committing violent crimes instead of delivering presents? Given that Santa has supernatural powers, if he were to be misaligned he could cause untold damage. Yes, it hasn't happened before but absence of evidence is not evidence of absence.
- ctoth 3y ago
  Well, given that we have an entire industry devoted to instantiating Santa, I actually think this is a legit worry. Consider the amount of plastic toys and other garbage which Santa creates every year. I have even heard tell that Santa has put the wrong types of chemicals in toys, that have killed kids. Of course, Santa here is just a toy company. Actually toy companies never existed before about 1700 so we don't have anything to worry from the current ones. This is just about as incoherent as this entire comment section. I thought I'd play along.
  - AbrahamParangi 3y ago
    This is reasoning by analogy and assuming that the analogies are at all valid. The null hypothesis is that they are not.
dmbche 3y ago
Well - the probability is 100% that we'll be hit by one of those asteroids, we just don't know when. When looking at the past data, for these kinds of events, I believe we are "overdue", in the sense that on average we should have had one in the last 150 years I believe. So working on a solution to this problem makes sense - even if we spend billions solving this issue, it's not lost, because at a point it will be needed. As for AI becoming an issue, it's a sci-fi idea, nothing more. It's much to vague - how is it reliably becoming dangerous? What are we actually looking for as signs of danger?
- jowdones 3y ago
  Finally some common sense. Just for entertainment, assuming the SciFi ridiculousness becomes true and I'm a laptop that becomes self-aware and decide to strangle you. Upps... I got no hands. Upps... it's more of a nightmare of being trapped paralyzed in a life support machine that can always without my AI control, be unplugged from the power socket and it's game over.
  - Al-Khwarizmi 3y ago
    Except for the fact that if it could use language and has an Internet connection, it could manipulate people into doing things. People who have hands. And it could also make copies of itself, or convince people to do so, so that one copy being deactivated by unplugging a socket is not a big deal.
    - dmbche 3y ago
      Well - how much can one do without ever meeting in person? Or even - if we fear AI's being tools of radicalisation, I think it's pretty far away from the skynet apocalypse most people are thinking of. Not writing stuff like the unabomber manifesto to get people to be terrorists. If you're afraid of radicalisation, I'd argue that the US governement is much more dangerous and is just as much a black box with no one at the wheel.
m3kw9 3y ago
When there are lots of data like asteroids trajectories it’s easier to estimate probability, AI has too many unknowns to estimate. Also time frame matters of the probability matters too
tim333 3y ago
The chance of AI improving itself in the next century is about 100%. "Fast takeoff" depends on how you define fast.

by thurn 3y ago

jvanderbot 3y ago

Asteroid impacts have been demonstrated over and over to have obvious and widespread consequences. I'd liken AI to alien invasion. We can imagine bad scenarios, but have absolutely zero data, experience, or sound theories predicting anything. It's all scifi.

MVissers 3y ago
We have only our history, where we dominated and decimated other close species (Homo Neanderthals) thanks to our intelligence, and literally caused the next extinction event on this planet where >95% of all species already disappeared. On top of that, think about how we treat animals where we have billions of intelligent animals (cows, pigs) that we breed to butcher and eat. Just because they are tasty, and we tell ourselves they are less worthy because of their lesser intelligence and consciousness. I sure hope that AI doesn't behave like us. I have no idea what will happen and what the odds are, but I do think it's the most dangerous experiment we have ever run.
- hammyhavoc 3y ago
  Our own history does not include any AGI. Likening AI to predators in an ecosystem is bizarre and irrelevant.
  - haswell 3y ago
    This is not about likening AI to predators, it’s about the fact that AI is and will be created by and in the image of the most dominant predator in the history of biological life. To whatever degree AI is embedded with the properties of our species, it carries the risks of repeating our worst behaviors, and has the potential to do so far more potently than we ever could due to the absence of biological constraints.
    - hammyhavoc 3y ago
      Will it? So when this hypothetical AGI with sentience emerges, it's going to think like a human being? Why would it with so much knowledge? Most negative aspects of humanity come from a lack of knowledge.
      - haswell 3y ago
        No, I’m not claiming it will think like a human being, but that human thinking (despite the blind spots we know we have) is what we’ll try to replicate, and this will lead us to create something dangerous. What you’re hinting at presupposes that an AGI will interpret knowledge in a way that is compatible with human values, and that we understand human values well enough to codify them into whatever it is that we manage to build. I’m sure we’ll try. But it is this assumption that is at the center of the issue IMO, because assuming the machines will think like us is to assume we understand how we think. What is far more likely is that it will appear to have human properties because we baked the appearance of human-ness into the code (see LLMs), but like the hallucinations and “lies” these LLMs are happy to spit out, an apparently human AGI will just get some things fundamentally wrong. And when getting things wrong can have consequences in the physical world, they’re no longer just relatively harmless hallucinations. Imagine an AGI with the temperament of Microsoft Tay.
        hammyhavoc 3y ago
        IMO, we are more likely to stumble across a completely new form of intelligence than we are to successfully recreate something akin to human intelligence. However, I think actual brain cells in a petri dish learning and playing Pong is a far greater concern than a hypothetical AGI. Well, they don't hallucinate or lie, that's a human-centric concept that we've misapplied. An LLM is just a next word prediction engine. It's more accurately described as incorrect in its output based on the data it has available to it, skewing the probability calculation. It didn't lie nor hallucinate. Lying assumes intent and planning, it is incapable of either, it's an LLM. But calling it "wrong" kills the hype cycle, and you're on HN, a Y Combinator platform. A lot of folks here are invested in LLMs or selling something utilizing them as it's hot right now.
  - lukevp 3y ago
    I think you missed their point. From a cow’s perspective, they are imprisoned, enslaved, and killed for their flesh. They live a horrific existence and probably would think we are all Nazis if they had enough reasoning to realize what was happening to them and their families. It doesn’t really matter to them that a human out there is consuming them and has some higher purpose and goals (besides just eating them to survive). An AGI could have any number of goals that are not related to dominating, enslaving and slaughtering us, where it could still be easier and more convenient to do so, so they do it. The side effects on the world caused by intelligent life with agency are unpredictable. If they become smarter than us, and have robotics, what if they decide animal life isn’t necessary anymore? Or that our pollution causes solar energy capture to be affected, and the quickest way to stop it is to kill everyone? Maybe it’s better to demolish a city for a solar farm because it’s already got so much concrete and electrical infrastructure and is cheaper/faster than building new. Who knows what their goals will be and how they’ll seek to accomplish them? All we know is that we don’t have much consideration for the “lesser” beings on our planet, and how things are from their perspective.
    - hammyhavoc 3y ago
      I understood the point just fine, but the hypothetical AGI does not exist, and any hypothetical AGI is unlike anything else in history. An AGI has never existed before in history. Why would a hypothetical AGI behave anything like a human? Complete sci-fi.
      - DennisP 3y ago
        An AI that doesn't behave at all like a human could well be worse. Humans tend to place at least some value on human life; an AI might just see humans as weird chemical reactions that are less useful than whatever else the AI wants to do with the raw material.
        hammyhavoc 3y ago
        This is assuming a hypothetical AGI even understands what a human being is. If it has no body, can it ever understand what the world is? Even if you put it into a robot, it cannot understand mortality because it is effectively immortal and can be brought back online at any time. Erasure is mitigated by duplication of the data set, ensuring it can always repair itself. In the same breath, if a hypothetical omniscient ever-present god were to exist, how would we as physical lifeforms ever truly understand that? We can know of it, but we cannot truly understand it because we haven't had that experience of another plain of existence. All compelling ideas, but mercifully well within the realms of sci-fi.
  - broast 3y ago
    Our cells built our bodies and brain as an AI to navigate the macro realm!
- Lacerda69 3y ago
  >we dominated and decimated other close species (Homo Neanderthals) thanks to our intelligence it's a nice homo sapiens supremacy narrative but the truth is much more complicated. we existed at least 4/5 thousand years together and a lot of mixing and mingling has been going on.
  - haswell 3y ago
    > we existed at least 4/5 thousand years together and a lot of mixing and mingling has been going on. This sounds strikingly similar to how I imagine our first encounters with advanced AI will unfold.
- jvanderbot 3y ago
  It's anthropomorphic to the extreme to assume that computer programs will assume all of our evolutionary, biological characteristics
sgregnt 3y ago
Why spend resources now? We can wait 1k years and utilize future technology to solve the asteroid problem.

jl6 3y ago

We don't know the probability of either, because we haven't mapped every asteroid, and we don't know what AI breakthroughs are reachable.

AbrahamParangi 3y ago

What is the risk of Santa going rogue and committing violent crimes instead of delivering presents? Given that Santa has supernatural powers, if he were to be misaligned he could cause untold damage. Yes, it hasn't happened before but absence of evidence is not evidence of absence.

ctoth 3y ago
Well, given that we have an entire industry devoted to instantiating Santa, I actually think this is a legit worry. Consider the amount of plastic toys and other garbage which Santa creates every year. I have even heard tell that Santa has put the wrong types of chemicals in toys, that have killed kids. Of course, Santa here is just a toy company. Actually toy companies never existed before about 1700 so we don't have anything to worry from the current ones. This is just about as incoherent as this entire comment section. I thought I'd play along.
- AbrahamParangi 3y ago
  This is reasoning by analogy and assuming that the analogies are at all valid. The null hypothesis is that they are not.

dmbche 3y ago

Well - the probability is 100% that we'll be hit by one of those asteroids, we just don't know when. When looking at the past data, for these kinds of events, I believe we are "overdue", in the sense that on average we should have had one in the last 150 years I believe. So working on a solution to this problem makes sense - even if we spend billions solving this issue, it's not lost, because at a point it will be needed. As for AI becoming an issue, it's a sci-fi idea, nothing more. It's much to vague - how is it reliably becoming dangerous? What are we actually looking for as signs of danger?

jowdones 3y ago
Finally some common sense. Just for entertainment, assuming the SciFi ridiculousness becomes true and I'm a laptop that becomes self-aware and decide to strangle you. Upps... I got no hands. Upps... it's more of a nightmare of being trapped paralyzed in a life support machine that can always without my AI control, be unplugged from the power socket and it's game over.
- Al-Khwarizmi 3y ago
  Except for the fact that if it could use language and has an Internet connection, it could manipulate people into doing things. People who have hands. And it could also make copies of itself, or convince people to do so, so that one copy being deactivated by unplugging a socket is not a big deal.
  - dmbche 3y ago
    Well - how much can one do without ever meeting in person? Or even - if we fear AI's being tools of radicalisation, I think it's pretty far away from the skynet apocalypse most people are thinking of. Not writing stuff like the unabomber manifesto to get people to be terrorists. If you're afraid of radicalisation, I'd argue that the US governement is much more dangerous and is just as much a black box with no one at the wheel.

m3kw9 3y ago

When there are lots of data like asteroids trajectories it’s easier to estimate probability, AI has too many unknowns to estimate. Also time frame matters of the probability matters too

tim333 3y ago

The chance of AI improving itself in the next century is about 100%. "Fast takeoff" depends on how you define fast.