7 ms·

When a new potentially-hazardous asteroid is discovered, it's normal for the probability of impact to go up a couple times before abruptly plummeting to zero, a

When a new potentially-hazardous asteroid is discovered, it's normal for the probability of impact to go up a couple times before abruptly plummeting to zero, as the radius of uncertainty shrinks until it shrinks past the Earth. See https://en.wikipedia.org/wiki/Torino_scale#/media/File:Apophis_ellipse.svg https://en.wikipedia.org/wiki/Torino_scale#/media/File:Apoph.... A 3% chance of impact right after discovery with the initial error ellipse isn't all that unusual; it will almost certainly be revised to 0 with more observation (and if it's not, you'll hear about it).

apatil 3y ago
This makes sense, thanks. However, it doesn't mean that the 3% estimate of the chance of impact was wrong at the time of initial observation given data available, and that still makes it a huge deal at that time. At minimum, it would seem to justify using the best available instruments to characterize the asteroid more precisely as soon as possible. If these large numbers happen so often that asteroids with initial impact probabilities of 3% are known to actually impact much less frequently than that, then the model is poorly calibrated, no? In other words, the reported probabilities aren't really probabilities and that is what has caused the confusion and anxiety in these comments.
- abathur 3y ago
  I don't know, but I suspect this is more about the limits of the observations (which I imagine are mostly from terrestrial observatories) needed to obtain much certainty about the object's size, course, speed, density, etc.
- cfraenkel 3y ago
  It's not a model that is poorly calibrated - you seem to be taking a software-centric concept far away from where it's useful. The uncertainty at initial observation is because when you first observe an object, you only have observations covering a tiny bit of the orbit, resulting in very wide error bars. The "model" (Newtonian orbital dynamics) is one of the most precise models we have. Doesn't help when the observations are noisy.
  - MauranKilom 3y ago
    Unless one in every 33 asteroids that have 3% impact probability at some point in time actually impacts earth, there is clearly some unwarranted assumption in the error bar/distribution calculation. "The measurement data has noise" does not explain why the noise has a bias towards "the asteroid will hit earth" whereas reality so far has been biased towards "the asteroid will not hit earth". (This assumes that significantly more than 33 asteroids have had >= 3% impact probability predicted at some point. The opposite would not be less concerning.)
    - noduerme 3y ago
      One explanation would be the Anthropic Principle. In 3% of universes you were killed today, you're just not living in one of those.
      - jacquesm 3y ago
        In 97% of the universes dinosaur descendants rule planet earth. But on this one they got unlucky.
      - MauranKilom 3y ago
        This only works if there is nothing between "no impact" and "you die the same day as the impact". But we know that's not the case.
        noduerme 3y ago
        Why "same day" and not same week or month? If it's not the same instant, then you're hypothesizing some kind of back-propagation (where alternate futures in which you die influence the likelihood of current events)[0]. Under that hypothesis, it would only matter whether some event would cause you to die sooner than you otherwise would. [0] https://arxiv.org/abs/0707.1919 https://arxiv.org/abs/0707.1919 [edit] FWIW, I actually corresponded with one of the authors of this paper back in 2007, and from what I could tell, this wasn't an attempt at parody, although now it might be dismissed as one. Personally I'm not willing to declare my (non)commitment to the theory either way.
- aaron695 3y ago
  > This makes sense, thanks. It makes no fucking sense. There is 3% chance it'll be revised to 100% chance and 97% chance it'll be revised to 0% chance. Are you Yogi Berra? It'll be reported like possible hurricanes hitting landfall. If this has happed ~33 times then one will hit us. If it's 400m it will kill 200,000 people assuming Vox is reporting correctly - https://www.vox.com/future-perfect/2019/7/26/8931776/near-earth-asteroid-tracking https://www.vox.com/future-perfect/2019/7/26/8931776/near-ea... That will be 9 in 10 it hit's boring ocean and looks cool on satellites and one in 10 kills 2 million people and there will be some cool live streams. Or the 3% in the title is a lie.
  - jryb 3y ago
    > If this has happed ~33 times then one will hit us. There would be a ~63.4% chance that at least one would hit us if this happened 33 times. To compute this, take 1-(0.97^33). But I agree with your broader intuition that these predictions must be getting inflated.
- apatil 3y ago
  In hindsight this comment was too short. Clarifying some points: By "This makes sense", I meant that this kind of thing can happen; as more data are gathered, the Bayesian probability of a candidate value can increase and then suddenly decrease. Here's a Colab notebook demonstrating the general phenomenon: https://colab.research.google.com/drive/1Eb1_humiGPdKb0c3qr_0k1yYDWBVODKm?usp=sharing https://colab.research.google.com/drive/1Eb1_humiGPdKb0c3qr_... "Calibration" in this context means "statistical consistency between distributional forecasts and observations" in the words of https://sites.stat.washington.edu/raftery/Research/PDF/Gneiting2007jrssb.pdf https://sites.stat.washington.edu/raftery/Research/PDF/Gneit... . If the model's early forecasts predict impact with probability >3% for a class of objects that end up impacting with frequency much less than 3%, then the model is not well calibrated with respect to its early forecasts for those objects. Based on the GP, it sounds like these early impact "probabilities" are no one's subjective (Bayesian) probability of impact because people who are closely familiar with this model know it is not well calibrated. The reported probabilities may still be useful to them as indicators or flags. However, those of us who are _not_ closely familiar with the model have found it confusing to see things that are not really probabilities reported as probabilities.
mortehu 3y ago
What percent of the times when the initial probability is 3% will it later be revised to zero? Is it more than 97%?
- jacquesm 3y ago
  To date the percentage that got revised to zero is 100%. And the only time we had a really close call we didn't see it coming.
  - Mrkvak 3y ago
    That's not correct at all: https://newton.spacedys.com/neodys2/NEOScan/index_past_imp.html https://newton.spacedys.com/neodys2/NEOScan/index_past_imp.h...
    - jacquesm 3y ago
      Fair enough, but those are all relatively small, and detected only hours before impact, their effect would not be such that anything major would happen on the surface of the planet beyond some broken windows and maybe a sunburn. Anything that size aimed straight down would most likely not reach the ground but burn up in the atmosphere and any remaining bits would just fall at regular terminal velocity. But from 10 meters and up things change and the Chelyabinsk meteor is remarkable in that it (1) was large enough to have been detected but wasn't and (2) struck while we were apparently focused on one that was more visible but that ultimately missed us. We were very lucky that it impacted where and at the angle that it did, otherwise the airburst might have happened far closer to the ground or to might have impacted directly over much more populated territory. That would have been very bad news. It doesn't matter how many 1 through 5 meter objects we can track because we have the atmosphere to protected us from the worst of these if we miss the 20 meter ones (or apparently even much larger) that travel at speeds high enough to give their relatively modest mass tremendous energy and for which the atmosphere does not give sufficient (or even any) protection.
mabbo 3y ago
I think, given the expected impact time windows have passed now, we can update it to 0% (this time).

apatil 3y ago

This makes sense, thanks. However, it doesn't mean that the 3% estimate of the chance of impact was wrong at the time of initial observation given data available, and that still makes it a huge deal at that time. At minimum, it would seem to justify using the best available instruments to characterize the asteroid more precisely as soon as possible. If these large numbers happen so often that asteroids with initial impact probabilities of 3% are known to actually impact much less frequently than that, then the model is poorly calibrated, no? In other words, the reported probabilities aren't really probabilities and that is what has caused the confusion and anxiety in these comments.

abathur 3y ago
I don't know, but I suspect this is more about the limits of the observations (which I imagine are mostly from terrestrial observatories) needed to obtain much certainty about the object's size, course, speed, density, etc.
cfraenkel 3y ago
It's not a model that is poorly calibrated - you seem to be taking a software-centric concept far away from where it's useful. The uncertainty at initial observation is because when you first observe an object, you only have observations covering a tiny bit of the orbit, resulting in very wide error bars. The "model" (Newtonian orbital dynamics) is one of the most precise models we have. Doesn't help when the observations are noisy.
- MauranKilom 3y ago
  Unless one in every 33 asteroids that have 3% impact probability at some point in time actually impacts earth, there is clearly some unwarranted assumption in the error bar/distribution calculation. "The measurement data has noise" does not explain why the noise has a bias towards "the asteroid will hit earth" whereas reality so far has been biased towards "the asteroid will not hit earth". (This assumes that significantly more than 33 asteroids have had >= 3% impact probability predicted at some point. The opposite would not be less concerning.)
  - noduerme 3y ago
    One explanation would be the Anthropic Principle. In 3% of universes you were killed today, you're just not living in one of those.
    - jacquesm 3y ago
      In 97% of the universes dinosaur descendants rule planet earth. But on this one they got unlucky.
    - MauranKilom 3y ago
      This only works if there is nothing between "no impact" and "you die the same day as the impact". But we know that's not the case.
      - noduerme 3y ago
        Why "same day" and not same week or month? If it's not the same instant, then you're hypothesizing some kind of back-propagation (where alternate futures in which you die influence the likelihood of current events)[0]. Under that hypothesis, it would only matter whether some event would cause you to die sooner than you otherwise would. [0] https://arxiv.org/abs/0707.1919 https://arxiv.org/abs/0707.1919 [edit] FWIW, I actually corresponded with one of the authors of this paper back in 2007, and from what I could tell, this wasn't an attempt at parody, although now it might be dismissed as one. Personally I'm not willing to declare my (non)commitment to the theory either way.
aaron695 3y ago
> This makes sense, thanks. It makes no fucking sense. There is 3% chance it'll be revised to 100% chance and 97% chance it'll be revised to 0% chance. Are you Yogi Berra? It'll be reported like possible hurricanes hitting landfall. If this has happed ~33 times then one will hit us. If it's 400m it will kill 200,000 people assuming Vox is reporting correctly - https://www.vox.com/future-perfect/2019/7/26/8931776/near-earth-asteroid-tracking https://www.vox.com/future-perfect/2019/7/26/8931776/near-ea... That will be 9 in 10 it hit's boring ocean and looks cool on satellites and one in 10 kills 2 million people and there will be some cool live streams. Or the 3% in the title is a lie.
- jryb 3y ago
  > If this has happed ~33 times then one will hit us. There would be a ~63.4% chance that at least one would hit us if this happened 33 times. To compute this, take 1-(0.97^33). But I agree with your broader intuition that these predictions must be getting inflated.
apatil 3y ago
In hindsight this comment was too short. Clarifying some points: By "This makes sense", I meant that this kind of thing can happen; as more data are gathered, the Bayesian probability of a candidate value can increase and then suddenly decrease. Here's a Colab notebook demonstrating the general phenomenon: https://colab.research.google.com/drive/1Eb1_humiGPdKb0c3qr_0k1yYDWBVODKm?usp=sharing https://colab.research.google.com/drive/1Eb1_humiGPdKb0c3qr_... "Calibration" in this context means "statistical consistency between distributional forecasts and observations" in the words of https://sites.stat.washington.edu/raftery/Research/PDF/Gneiting2007jrssb.pdf https://sites.stat.washington.edu/raftery/Research/PDF/Gneit... . If the model's early forecasts predict impact with probability >3% for a class of objects that end up impacting with frequency much less than 3%, then the model is not well calibrated with respect to its early forecasts for those objects. Based on the GP, it sounds like these early impact "probabilities" are no one's subjective (Bayesian) probability of impact because people who are closely familiar with this model know it is not well calibrated. The reported probabilities may still be useful to them as indicators or flags. However, those of us who are _not_ closely familiar with the model have found it confusing to see things that are not really probabilities reported as probabilities.

mortehu 3y ago

What percent of the times when the initial probability is 3% will it later be revised to zero? Is it more than 97%?

jacquesm 3y ago
To date the percentage that got revised to zero is 100%. And the only time we had a really close call we didn't see it coming.
- Mrkvak 3y ago
  That's not correct at all: https://newton.spacedys.com/neodys2/NEOScan/index_past_imp.html https://newton.spacedys.com/neodys2/NEOScan/index_past_imp.h...
  - jacquesm 3y ago
    Fair enough, but those are all relatively small, and detected only hours before impact, their effect would not be such that anything major would happen on the surface of the planet beyond some broken windows and maybe a sunburn. Anything that size aimed straight down would most likely not reach the ground but burn up in the atmosphere and any remaining bits would just fall at regular terminal velocity. But from 10 meters and up things change and the Chelyabinsk meteor is remarkable in that it (1) was large enough to have been detected but wasn't and (2) struck while we were apparently focused on one that was more visible but that ultimately missed us. We were very lucky that it impacted where and at the angle that it did, otherwise the airburst might have happened far closer to the ground or to might have impacted directly over much more populated territory. That would have been very bad news. It doesn't matter how many 1 through 5 meter objects we can track because we have the atmosphere to protected us from the worst of these if we miss the 20 meter ones (or apparently even much larger) that travel at speeds high enough to give their relatively modest mass tremendous energy and for which the atmosphere does not give sufficient (or even any) protection.

mabbo 3y ago

I think, given the expected impact time windows have passed now, we can update it to 0% (this time).