22 ms·

While the article obliquely makes reference to this, I wonder if this is somewhat similar to the "birthday" problem in statistics. Any given person has ~1 in 36

While the article obliquely makes reference to this, I wonder if this is somewhat similar to the "birthday" problem in statistics. Any given person has ~1 in 365 chance of being born on a particular day. Yet, if you have 30 people in a room, there is a greater than 50 percent chance that at least one PAIR has the same birthday. Given the hundreds of cities (thousands?) in the US that may be affected by a flood, it's actually quite likely that at least one of them could have a pretty high probability of experiencing one in 500 chance of something each year. I haven't run the numbers, of course, but we're looking at Houston in isolation - it needs to be seen in the broader context of all possible cities. We're focusing on the individual probability of Houston having this improbable string of events, rather than looking at Houston only being the city that experienced this phenomenon. Of course, the "once in 500 years flood" may also be an inaccurate probability as well, but if its not, then I'm not surprised this is happening SOMEWHERE.

dsfyu404ed 9y ago
Nobody notices the 500yr events that happen in the middle of nowhere.
- deleted 9y ago
  [deleted]
- Piskvorrr 9y ago
  With water? Unlikely. Water doesn't just drop and disappear on the same spot in the middle of nowhere, it tends to move downstream, into places that are in the middle of somewhere - as we have seen with the Mississippi watershed: first Out In Nowhere, IN, then ORCS, then Lake Ponchartrain. A 500y water event in nowhere land would still propagate; the speed of water release from the drop point (sorry for not using the correct jargon) would mean the difference between a 10y event downstream or a 500y event on the go.
TallGuyShort 9y ago
>> Of course, the "once in 500 years flood" may also be an inaccurate probability as well I suspect that's it. Of course I may be falling into the same birthday paradox here, but a similar thing happened when it flooded where I lived now. The x in "x-year Flood" kept increasing (I'm inclinced to blame that on sensationalizing news for ratings), but people kept comparing it to the 3 or 4 similar floods that happened in the last hundred years. It would seem the probably has gone up due to climate change, or that we're noticing the floods more now because there's more damage to be caused to more things that have been built up, and we have better records and better news transmission, but it certainly seems that can objectively say they're no longer 1-in-500 year flood regardless of the reason.
- mannykannot 9y ago
  There is always considerable political pressure to low-ball the estimates. That is hinted at in this article, and it happened in New York, where the first post-Sandy flood zone maps provoked widespread opposition from those in the newly-expanded zones, and they were rapidly scaled back. There is, of course, some uncertainty in the estimation, but it seems likely that the official maps will be, at best, based on the most favorable possible interpretation of the data. Mother nature does not respect grandfather clauses.
  - x0x0 9y ago
    Because, crucially, your mortgage may require you to buy insurance in flood plains. And it can be quite expensive. see also https://qz.com/1063985/hurricane-harvey-why-85-of-homeowners-in-houston-dont-have-federal-flood-insurance/ https://qz.com/1063985/hurricane-harvey-why-85-of-homeowners... There's gonna be lots of screaming when homeowners in Houston realize how much flood insurance is going to cost them in the future. We all got a preview of this in coastal Florida.
    - cat199 9y ago
      Pretty much everyone knows it's a good idea to get flood insurance in Houston.. We still get lots of localized downpours, floodplane or no, which can easily overwhelm local drainage, etc. Whether they actually do it or not is another idea
      - mannykannot 9y ago
        I am aware that flood insurance itself is subject to government influence - e.g. the National Flood Insurance Program, and possibly state regulation (even in Texas) - but I don't know the details. Does it result in premiums being affected by official flood-zone designations? As you appear to be in or near Houston, I hope things are OK with you and your family.
        cat199 9y ago
waqf 9y ago
Good point, but the chance of a given city having a 500-year flood in three given successive years is only (0.002)³ = 8 × 10⁻⁹. So even if we allowed for 100 U.S. cities, 10 different natural disasters and 10 different three-year periods we're still talking about a probability of less than 0.01%.
- empath75 9y ago
  And it's not like Houston is the only city having 100 year+ floods regularly.
  - Taniwha 9y ago
    yes, it's almost as if the climate is changing
    - _Tev 9y ago
      Or our assumptions about climate / wheather were wrong.
- Piskvorrr 9y ago
  ...the assumption being that the risks are separate. This is clearly false - you are not rolling dice here, the probabilities of the same disaster at geographically "close" cities are linked.
  - waqf 9y ago
    That is true but it's an error in the unproblematic direction: fixing it makes the argument even stronger. That is, if the probability for each city flooding is 0.002 but the different cities are positively correlated, then the probability of at least one city flooding is lower than it would be if they were uncorrelated. (Because the probability of no cities flooding is higher, because of positive correlation.)
ChadyWady 9y ago
From what I understand of the article, the "1 in 500 years" measure is based on the estimated probability of a flood on a given year in a local area. So the probability of a "1 in 500 year" flood in Houston is independent of the probability of a "1 in 500 year" flood for another city.
- twinkletwinkle 9y ago
  Right what the parent comment is saying is that look at a map of the US, how many major cities are there. The odds of two "1 in 500 year" floods happening consecutively are low for any individual city. But if you have enough cities it will happen somewhere. And then you write the article about it. It's kind of analogous to hitting a golf ball into an open field and then exclaiming "Of all the blades of grass, the ball landed on this one!" It's only interesting if it goes in the hole with the flag, otherwise you have a sort of selection bias going.
  - ChadyWady 9y ago
    That makes sense, I was confused since the Birthday problem is more about the pigeonhole principle rather than larger samples being more likely to have individually unlikely outcomes. But for this case, it would still be very unlikely. For any arbitrary 3-year span, the probability of consecutively getting a "1 in 500 year" flood for 3 years is (1/500)^3. If we have N cities, then the probability of none of these cities having 3 consecutive floods is (1-(1/500)^3)^N. For even a vast overestimate such as N=20000, it is still a significantly improbable event. Of course, this doesn't account for "3 consecutive years or more within some year range" and it is a gross simplification, but I think there are probably better explanations than selection bias, such as the inaccuracy of the model or the fact that these events might be temporally dependent on each other.
- analog31 9y ago
  My understanding is that by definition, a 500 year flood has a probability of 1/500 per year. But I live at the top of a hill in the Midwest, so a 500 year flood in my locale has a much different severity than a 500 year flood in Houston.
curiouscats 9y ago
A 500 year flood event is very likely new data that tells us our prediction it was a 1 in 500 year event was wrong. http://engineering.curiouscatblog.net/2008/07/13/500-year-floods/ It is of course possible, it was actually a once in 500 year event. I just believe it is much more likely our previous conclusion was faulty. Our prediction of 500 year floods is not very good, we rely on way less than 500 years of data. Also in most places where this happens the massive changes to the environment (roads, cities, paved over wetlands, constrained rivers...) are not factored in well at all. Add to that global climate change and 500 year flood estimates are likely still poor today. The birthday problem doesn't relate to us getting new data that changes what we used to know. A 500 year flood probably is new data (that gives us a strong indication our previous belief was wrong).
- autokad 9y ago
  more likely their model is wrong. it reminds me of fukushima: https://ml.berkeley.edu/blog/2017/07/13/tutorial-4/ https://ml.berkeley.edu/blog/2017/07/13/tutorial-4/
  - enraged_camel 9y ago
    Their model is wrong, but it's because it assumes climate is stationary. Climatologists are actively debating what the models should look like when global warming is taken into account: https://i.imgur.com/HLxUBim.png https://i.imgur.com/HLxUBim.png edit: not sure why I'm being downvoted. I can only assume the downvotes are political in nature, which is a shame.
    - tedunangst 9y ago
      That's not such a great reference...
    - sdenton4 9y ago
      A screen cap of a discussion forum is a poor reference indeed...
- fjfiggiguffuf 9y ago
walshemj 9y ago
its the gamblers fallacy black 17 has not come up for ages so its more likely to come up - looks like the journalist does not understand probability
- ncallaway 9y ago
  Except there's a very good ability to know precisely the odds of what the ball landing on black 17 are. It's a very complicated question to determine what a 1/500 odds of flooding are. So, it's very possible that this isn't the gambler's fallacy, but rather the model itself is inaccurate.
  - walshemj 9y ago
    or the massive building growth without building the required infrastructure
- mikeash 9y ago
  It's more analogous to getting black 17 six times in a row. It is possible, but it's weird enough to justify thinking that you've misunderstood the game, or the game has changed.
- deleted 9y ago
  [deleted]
- walshemj 9y ago
  really voted down for pointing out probability 101 I would have expected better mathematical knowledge on HN
  - Piskvorrr 9y ago
    Probability 101 begins with a qualifier: "Assuming independent events..." This can mean two things: a) a set of events that is evidently possible, but increasingly improbable b) events that are not independent
polotics 9y ago
No it's not the same thing. The 500 was a Houston stat, not a US overall one.
SideburnsOfDoom 9y ago
> Of course, the "once in 500 years flood" may also be an inaccurate probability as well Or rather, the odds of that kind of flood increase a bit each year.
pesfandiar 9y ago
I understand your argument, but you can't compare floods to the birthday problem. Unlike randomly-sampled people's birthdays, floods in US cities are not independent random variables. For instance, chances of 500-year floods in two Texas cities is not 1 out of 500*500 in a year. If they're geographically similar and physically close, it should be closer to 1 out of 500.
- saas_sam 9y ago
  In 2 Texas cities, sure, but in every U.S. city? There are going to be several that end up getting a few 500-year floods... and several others that don't flood as expected in several 1000 years.
  - Spooky23 9y ago
    In many US cities. Recall that during hurricane sandy, pretty much any landfill or low lying area was flooded in the NY metro area. That means dozens of cities and towns with 7% of the US population.
hexane360 9y ago
Birthday paradox relies on non-independent events. I'd assume floods should be relatively independent between different cities, meaning that the number of 500-year flood events in the U.S. would follow a binomial distribution (https://en.wikipedia.org/wiki/Binomial_distribution https://en.wikipedia.org/wiki/Binomial_distribution). p = 1/500 years = 0.002 For 100 cities, the probability of at least one city having a 500-year flood is 18.14%. For a 3-year run: p = (1/500)^3 = 8e-9 For 100 cities, the probability is 8e-7 of at least one success. For 1000 cities, the probability is still only 8e-6. Wolfram alpha helps in calculating PDFs and CDFs: http://www.wolframalpha.com/input/?i=binomial+distribution+n+%3D+1000,+p+%3D+8e-9 http://www.wolframalpha.com/input/?i=binomial+distribution+n...
- learc83 9y ago
  How does the birthday paradox rely on non-independent events?
  - hexane360 9y ago
    I guess what I meant wasn't "non-independent events". However, there is still a fundamental difference. The probability of a 500-year flood happening next year is still 0.2%. With a birthday paradox, the filling of birthdays is what causes the explosion in probability. Assuming no collisions so far, the 20th person you add to the room has a 19/365 chance of colliding, while the 2nd person only has a 1/365 chance. I'm not sure how to succinctly phrase this idea. Maybe "non-constant probabilities"?
    - thaumasiotes 9y ago
      In the terms of an example: If you put 400 people in a room, the odds that no two of them share a birthday have gone all the way down to zero percent. No matter how many independent cities you measure flooding in, you can never achieve a perfect 0 or 1 probability of (whatever), because the cities are independent. We don't define "interesting" flooding in one city by reference to what happened in another city, so this situation is not described by the birthday paradox.
- 9y ago
T-hawk 9y ago
> it's actually quite likely that at least one of them could have a pretty high probability of experiencing one in 500 chance of something each year. This is the key point here, and it's called the fallacy of multiple endpoints. If there are 500 observable cities, then somewhere among them should occur a "500-year event" every year. It's just that nobody ever notices the big bulk of the population that hasn't yet had any 500-year event in its recorded history.
kurthr 9y ago
But storms aren't people with randomly distributed cities to flood so you don't get the combinatorial effect. It's not like there are a fixed number of 500 year storms every 3 years that have to flood a uniformly distributed list of cities. The floods are supposedly independent (even if our estimates of likelihood are bad) so there's no binomial coeeficient. The simple calculation is P=1-(1-1/500^3)^N and the probability is less than 1:10000 assuming independence and even given 6000 cities of a million people (the entire world population is in a similar city). If it was birthday like (every possible 500 year storm for 3 years is in a room choosing out of 100 random million person cities), the probablility would be reasonable. You can use a Poisson distribution to estimate the 3 person birthday problem. P=1-exp(-3choose100/500^2)=48%
- roenxi 9y ago
  Actually, in the US it is approximately 1 in 250,000 /assuming independence which is a bit of an assumption/. Obviously, nature doesn't know what a year is so it is totally possible that big storms come in batches. Assuming 1 in 500 year floods are independent, note the ~300 cities in the US [0]. The fact that Huston is a big city doesn't seem relevant to me, because the storm doesn't care. Much like picking a specific number plate, it had to land somewhere. The chance of there being a city in the US which has a 1 in 500 year flood is therefore 60% (300/500) each year. The chance of that city having another flood of similar magnitude in Year 2 is 1/500, because the odds are independent and we started monitoring the city in Year 1. The chance of it flooding again the third year brings the odds down to 1/(500 * 500) = 1 in 250,000. [0] https://en.wikipedia.org/wiki/List_of_United_States_cities_by_population https://en.wikipedia.org/wiki/List_of_United_States_cities_b...
rothbardrand 9y ago
Having lived in Houston, this isn't a measure of the flood but a measure of how terrible the infrastructure is. They have terrible flood management systems but are "rated" to handle everything but a "100" year flood... yet Houston gets 100 year floods every year- multiple ones- and has for 40 years. It's a scam involving the ratings much like the mooody's ratings scam of the last decade IMNSHO
danmaz74 9y ago
How do you count "floods"? If there is one hurricane that causes floods in 3 cities, is that 1 flood, or 3? If it's 1 - which is what I think is meant when saying "a 1 in XXX years flood" - then your comparison doesn't hold, because the number of floods isn't dependent on the number of cities, but on the distribution of cities on the land, and specifically in flood-prone areas.
- Spooky23 9y ago
  IMO you need to look at something other than city limits. Cities like Houston and LA are huge cities geographically. IMO you need to look at regional and per capita impacts.
epitomix 9y ago
I think that this statistic only makes sense if it is specific to the location. Otherwise, it is useless for planning purposes. If it is specific to the location then the birthday paradox does not apply. Interesting note, this is Houstons third 500-year flood in three years [1]. [1] -https://www.washingtonpost.com/news/wonk/wp/2017/08/29/houston-is-experiencing-its-third-500-year-flood-in-3-years-how-is-that-possible https://www.washingtonpost.com/news/wonk/wp/2017/08/29/houst...
oso2k 9y ago
The "Birthday Problem" has a simple explanation. Something like nearly 45% of people are born in the Summer months. Ensure that you get people of different ages and your chances of finding someone with similar/same birthday goes up enormously.
- brewdad 9y ago
  Is this true? Which months are considered "summer"? Northern Hemisphere or Southern or both? This can easily turn into a "40% of sick days are on Monday or Friday" type fallacy.
- pmalynin 9y ago
  Thats....not how that works. At all.
- bitwize 9y ago
  The birthday problem still exists even assuming that everyone has an equal chance of being born on any day of the year. It's a phenomenon of the statistical likelihood that given a large enough group of people, the likelihood that any two will have the same birthday goes to almost certainty very rapidly.
- umanwizard 9y ago
  No, the birthday problem still works if you consider that the probability of being born on a particular day is uniform. It's related to the fact that the probability of two people in a room having a birthday is more closely related to the number of pairs of people in the room (which in turn grows as O(n^2)), than to the number of people in the room. A detailed proof/explanation can be looked up easily with Google, if you have a bit of mathematical maturity.
- oso2k 9y ago
  For those who down voted me, you do not know your sample set or your test subjects. Go to the empirical data sources. Human behavior certainly affects the real probabilities. As of 2015 in the US [0], a person is more likely to be born in July, August or September. They're also more likely to be born during the middle of the work week rather than the weekends. Draw your own conclusions from the data. [0] https://www.cdc.gov/nchs/data/nvsr/nvsr66/nvsr66_01_tables.pdf https://www.cdc.gov/nchs/data/nvsr/nvsr66/nvsr66_01_tables.p...
curiousgeorgio 9y ago
A more accurate term for what you describe would be a clustering illusion[1], and it certainly could be a valid explanation (among other possibilities) for the observation made in the article. [1] https://en.wikipedia.org/wiki/Clustering_illusion https://en.wikipedia.org/wiki/Clustering_illusion
- honestoHeminway 9y ago
  Yes, or as with nuclear power, it could be that the base assumption regarding safety just was wrong.
simonebrunozzi 9y ago
Ahem, no - your birthdays example can't apply here. When you increase the number of people in the room, each one of them benefits from every new person added, in terms of probability of having been born on the same day. In the flood case, a flood in California and one in Miami don't really increase the chance of another flood simply because they have happened.
bnegreve 9y ago
> Yet, if you have 30 people in a room, there is a greater than 50 percent chance that at least one PAIR has the same birthday. Not quite the same because Huston is not the only city that has experienced extreme climate these two years, many other cities also have (maybe not 500 years flood, but close). So that would be like: one pair is born the same day, and 20 other people in the room are born a week around that day. Hardly a coincidence. It's more likely that there was a party about 9 months and X years before that day. (I'm not sure how far we can go with this silly birthday analogy:)

dsfyu404ed 9y ago

Nobody notices the 500yr events that happen in the middle of nowhere.

deleted 9y ago
[deleted]
Piskvorrr 9y ago
With water? Unlikely. Water doesn't just drop and disappear on the same spot in the middle of nowhere, it tends to move downstream, into places that are in the middle of somewhere - as we have seen with the Mississippi watershed: first Out In Nowhere, IN, then ORCS, then Lake Ponchartrain. A 500y water event in nowhere land would still propagate; the speed of water release from the drop point (sorry for not using the correct jargon) would mean the difference between a 10y event downstream or a 500y event on the go.

TallGuyShort 9y ago

>> Of course, the "once in 500 years flood" may also be an inaccurate probability as well I suspect that's it. Of course I may be falling into the same birthday paradox here, but a similar thing happened when it flooded where I lived now. The x in "x-year Flood" kept increasing (I'm inclinced to blame that on sensationalizing news for ratings), but people kept comparing it to the 3 or 4 similar floods that happened in the last hundred years. It would seem the probably has gone up due to climate change, or that we're noticing the floods more now because there's more damage to be caused to more things that have been built up, and we have better records and better news transmission, but it certainly seems that can objectively say they're no longer 1-in-500 year flood regardless of the reason.

mannykannot 9y ago
There is always considerable political pressure to low-ball the estimates. That is hinted at in this article, and it happened in New York, where the first post-Sandy flood zone maps provoked widespread opposition from those in the newly-expanded zones, and they were rapidly scaled back. There is, of course, some uncertainty in the estimation, but it seems likely that the official maps will be, at best, based on the most favorable possible interpretation of the data. Mother nature does not respect grandfather clauses.
- x0x0 9y ago
  Because, crucially, your mortgage may require you to buy insurance in flood plains. And it can be quite expensive. see also https://qz.com/1063985/hurricane-harvey-why-85-of-homeowners-in-houston-dont-have-federal-flood-insurance/ https://qz.com/1063985/hurricane-harvey-why-85-of-homeowners... There's gonna be lots of screaming when homeowners in Houston realize how much flood insurance is going to cost them in the future. We all got a preview of this in coastal Florida.
  - cat199 9y ago
    Pretty much everyone knows it's a good idea to get flood insurance in Houston.. We still get lots of localized downpours, floodplane or no, which can easily overwhelm local drainage, etc. Whether they actually do it or not is another idea
    - mannykannot 9y ago
      I am aware that flood insurance itself is subject to government influence - e.g. the National Flood Insurance Program, and possibly state regulation (even in Texas) - but I don't know the details. Does it result in premiums being affected by official flood-zone designations? As you appear to be in or near Houston, I hope things are OK with you and your family.
      - cat199 9y ago

waqf 9y ago

Good point, but the chance of a given city having a 500-year flood in three given successive years is only (0.002)³ = 8 × 10⁻⁹. So even if we allowed for 100 U.S. cities, 10 different natural disasters and 10 different three-year periods we're still talking about a probability of less than 0.01%.

empath75 9y ago
And it's not like Houston is the only city having 100 year+ floods regularly.
- Taniwha 9y ago
  yes, it's almost as if the climate is changing
  - _Tev 9y ago
    Or our assumptions about climate / wheather were wrong.
Piskvorrr 9y ago
...the assumption being that the risks are separate. This is clearly false - you are not rolling dice here, the probabilities of the same disaster at geographically "close" cities are linked.
- waqf 9y ago
  That is true but it's an error in the unproblematic direction: fixing it makes the argument even stronger. That is, if the probability for each city flooding is 0.002 but the different cities are positively correlated, then the probability of at least one city flooding is lower than it would be if they were uncorrelated. (Because the probability of no cities flooding is higher, because of positive correlation.)

ChadyWady 9y ago

From what I understand of the article, the "1 in 500 years" measure is based on the estimated probability of a flood on a given year in a local area. So the probability of a "1 in 500 year" flood in Houston is independent of the probability of a "1 in 500 year" flood for another city.

twinkletwinkle 9y ago
Right what the parent comment is saying is that look at a map of the US, how many major cities are there. The odds of two "1 in 500 year" floods happening consecutively are low for any individual city. But if you have enough cities it will happen somewhere. And then you write the article about it. It's kind of analogous to hitting a golf ball into an open field and then exclaiming "Of all the blades of grass, the ball landed on this one!" It's only interesting if it goes in the hole with the flag, otherwise you have a sort of selection bias going.
- ChadyWady 9y ago
  That makes sense, I was confused since the Birthday problem is more about the pigeonhole principle rather than larger samples being more likely to have individually unlikely outcomes. But for this case, it would still be very unlikely. For any arbitrary 3-year span, the probability of consecutively getting a "1 in 500 year" flood for 3 years is (1/500)^3. If we have N cities, then the probability of none of these cities having 3 consecutive floods is (1-(1/500)^3)^N. For even a vast overestimate such as N=20000, it is still a significantly improbable event. Of course, this doesn't account for "3 consecutive years or more within some year range" and it is a gross simplification, but I think there are probably better explanations than selection bias, such as the inaccuracy of the model or the fact that these events might be temporally dependent on each other.
analog31 9y ago
My understanding is that by definition, a 500 year flood has a probability of 1/500 per year. But I live at the top of a hill in the Midwest, so a 500 year flood in my locale has a much different severity than a 500 year flood in Houston.

curiouscats 9y ago

A 500 year flood event is very likely new data that tells us our prediction it was a 1 in 500 year event was wrong. http://engineering.curiouscatblog.net/2008/07/13/500-year-floods/ It is of course possible, it was actually a once in 500 year event. I just believe it is much more likely our previous conclusion was faulty. Our prediction of 500 year floods is not very good, we rely on way less than 500 years of data. Also in most places where this happens the massive changes to the environment (roads, cities, paved over wetlands, constrained rivers...) are not factored in well at all. Add to that global climate change and 500 year flood estimates are likely still poor today. The birthday problem doesn't relate to us getting new data that changes what we used to know. A 500 year flood probably is new data (that gives us a strong indication our previous belief was wrong).

autokad 9y ago
more likely their model is wrong. it reminds me of fukushima: https://ml.berkeley.edu/blog/2017/07/13/tutorial-4/ https://ml.berkeley.edu/blog/2017/07/13/tutorial-4/
- enraged_camel 9y ago
  Their model is wrong, but it's because it assumes climate is stationary. Climatologists are actively debating what the models should look like when global warming is taken into account: https://i.imgur.com/HLxUBim.png https://i.imgur.com/HLxUBim.png edit: not sure why I'm being downvoted. I can only assume the downvotes are political in nature, which is a shame.
  - tedunangst 9y ago
    That's not such a great reference...
  - sdenton4 9y ago
    A screen cap of a discussion forum is a poor reference indeed...
fjfiggiguffuf 9y ago

walshemj 9y ago

its the gamblers fallacy black 17 has not come up for ages so its more likely to come up - looks like the journalist does not understand probability

ncallaway 9y ago
Except there's a very good ability to know precisely the odds of what the ball landing on black 17 are. It's a very complicated question to determine what a 1/500 odds of flooding are. So, it's very possible that this isn't the gambler's fallacy, but rather the model itself is inaccurate.
- walshemj 9y ago
  or the massive building growth without building the required infrastructure
mikeash 9y ago
It's more analogous to getting black 17 six times in a row. It is possible, but it's weird enough to justify thinking that you've misunderstood the game, or the game has changed.
deleted 9y ago
[deleted]
walshemj 9y ago
really voted down for pointing out probability 101 I would have expected better mathematical knowledge on HN
- Piskvorrr 9y ago
  Probability 101 begins with a qualifier: "Assuming independent events..." This can mean two things: a) a set of events that is evidently possible, but increasingly improbable b) events that are not independent

polotics 9y ago

No it's not the same thing. The 500 was a Houston stat, not a US overall one.

SideburnsOfDoom 9y ago

> Of course, the "once in 500 years flood" may also be an inaccurate probability as well Or rather, the odds of that kind of flood increase a bit each year.

pesfandiar 9y ago

I understand your argument, but you can't compare floods to the birthday problem. Unlike randomly-sampled people's birthdays, floods in US cities are not independent random variables. For instance, chances of 500-year floods in two Texas cities is not 1 out of 500*500 in a year. If they're geographically similar and physically close, it should be closer to 1 out of 500.

saas_sam 9y ago
In 2 Texas cities, sure, but in every U.S. city? There are going to be several that end up getting a few 500-year floods... and several others that don't flood as expected in several 1000 years.
- Spooky23 9y ago
  In many US cities. Recall that during hurricane sandy, pretty much any landfill or low lying area was flooded in the NY metro area. That means dozens of cities and towns with 7% of the US population.

hexane360 9y ago

Birthday paradox relies on non-independent events. I'd assume floods should be relatively independent between different cities, meaning that the number of 500-year flood events in the U.S. would follow a binomial distribution (https://en.wikipedia.org/wiki/Binomial_distribution https://en.wikipedia.org/wiki/Binomial_distribution). p = 1/500 years = 0.002 For 100 cities, the probability of at least one city having a 500-year flood is 18.14%. For a 3-year run: p = (1/500)^3 = 8e-9 For 100 cities, the probability is 8e-7 of at least one success. For 1000 cities, the probability is still only 8e-6. Wolfram alpha helps in calculating PDFs and CDFs: http://www.wolframalpha.com/input/?i=binomial+distribution+n+%3D+1000,+p+%3D+8e-9 http://www.wolframalpha.com/input/?i=binomial+distribution+n...

learc83 9y ago
How does the birthday paradox rely on non-independent events?
- hexane360 9y ago
  I guess what I meant wasn't "non-independent events". However, there is still a fundamental difference. The probability of a 500-year flood happening next year is still 0.2%. With a birthday paradox, the filling of birthdays is what causes the explosion in probability. Assuming no collisions so far, the 20th person you add to the room has a 19/365 chance of colliding, while the 2nd person only has a 1/365 chance. I'm not sure how to succinctly phrase this idea. Maybe "non-constant probabilities"?
  - thaumasiotes 9y ago
    In the terms of an example: If you put 400 people in a room, the odds that no two of them share a birthday have gone all the way down to zero percent. No matter how many independent cities you measure flooding in, you can never achieve a perfect 0 or 1 probability of (whatever), because the cities are independent. We don't define "interesting" flooding in one city by reference to what happened in another city, so this situation is not described by the birthday paradox.
9y ago

T-hawk 9y ago

> it's actually quite likely that at least one of them could have a pretty high probability of experiencing one in 500 chance of something each year. This is the key point here, and it's called the fallacy of multiple endpoints. If there are 500 observable cities, then somewhere among them should occur a "500-year event" every year. It's just that nobody ever notices the big bulk of the population that hasn't yet had any 500-year event in its recorded history.

kurthr 9y ago

But storms aren't people with randomly distributed cities to flood so you don't get the combinatorial effect. It's not like there are a fixed number of 500 year storms every 3 years that have to flood a uniformly distributed list of cities. The floods are supposedly independent (even if our estimates of likelihood are bad) so there's no binomial coeeficient. The simple calculation is P=1-(1-1/500^3)^N and the probability is less than 1:10000 assuming independence and even given 6000 cities of a million people (the entire world population is in a similar city). If it was birthday like (every possible 500 year storm for 3 years is in a room choosing out of 100 random million person cities), the probablility would be reasonable. You can use a Poisson distribution to estimate the 3 person birthday problem. P=1-exp(-3choose100/500^2)=48%

roenxi 9y ago
Actually, in the US it is approximately 1 in 250,000 /assuming independence which is a bit of an assumption/. Obviously, nature doesn't know what a year is so it is totally possible that big storms come in batches. Assuming 1 in 500 year floods are independent, note the ~300 cities in the US [0]. The fact that Huston is a big city doesn't seem relevant to me, because the storm doesn't care. Much like picking a specific number plate, it had to land somewhere. The chance of there being a city in the US which has a 1 in 500 year flood is therefore 60% (300/500) each year. The chance of that city having another flood of similar magnitude in Year 2 is 1/500, because the odds are independent and we started monitoring the city in Year 1. The chance of it flooding again the third year brings the odds down to 1/(500 * 500) = 1 in 250,000. [0] https://en.wikipedia.org/wiki/List_of_United_States_cities_by_population https://en.wikipedia.org/wiki/List_of_United_States_cities_b...

rothbardrand 9y ago

Having lived in Houston, this isn't a measure of the flood but a measure of how terrible the infrastructure is. They have terrible flood management systems but are "rated" to handle everything but a "100" year flood... yet Houston gets 100 year floods every year- multiple ones- and has for 40 years. It's a scam involving the ratings much like the mooody's ratings scam of the last decade IMNSHO

danmaz74 9y ago

How do you count "floods"? If there is one hurricane that causes floods in 3 cities, is that 1 flood, or 3? If it's 1 - which is what I think is meant when saying "a 1 in XXX years flood" - then your comparison doesn't hold, because the number of floods isn't dependent on the number of cities, but on the distribution of cities on the land, and specifically in flood-prone areas.

Spooky23 9y ago
IMO you need to look at something other than city limits. Cities like Houston and LA are huge cities geographically. IMO you need to look at regional and per capita impacts.

epitomix 9y ago

I think that this statistic only makes sense if it is specific to the location. Otherwise, it is useless for planning purposes. If it is specific to the location then the birthday paradox does not apply. Interesting note, this is Houstons third 500-year flood in three years [1]. [1] -https://www.washingtonpost.com/news/wonk/wp/2017/08/29/houston-is-experiencing-its-third-500-year-flood-in-3-years-how-is-that-possible https://www.washingtonpost.com/news/wonk/wp/2017/08/29/houst...

oso2k 9y ago

The "Birthday Problem" has a simple explanation. Something like nearly 45% of people are born in the Summer months. Ensure that you get people of different ages and your chances of finding someone with similar/same birthday goes up enormously.

brewdad 9y ago
Is this true? Which months are considered "summer"? Northern Hemisphere or Southern or both? This can easily turn into a "40% of sick days are on Monday or Friday" type fallacy.
pmalynin 9y ago
Thats....not how that works. At all.
bitwize 9y ago
The birthday problem still exists even assuming that everyone has an equal chance of being born on any day of the year. It's a phenomenon of the statistical likelihood that given a large enough group of people, the likelihood that any two will have the same birthday goes to almost certainty very rapidly.
umanwizard 9y ago
No, the birthday problem still works if you consider that the probability of being born on a particular day is uniform. It's related to the fact that the probability of two people in a room having a birthday is more closely related to the number of pairs of people in the room (which in turn grows as O(n^2)), than to the number of people in the room. A detailed proof/explanation can be looked up easily with Google, if you have a bit of mathematical maturity.
oso2k 9y ago
For those who down voted me, you do not know your sample set or your test subjects. Go to the empirical data sources. Human behavior certainly affects the real probabilities. As of 2015 in the US [0], a person is more likely to be born in July, August or September. They're also more likely to be born during the middle of the work week rather than the weekends. Draw your own conclusions from the data. [0] https://www.cdc.gov/nchs/data/nvsr/nvsr66/nvsr66_01_tables.pdf https://www.cdc.gov/nchs/data/nvsr/nvsr66/nvsr66_01_tables.p...

curiousgeorgio 9y ago

A more accurate term for what you describe would be a clustering illusion[1], and it certainly could be a valid explanation (among other possibilities) for the observation made in the article. [1] https://en.wikipedia.org/wiki/Clustering_illusion https://en.wikipedia.org/wiki/Clustering_illusion

honestoHeminway 9y ago
Yes, or as with nuclear power, it could be that the base assumption regarding safety just was wrong.

simonebrunozzi 9y ago

Ahem, no - your birthdays example can't apply here. When you increase the number of people in the room, each one of them benefits from every new person added, in terms of probability of having been born on the same day. In the flood case, a flood in California and one in Miami don't really increase the chance of another flood simply because they have happened.

bnegreve 9y ago

> Yet, if you have 30 people in a room, there is a greater than 50 percent chance that at least one PAIR has the same birthday. Not quite the same because Huston is not the only city that has experienced extreme climate these two years, many other cities also have (maybe not 500 years flood, but close). So that would be like: one pair is born the same day, and 20 other people in the room are born a week around that day. Hardly a coincidence. It's more likely that there was a party about 9 months and X years before that day. (I'm not sure how far we can go with this silly birthday analogy:)