3 ms·

On the first problem, if you’re already going to estimate the answer with finite iterations over an infinite series, I think you should just skip ahead to the s

On the first problem, if you’re already going to estimate the answer with finite iterations over an infinite series, I think you should just skip ahead to the simulation that was used to test it. The header for the test section says “Validation” but that’s not what’s happening. It’s not taking some answer and seeing if it’s correct. It’s coming to the answer in a different way, in a way that’s faster to write, faster to run and easier to understand than the other code. Easier to extend and modify, too. Uneven weights for the toys? Easy. Want the P99 case? The standard deviation? Easy. Simulate it ten million times and see what comes out. I’ve just been hyped on simulations over calculations for probabilities lately and this is wonderful confirmation bias.

danuker 4y ago
"The Unreasonable Effectiveness of Monte Carlo Methods"
- LudwigNagasena 4y ago
  "The Unreasonable Effectiveness of reading a statistics textbook." There is enough material for, like, 100 blogposts or maybe even more!
LegionMammal978 4y ago
Depending on how much precision you'd like for your answer, Monte Carlo methods can converge unbearably slowly, since the standard deviation is proportional to 1/sqrt(N) as you add more samples. I recall once having to run 100 billion simulations to get the mean of a certain quantity to within a few decimal places. I'd take an infinite series over that ant day of the week (well, as long as it's not something as slow as the Leibniz series for pi/4). Of course, such precision relative to the standard deviation is rarely necessary for most practical situations.
- dekhn 4y ago
  As a friend in grad school used to say... "An infinitely long Monte Carlo simulation recapitulates the underlying probability distribution perfectly, but we don't want to wait that long.":
- travisjungroth 4y ago
  Yeah, it’s the practicality. Most business decisions don’t need more than three sigfig [source?]. We only tend to think about precision on things that already have it.
  - 0cf8612b2e1e 4y ago
    Unless you are trying to prove the existence of a subatomic particle, I am not sure what business decisions have anything close to that level of certainty.
    - travisjungroth 4y ago
      Kinda depends on what you mean by "business decisions". Repetition can bring in a high need for confidence. You don't want to have your life depend on a piece of equipment that has a one in a million chance of failure if it gets tested a thousand times an hour. Pricing things can also need a lot of certainty, like in HFT. But most things aren't either of those cases and people are commonly off by an order of magnitude.
jrumbut 4y ago
Yeah I think simulations are massively underused. They might not help you prepare for a black swan type event, but a lot of times we're not ready for very normal problems. I suspect a day or two per year spent doing basic modeling and simulation would benefit a lot of teams, even back of the envelop stuff. Then you can return to the exercise a year later and see if your assumptions were on point or what turned out to be much different than expected and refine the model and parameters of the simulation.
- college_physics 4y ago
  > They might not help you prepare for a black swan type event at some point probabilistic simulations were all the rage in the financial industry and then the financial crisis happened it is a fine art to figure out how to elicit the benefits of Monte Carlo simulation to put some structure around the future without falling into a mental trap that blinds you to non-quantifiable scenarios
- travisjungroth 4y ago
  They're actually great for preparing black swan events. Taleb talks about them in "The Black Swan". Usually the rules of the game aren't as clearly stated as in this problem. Sometimes there is a literal coupon contest, or a node randomly pairs with another node. But usually you're looking at history to try to to understand the odds and then forecast some futures. The problem with doing that with statistics is it's not very receptive to "fiddling", even if it's Bayesian. With a simulation it's generally easier to throw something weird in there, like an effect orders of magnitude larger, dynamic effects, or a totally new rule.
0cf8612b2e1e 4y ago
Just making a simulation and codifying assumptions is hugely helpful. Forcing SMEs to rationalize a number and its spread has so much more utility than a closed form calculation which people have to take on faith.

danuker 4y ago

"The Unreasonable Effectiveness of Monte Carlo Methods"

LudwigNagasena 4y ago
"The Unreasonable Effectiveness of reading a statistics textbook." There is enough material for, like, 100 blogposts or maybe even more!

LegionMammal978 4y ago

Depending on how much precision you'd like for your answer, Monte Carlo methods can converge unbearably slowly, since the standard deviation is proportional to 1/sqrt(N) as you add more samples. I recall once having to run 100 billion simulations to get the mean of a certain quantity to within a few decimal places. I'd take an infinite series over that ant day of the week (well, as long as it's not something as slow as the Leibniz series for pi/4). Of course, such precision relative to the standard deviation is rarely necessary for most practical situations.

dekhn 4y ago
As a friend in grad school used to say... "An infinitely long Monte Carlo simulation recapitulates the underlying probability distribution perfectly, but we don't want to wait that long.":
travisjungroth 4y ago
Yeah, it’s the practicality. Most business decisions don’t need more than three sigfig [source?]. We only tend to think about precision on things that already have it.
- 0cf8612b2e1e 4y ago
  Unless you are trying to prove the existence of a subatomic particle, I am not sure what business decisions have anything close to that level of certainty.
  - travisjungroth 4y ago
    Kinda depends on what you mean by "business decisions". Repetition can bring in a high need for confidence. You don't want to have your life depend on a piece of equipment that has a one in a million chance of failure if it gets tested a thousand times an hour. Pricing things can also need a lot of certainty, like in HFT. But most things aren't either of those cases and people are commonly off by an order of magnitude.

jrumbut 4y ago

Yeah I think simulations are massively underused. They might not help you prepare for a black swan type event, but a lot of times we're not ready for very normal problems. I suspect a day or two per year spent doing basic modeling and simulation would benefit a lot of teams, even back of the envelop stuff. Then you can return to the exercise a year later and see if your assumptions were on point or what turned out to be much different than expected and refine the model and parameters of the simulation.

college_physics 4y ago
> They might not help you prepare for a black swan type event at some point probabilistic simulations were all the rage in the financial industry and then the financial crisis happened it is a fine art to figure out how to elicit the benefits of Monte Carlo simulation to put some structure around the future without falling into a mental trap that blinds you to non-quantifiable scenarios
travisjungroth 4y ago
They're actually great for preparing black swan events. Taleb talks about them in "The Black Swan". Usually the rules of the game aren't as clearly stated as in this problem. Sometimes there is a literal coupon contest, or a node randomly pairs with another node. But usually you're looking at history to try to to understand the odds and then forecast some futures. The problem with doing that with statistics is it's not very receptive to "fiddling", even if it's Bayesian. With a simulation it's generally easier to throw something weird in there, like an effect orders of magnitude larger, dynamic effects, or a totally new rule.

0cf8612b2e1e 4y ago

Just making a simulation and codifying assumptions is hugely helpful. Forcing SMEs to rationalize a number and its spread has so much more utility than a closed form calculation which people have to take on faith.