4 ms·

Show HN: Fast Monte Carlo Permutation Simulator in C

28 points by chrisconley 13y ago

minimax 13y ago
Cool project. Can you say how much performance improvement you are getting with dSFMT versus a naive C loop? Another fast monte carlo anecdote: at Bloomberg they are doing this sort of thing with GPUs. http://www.wallstreetandtech.com/it-infrastructure/bloomberg-uses-gpus-to-speed-up-bond-pri/220200055 http://www.wallstreetandtech.com/it-infrastructure/bloomberg...
- chrisconley 13y ago
  I'm trying to parse my HipChat conversation with our Architect (who did most of the speed improvements) and it looks like going from Python to naive C loop was a 200x improvement (though that's not a direct comparison of course). Going from naive C to dSFMT we went from one particular simulation taking 14:30 down to 2:30 (minutes).
marmaduke 13y ago
You could have stayed in Python and used PyOpenCL or Numba or weave inline or f2py or... What is the real reason to use C here?
- deleted 13y ago
  [deleted]
- chrisconley 13y ago
  Interesting, I'll have to take a look at those. The real reason is that it was just an idea that we came up with and decided to try it. It worked well, so we stuck with it. :)
- sanskritabelt 13y ago
  ??? where did "f2py" come from in this list?
mrcactu5 13y ago
Is the random number generator the bottleneck for significance-testing experiments like this? Monte-Carlo is more meaningful than using some sort of chi-squared. But why is it faster?
sanskritabelt 13y ago
From: "If we see that 1000 of the 10,000 random iterations had a difference of more than $1.50, we can say that there is a 10% chance that our $1.50 observed difference was due to randomness." I'm wondering if maybe you're doing way too many simulations in the calibration, do you really need more than a few hundred to a thousand or so? The 0.1 quantile is reasonably well separated from 0, and I would have expected you'd get "good enough" convergence pretty quickly. Also: "we can compare our computed p-value of 11% to our simulated 10% result to determine whether or not the model is accurate enough." you're getting a full pdf out of the simulations, are you also comparing to the full distribution of your test statistics?

28 points by chrisconley 13y ago

Cool project. Can you say how much performance improvement you are getting with dSFMT versus a naive C loop? Another fast monte carlo anecdote: at Bloomberg they are doing this sort of thing with GPUs. http://www.wallstreetandtech.com/it-infrastructure/bloomberg-uses-gpus-to-speed-up-bond-pri/220200055 http://www.wallstreetandtech.com/it-infrastructure/bloomberg...

chrisconley 13y ago
I'm trying to parse my HipChat conversation with our Architect (who did most of the speed improvements) and it looks like going from Python to naive C loop was a 200x improvement (though that's not a direct comparison of course). Going from naive C to dSFMT we went from one particular simulation taking 14:30 down to 2:30 (minutes).

marmaduke 13y ago

You could have stayed in Python and used PyOpenCL or Numba or weave inline or f2py or... What is the real reason to use C here?

deleted 13y ago
[deleted]
chrisconley 13y ago
Interesting, I'll have to take a look at those. The real reason is that it was just an idea that we came up with and decided to try it. It worked well, so we stuck with it. :)
sanskritabelt 13y ago
??? where did "f2py" come from in this list?

mrcactu5 13y ago

Is the random number generator the bottleneck for significance-testing experiments like this? Monte-Carlo is more meaningful than using some sort of chi-squared. But why is it faster?

sanskritabelt 13y ago

From: "If we see that 1000 of the 10,000 random iterations had a difference of more than $1.50, we can say that there is a 10% chance that our $1.50 observed difference was due to randomness." I'm wondering if maybe you're doing way too many simulations in the calibration, do you really need more than a few hundred to a thousand or so? The 0.1 quantile is reasonably well separated from 0, and I would have expected you'd get "good enough" convergence pretty quickly. Also: "we can compare our computed p-value of 11% to our simulated 10% result to determine whether or not the model is accurate enough." you're getting a full pdf out of the simulations, are you also comparing to the full distribution of your test statistics?