3 ms·
I honestly don't think R and Python would have worked here. While I was able to clone the repo and run it for Ireland, the UK requires at least 26Gb of RAM, wh
by disgruntledphd2 6y ago
I honestly don't think R and Python would have worked here.
While I was able to clone the repo and run it for Ireland, the UK requires at least 26Gb of RAM, which is not common on most personal computers. The US requires much, much more.
And given that it's pretty slow when written in C++, imagine how slow it would have been in R or Python?
I agree with you in principle with respect to this stuff being better, but the incentives skew otherwise at the moment.
- Nursie 6y agoGiven we're talking about something that it seems people say does not have good code quality, I would guess that whether the C++ implementation is more efficient due to language choice would be a wide open question.
- silveraxe93 6y agoAny serious attempt at modelling this over python would use the pydata stack (numpy, pandas, etc), which run on top of C++ anyways.
- disgruntledphd2 6y agoYeah of course, apologies if that wasn't clear. The best solution here would probably be to package up the core routines into a library and use this from either R or Python.
- silveraxe93 6y agoSorry if I came out a bit snippy out there. But yeah I assumed you meant python without numpy, etc. A lot of the criticism I saw was because the core routines did not need to be packaged up. There were a lot of common data structures reimplemented, etc. I don't think the model had many novel routines. It could be built just using industry standard and tested tools in python, R, Julia (if you really want speed) etc. But it reinvented the whole ecosystem in one big ball of C. tbh, this should have been built on STAN or similar. There's so many variables and assumptions that the output is completely dominated by the parameters chosen. Seeing the distribution of outcomes instead of a point estimate would be actually useful.
- disgruntledphd2 6y agoTrue, I think that's worth noting. To be fair though, this codebase is pretty old, and it's unlikely that the technology landscape looked much like today (especially in terms of R and Python), so I can see how they ended up here. I'd love if this was written in R, Python or Stan so I could contribute, but that's probably not the researchers focus ;) While Stan is amazing, I shudder to think as to how long this model would have taken to run using MCMC (1 week plus maybe?).