4 ms·
I'm trying to be charitable on this code base issue, and the institutions will need to accept they need to be able to bear more scrutiny on the inputs to policy
by motohagiography 6y ago
I'm trying to be charitable on this code base issue, and the institutions will need to accept they need to be able to bear more scrutiny on the inputs to policy recommendations.
Both R and Python are taught in highschool and undergraduate courses for scientific data analysis. Ferguson and other scientists did not need software engineers. A free co-op student, or to have spent the three weeks learning a language that met the needs of the level of abstraction he was working at would have sufficed.
The culture that enables that code as described to be acceptable for policymaking is one that intentionally produces complex black boxes that obfuscate risk and attribution, and to launder decision accountability through technology. I've seen this in other institutional code as well.
However, while the policy recommendations that resulted from his model may (or may not) have saved tens of thousands of lives, it did so at the risk of losing the credibility to do it again.
Deflecting blame to nebulous software engineers is disingenuous and serves mainly to exacerbate the suspicions of reasonable people, and further polarize those most harmed by the policy response.
- disgruntledphd2 6y agoI honestly don't think R and Python would have worked here. While I was able to clone the repo and run it for Ireland, the UK requires at least 26Gb of RAM, which is not common on most personal computers. The US requires much, much more. And given that it's pretty slow when written in C++, imagine how slow it would have been in R or Python? I agree with you in principle with respect to this stuff being better, but the incentives skew otherwise at the moment.
- Nursie 6y agoGiven we're talking about something that it seems people say does not have good code quality, I would guess that whether the C++ implementation is more efficient due to language choice would be a wide open question.
- silveraxe93 6y agoAny serious attempt at modelling this over python would use the pydata stack (numpy, pandas, etc), which run on top of C++ anyways.
- disgruntledphd2 6y agoYeah of course, apologies if that wasn't clear. The best solution here would probably be to package up the core routines into a library and use this from either R or Python.
- silveraxe93 6y agoSorry if I came out a bit snippy out there. But yeah I assumed you meant python without numpy, etc. A lot of the criticism I saw was because the core routines did not need to be packaged up. There were a lot of common data structures reimplemented, etc. I don't think the model had many novel routines. It could be built just using industry standard and tested tools in python, R, Julia (if you really want speed) etc. But it reinvented the whole ecosystem in one big ball of C. tbh, this should have been built on STAN or similar. There's so many variables and assumptions that the output is completely dominated by the parameters chosen. Seeing the distribution of outcomes instead of a point estimate would be actually useful.
- disgruntledphd2 6y agoTrue, I think that's worth noting. To be fair though, this codebase is pretty old, and it's unlikely that the technology landscape looked much like today (especially in terms of R and Python), so I can see how they ended up here. I'd love if this was written in R, Python or Stan so I could contribute, but that's probably not the researchers focus ;) While Stan is amazing, I shudder to think as to how long this model would have taken to run using MCMC (1 week plus maybe?).
- Mikhail_K 6y agoPython is unsuitable for writing Monte-Carlo simulations due to its abysmal performance and built-in unfixable inability for parallelism. In addition, it is not a good language in general. Complex simulations should be written in multiple-dispatch high-perfomance languages like Fortran or Julia.