3 ms·

You're asking a scientific question. But it's not just a science question. That is the point of the criticism. Assuming you are the Matthew D Green known for

You're asking a scientific question. But it's not just a science question. That is the point of the criticism. Assuming you are the Matthew D Green known for his cryptography work, surely you of all people understand that a poor implementation can have profound consequences even if the original mathematics is sound?

matthewdgreen 6y ago
I am, and I've written both production and non-production software. I know that every piece of software has limited development resources, and has different demands on those resources. We're in the midst of a pandemic right now and demands are particularly acute for fresh simulation data, with simulation parameters changing constantly based on new information and policy proposals. So what do you do with your limited resources? Do you tell the government you can't get them any preliminary data because your multithreading system doesn't work, and you need four months to refactor the code (even though you could just run single-core)? Do you tell them that because you have to write a complete unit testing framework, you can't give them any modeling data or update your model parameters, sorry, good luck with everything. Do you tell them that another 30K people may have to die based on wrong policy choices because you have decided to prioritize "shipping-production-quality" software standards, even though you are not shipping your software? You can pretend that this is not a trolley problem. But in this case it literally is. You are standing at the switch in the tracks, and on one side is a detailed set of unit tests and multi-core support, and on the other hand might be tens of thousands of living people. And you know your code better than anyone else in the world. What do you choose? I don't have an answer. I'm saying the answer is hard, and we're not equipped to lay claims on it based on glancing at a Git repo.
- Silhouette 6y ago
  I'm not arguing that there is much you can usefully change now. It's probably far too late to make much material difference to public policy now, for the reasons you have set out yourself. But this could surely have been much better handled during a usefully long window back towards the start of the year when attention was starting to be paid to the virus but it hadn't taken hold in the UK on any significant scale yet. Also, just to be clear, all of the specific issues with the Imperial code that you've mentioned here and in other parts of our discussion today, such as whether to rewrite for better multi-core behaviour or just use single-core, are coming from you not me. My argument is a much simpler one: when it started to become clear that this virus was a significant risk, the government should have provided the necessary resources and the expert advisors should have switched to a model where scientists did the science and programmers did the programming. I don't think anyone can credibly claim that there wasn't enough awareness that this could potentially become a serious issue and so that bringing in broader expertise was appropriate by the time we had things like COBRA meetings being held and secret expert advisory groups being assembled.
- thu2111 6y ago
  You don't appear to know much about the history of this program, as you keep defending it on the basis it was written quickly - and that the alternative is no modelling. It started being developed in 2007. It's by now over a decade old, with a lot of work done in 2014. This program wasn't written quickly, it was re-parameterised with a few values for COVID and otherwise thinks it's simulating the flu. Some other pieces of code date back multiple decades. You also appear to think that "limited resources" is a valid excuse. The UK science budget is over 8 billion pounds. Universities there churn out a large number of papers. The funding is there if they wanted to do less work but of higher quality, yet they don't. And finally, just to make that clear: yes, saying "We don't have a reliable model for you" is an entirely valid thing to say to a government. You seem to think even low quality simulations are better than none. Humanity has been fighting disease for thousands of years without bug-ridden 15,000 LOC C programs to "help" us. Not having an academic model is a completely reasonable position to be in. In particular there's a lot of evidence this particular model has already killed a lot of people: it predicted a surge in hospital needs that never happened. In anticipation of this surge a lot of elderly patients that had tested positive for COVID were discharged back into care homes, because the hospitals thought they'd need the beds within days. Those elderly people then infected the others in the care homes, causing the death rate to go up. If there had been no model, there'd have been no panic-induced mass discharge, patients would have been kept in wards until they were fully recovered and the virus wouldn't have been flushed straight into the populations of the most vulnerable. Outcomes could have been much better as a result.
- c0sin 6y ago
  > write a complete unit testing framework Why? Aren't there already frameworks for unit testing? Didn't they have over a decade to write one? > Do you tell them that another 30K people may have to die based on wrong policy choices "How many more people should die before...." - where have we heard this, ooh just a couple thousand times before? Here's a simple fact, if you want to go this route: every extra 1% of unemployment rate (for US) leads to extra ~40,000 death a year due to a whole stew of reasons. So, which one is better: have a team of professional developers use best practices and existing tools (like yes - testing framework) and minimize what seems to be a result of amateur programming done over the span of 15 years, or release an unreliable yet sensationalist report and cause tens of millions of people loose their jobs and livelihood? @matthewdgreen , with all due respect you're avoiding answering to concrete questions and keep defending lousy programming on the ground of urgency and appeal to emotions. This is a mere demagoguery.

matthewdgreen 6y ago

I am, and I've written both production and non-production software. I know that every piece of software has limited development resources, and has different demands on those resources. We're in the midst of a pandemic right now and demands are particularly acute for fresh simulation data, with simulation parameters changing constantly based on new information and policy proposals. So what do you do with your limited resources? Do you tell the government you can't get them any preliminary data because your multithreading system doesn't work, and you need four months to refactor the code (even though you could just run single-core)? Do you tell them that because you have to write a complete unit testing framework, you can't give them any modeling data or update your model parameters, sorry, good luck with everything. Do you tell them that another 30K people may have to die based on wrong policy choices because you have decided to prioritize "shipping-production-quality" software standards, even though you are not shipping your software? You can pretend that this is not a trolley problem. But in this case it literally is. You are standing at the switch in the tracks, and on one side is a detailed set of unit tests and multi-core support, and on the other hand might be tens of thousands of living people. And you know your code better than anyone else in the world. What do you choose? I don't have an answer. I'm saying the answer is hard, and we're not equipped to lay claims on it based on glancing at a Git repo.

Silhouette 6y ago
I'm not arguing that there is much you can usefully change now. It's probably far too late to make much material difference to public policy now, for the reasons you have set out yourself. But this could surely have been much better handled during a usefully long window back towards the start of the year when attention was starting to be paid to the virus but it hadn't taken hold in the UK on any significant scale yet. Also, just to be clear, all of the specific issues with the Imperial code that you've mentioned here and in other parts of our discussion today, such as whether to rewrite for better multi-core behaviour or just use single-core, are coming from you not me. My argument is a much simpler one: when it started to become clear that this virus was a significant risk, the government should have provided the necessary resources and the expert advisors should have switched to a model where scientists did the science and programmers did the programming. I don't think anyone can credibly claim that there wasn't enough awareness that this could potentially become a serious issue and so that bringing in broader expertise was appropriate by the time we had things like COBRA meetings being held and secret expert advisory groups being assembled.
thu2111 6y ago
You don't appear to know much about the history of this program, as you keep defending it on the basis it was written quickly - and that the alternative is no modelling. It started being developed in 2007. It's by now over a decade old, with a lot of work done in 2014. This program wasn't written quickly, it was re-parameterised with a few values for COVID and otherwise thinks it's simulating the flu. Some other pieces of code date back multiple decades. You also appear to think that "limited resources" is a valid excuse. The UK science budget is over 8 billion pounds. Universities there churn out a large number of papers. The funding is there if they wanted to do less work but of higher quality, yet they don't. And finally, just to make that clear: yes, saying "We don't have a reliable model for you" is an entirely valid thing to say to a government. You seem to think even low quality simulations are better than none. Humanity has been fighting disease for thousands of years without bug-ridden 15,000 LOC C programs to "help" us. Not having an academic model is a completely reasonable position to be in. In particular there's a lot of evidence this particular model has already killed a lot of people: it predicted a surge in hospital needs that never happened. In anticipation of this surge a lot of elderly patients that had tested positive for COVID were discharged back into care homes, because the hospitals thought they'd need the beds within days. Those elderly people then infected the others in the care homes, causing the death rate to go up. If there had been no model, there'd have been no panic-induced mass discharge, patients would have been kept in wards until they were fully recovered and the virus wouldn't have been flushed straight into the populations of the most vulnerable. Outcomes could have been much better as a result.
c0sin 6y ago
> write a complete unit testing framework Why? Aren't there already frameworks for unit testing? Didn't they have over a decade to write one? > Do you tell them that another 30K people may have to die based on wrong policy choices "How many more people should die before...." - where have we heard this, ooh just a couple thousand times before? Here's a simple fact, if you want to go this route: every extra 1% of unemployment rate (for US) leads to extra ~40,000 death a year due to a whole stew of reasons. So, which one is better: have a team of professional developers use best practices and existing tools (like yes - testing framework) and minimize what seems to be a result of amateur programming done over the span of 15 years, or release an unreliable yet sensationalist report and cause tens of millions of people loose their jobs and livelihood? @matthewdgreen , with all due respect you're avoiding answering to concrete questions and keep defending lousy programming on the ground of urgency and appeal to emotions. This is a mere demagoguery.