3 ms·

Most scientific software developers have been developing and working with enormously complex codebases for years. You're clearly starting from the perspective t

Most scientific software developers have been developing and working with enormously complex codebases for years. You're clearly starting from the perspective that they're eggheads who have no experience with this area; that assumption is almost certainly wrong.

OneGuy123 6y ago
"Working with and enormous codebase" and "actualing ensuring it works" are two very different things. Academic "scientific software developers" (lol) never have to prove anything works in production long term in majority of cases. They make a one-off project and never touch it again. So this generlization stands. And yes, I am saying that they are eggheads. Because this article and the covid sim clearly shows that "scientific software developers" (lol) must not be trusted with real world-affecting systems without some other real world developer checking it. The fact is that the covid sim is garbage and anyone who tries to defend it does not understand programming and complex systems well enough.
- matthewdgreen 6y ago
  Scientists have to undergo extremely rigorous peer review from other experts who are massively incentivized to challenge them and show that their results are wrong. The individual code may not be reviewed, but spurious results will be noticed and called out. The closest thing we have to this in industry is bug reporting and software security evaluation, and this is sporadic at best. Most commercial software developers keep their code closed source and do everything they can to stifle public knowledge of bugs, even after they've been fixed.
  - thu2111 6y ago
    Why are you defending this work Matthew? You have your own reputation to defend as well. This code has never been peer reviewed, ever. That is indisputable because Ferguson has claimed multiple times that the code is/was entirely undocumented, and nobody outside him and his few colleagues understands how it works. The moment it started being actually reviewed, people found a breathtaking number of bugs like out-of-bounds reads, typos in random number generator constants etc. Broken RNGs! Of all things I'd hope an expert in cryptography would be very worried by, a missing digit in an RNG constant should surely be one of them?
Silhouette 6y ago
Most scientific software developers have been developing and working with enormously complex codebases for years. I've spent many years of my career working on highly technical software. I've worked in offices full of professional programmers where many also had postgraduate qualifications in STEM subjects. Those teams did indeed work on quite complex codebases for years. Do you know what else they typically had? Testing infrastructure that was ahead of its time, careful validation of the mathematics being used before it was implemented, a reasonably clean and systematic software design, and good traceability from the mathematical code back to the underlying mathematics. This is not rocket science. This is not even unusual if you work in this field. It's just what happens when you let professionals with an appropriate skill set do the job: programmers with technical backgrounds implement models constructed by subject matter experts. Having one set of skills without the other is not a recipe for success.
- matthewdgreen 6y ago
  Then you and your team should take over the epidemic modeling software development, there's a real need. I assume you can deliver on an academic budget and produce high-quality code in time for it to be useful in this pandemic? Because if you cannot do those things, you're not really contributing anything of any quality.
  - Silhouette 6y ago
    The total cost of the pandemic to this country could plausibly be approaching the trillions of pounds scale by the time we're through the worst of it. I think the government could have sprung for a few software developers who knew what they were doing back at the start of the year. Apparently they'd already had plenty of time to establish a relatively large group of privileged expert advisors, including some who worked on these kinds of models. Perhaps if they'd been more open about that process, the gap in their capabilities could have been pointed out soon enough to do something useful about it. You write as if you think what I'm describing is somehow unusual. It is not. I have probably worked with at least 30-40 well-qualified people just in my own career from whom any well-mixed team of say 3-5 developers could have done what we're talking about here within a relevant time frame. We are not talking about rocket science, just good software development practices being employed by experienced programmers with technical backgrounds who are used to implementing the models of subject matter experts. This isn't some magical unicorn skill set.

OneGuy123 6y ago

"Working with and enormous codebase" and "actualing ensuring it works" are two very different things. Academic "scientific software developers" (lol) never have to prove anything works in production long term in majority of cases. They make a one-off project and never touch it again. So this generlization stands. And yes, I am saying that they are eggheads. Because this article and the covid sim clearly shows that "scientific software developers" (lol) must not be trusted with real world-affecting systems without some other real world developer checking it. The fact is that the covid sim is garbage and anyone who tries to defend it does not understand programming and complex systems well enough.

matthewdgreen 6y ago
Scientists have to undergo extremely rigorous peer review from other experts who are massively incentivized to challenge them and show that their results are wrong. The individual code may not be reviewed, but spurious results will be noticed and called out. The closest thing we have to this in industry is bug reporting and software security evaluation, and this is sporadic at best. Most commercial software developers keep their code closed source and do everything they can to stifle public knowledge of bugs, even after they've been fixed.
- thu2111 6y ago
  Why are you defending this work Matthew? You have your own reputation to defend as well. This code has never been peer reviewed, ever. That is indisputable because Ferguson has claimed multiple times that the code is/was entirely undocumented, and nobody outside him and his few colleagues understands how it works. The moment it started being actually reviewed, people found a breathtaking number of bugs like out-of-bounds reads, typos in random number generator constants etc. Broken RNGs! Of all things I'd hope an expert in cryptography would be very worried by, a missing digit in an RNG constant should surely be one of them?

Silhouette 6y ago

Most scientific software developers have been developing and working with enormously complex codebases for years. I've spent many years of my career working on highly technical software. I've worked in offices full of professional programmers where many also had postgraduate qualifications in STEM subjects. Those teams did indeed work on quite complex codebases for years. Do you know what else they typically had? Testing infrastructure that was ahead of its time, careful validation of the mathematics being used before it was implemented, a reasonably clean and systematic software design, and good traceability from the mathematical code back to the underlying mathematics. This is not rocket science. This is not even unusual if you work in this field. It's just what happens when you let professionals with an appropriate skill set do the job: programmers with technical backgrounds implement models constructed by subject matter experts. Having one set of skills without the other is not a recipe for success.

matthewdgreen 6y ago
Then you and your team should take over the epidemic modeling software development, there's a real need. I assume you can deliver on an academic budget and produce high-quality code in time for it to be useful in this pandemic? Because if you cannot do those things, you're not really contributing anything of any quality.
- Silhouette 6y ago
  The total cost of the pandemic to this country could plausibly be approaching the trillions of pounds scale by the time we're through the worst of it. I think the government could have sprung for a few software developers who knew what they were doing back at the start of the year. Apparently they'd already had plenty of time to establish a relatively large group of privileged expert advisors, including some who worked on these kinds of models. Perhaps if they'd been more open about that process, the gap in their capabilities could have been pointed out soon enough to do something useful about it. You write as if you think what I'm describing is somehow unusual. It is not. I have probably worked with at least 30-40 well-qualified people just in my own career from whom any well-mixed team of say 3-5 developers could have done what we're talking about here within a relevant time frame. We are not talking about rocket science, just good software development practices being employed by experienced programmers with technical backgrounds who are used to implementing the models of subject matter experts. This isn't some magical unicorn skill set.