6 ms·

Yeah but pharma in my personal experience is VERY conservative. A lot of them still use SAS as their bread and butter for analysis. And guess what? All their ol

by skwb 6y ago

Yeah but pharma in my personal experience is VERY conservative. A lot of them still use SAS as their bread and butter for analysis. And guess what? All their older trials that they have to maintain are all written in SAS. It's basically how banking got stuck in a lot of legacy COBOL code, pharma got stuck in SAS. Want to do healthcare economic modeling? Buy a license for TreeAge (and it ain't cheap!). And it's not like you cannot do complex analysis in it, it just doesn't have a sleek interface. The "biggest" technology disputer in pharma I would argue would be Salesforce! They support creating all tons of custom input interfaces for patient consenting and record keeping. Enterprise ain't supposed to be sexy! I just sorta worry if there is correct culture fit in pharmaceuticals when there is a prevailing mentality of "no one ever got fired for buying Microsoft".

KenoFischer 6y ago
You're not wrong - Pharma is quite conservative. I wasn't convinced the the Pumas folks had any hope, but apparently the legacy workflow for what they have is so aweful that people are aching for something better. I'm not in the day to day, but from my perspective Pumas seems to be quite successful and well received in Pharma, despite the conservatism.
ViralBShah 6y ago
That might be true about some pharma software, but that does not mean there's no appetite for innovation. A lot of the computational work is done with decades old Fortran libraries, plugged into R (yup, no Python in this world) and spreadsheets. It is very hard to upgrade these old Fortran libraries to leverage capabilities of modern numerical stacks. The reason why companies have to hold on to their SAS and COBOL is that once a drug is out in the real world, and there is a lawsuit, or a recall - you need to be able to have everything available for review. These things can happen 20 or 30 years after a drug becomes available. That would naturally induce risk aversion. New drugs today will use newer tools, and 30 years out, we'll be saying the same things about today's tools. In fact pharma companies have no choice but to adapt. Technology moves fast, and anything that gives an edge, if not adopted, will leave you behind.
- fock 6y ago
  What are these old Fortran libraries doing, which requires "new numerical stacks"? From my naive point of view, all what you are describing should be trivially done on a workstation. As I'm sure that's probably me underestimating the field: can you point me to some algorithm/technique which would benefit from new numerical stacks? Also is your stack open source? I somehow doubt the long-term advantage of one proprietary stack over another.
  - DrScientist 6y ago
    Winnonlin is one such tool - been around forever, written in fortran. One of the reasons it is still popular is for regulatory use it's very well understood. I'm not enough of an expert to answer the second part of your question - on why new numerical stacks are beneficial. It may be around building more complex models of PK/PD but I'm not up to speed on what winnonlin can and can't do.
  - ChrisRackauckas 6y ago
    Pharmacometrics is built around small stiff ODE models. You might think that this might mean it's "simple" computationally, but what it actually means is that every little detail matters a lot for performance. The way you do lu-factorization for 4 ODEs vs 6 ODEs needs to change in an architecture-dependent fashion, and BLAS libraries are not efficient in this range. Standard floating point exponentiation can be too expensive in some spots, while at the same time the standard stiff ODE solvers make assumptions about regularity which are regularly violated by pharmacometric models (due to how dosing works). There is more than a little bit of "by-hand SIMD" in this software stack. Specializing on all of these features is an interplay between algorithms and JIT compilation, where not utilizing statically compiled optimizations will hurt you at this size. We see this in benchmarks like the SciML Hires [1] which is able to outperform the classic Fortran libraries like LSODA with newly developed Rodas methods. As a less direct and more illustrative example, these benchmarks vs PyTorch [2] demonstrate what happens when you compare against an optimizing JIT compiler which doesn't specialize on small sized interactions (>30x performance difference!), showing how this kind of application is very much not in the regime of large kernels which a lot of recent compiler work tends to optimize for and instead heavily relies on being able to cross-compile and inline computations to remove every little overhead, all while improving the ODE algorithms themselves. We will have a paper that goes into more detail on this fairly soon. [1] https://benchmarks.sciml.ai/html/StiffODE/Hires.html [2] https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e0584a2bfd320
videep90 6y ago
Pumas-AI co-founder here. Indeed, Pharma is conservative due to the impact it can have on patient lives (good or bad). If something is working, it stays, and something failed them, it gets axed, both people and tools. Having said that, let's give them some credit, as along with the regulators at the FDA, pharma has come a long way from the days of SAS. Even the regulators at the FDA have started gravitating away from SAS into more R-based workflows. While it was a good switch away from SAS, scientists started realizing the bottlenecks with R and the requirement to learn multiple tools to get things working, especially when one has to scale up for the new age analytics. While it is good to have a swiss army knife of tools, there is more overhead in making them work together. I think that is why SAS had shined for the longest time. But as you rightly said, it is cost prohibitive and in the hands of very few. Pumas and the Julia ecosystem are about integrated modeling speak across multiple sectors that are composable and the best part is that one does not have to develop in one environment and switch to another for production. Your point about Salesforce may be right for trial operations, but someone has to do the anlaytics that connect back to the first principles of physiology and pharmacology. Pumas provides this connection to first principles from the core, and lets you build on it for both internal decision making or to serve the stakeholders with fancy UI's. A subtle but important point here is that compared to a 30-40 years ago since when SAS became a mainstay, I would argue that more of the younger generation are not averse to coding. In fact programming at the basic level is a fundamental expected skill set, and I personally see more and more life-science majors be efficient coders, thanks to the democratization of tools and education. Combined with this new workforce, and tools like Pumas and Julia, we are not far away from disrupting this sector.
- DrScientist 6y ago
  Why Julia? By choosing a language that a tiny minority know and use, haven't you crippled your adoption from the start?
  - ViralBShah 6y ago
    I think Chris Rackauckas answers "Why Julia" below in https://news.ycombinator.com/item?id=24132382 https://news.ycombinator.com/item?id=24132382 This is a very clear case of a well-defined application. The field of pharma needs a combination of mechanistic models (based on differential equations), statistics, and ability to work with large datasets. The existing tools are far from sufficient. If anything, the need for high performance and scalable tools has been felt in the industry for a very long time. There are hundreds of thousands of Julia programmers worldwide now, and the user base is doubling every year. After all, new ideas do start with a tiny minority, but one that truly believes in those ideas and is enthusiastic in adopting them early.
jogagobburu 6y ago
You are right, a portion of pharma is conservative. There are two different needs in Pharma - one is more like assembly line (pure operations) and other other is "innovation". One has to decide what excites scientists - operations (eg calculating group means over and over where the answer has to be exact) OR innovation (get the questions right, designs right, analysis right, answers approximately accurate). I think there is a place for "operations", but they constitute "sustaining innovations". Meaning can I find the most economical solution for this problem? Operations is a separate discussion. Let us talk about "disruptive innovations". What if I told you I was part of the team that approved a treatment for H1N1 flu pandemic in children without ANY trial based on unapproved data in adults? What if I told you the development pathway for pulmonary arterial hypertension in pediatrics was driven by analysis performed by Pumas-like software to establish a reasonable endpoint? These are patients who cannot perform most daily functions we take for granted. There would have been no drugs approved otherwise. How can you, me and others contribute to the disruptive innovation which will transform how drugs are developed and patients are treated. Pumas is designed to disrupt drug development and precision medicine. There are plenty of opportunities for disruption in healthcare. these opportunities are the motivation for Pumas..Joga Gobburu (Co-Founder, Pumas-AI)
- NicoJuicy 6y ago
  Which country are you tlaking about? ( cfr. "H1N1 flu pandemic in children without ANY trial based on unapproved data in adults")
  - jogagobburu 6y ago
    USA for emergency use.

by skwb 6y ago

KenoFischer 6y ago

You're not wrong - Pharma is quite conservative. I wasn't convinced the the Pumas folks had any hope, but apparently the legacy workflow for what they have is so aweful that people are aching for something better. I'm not in the day to day, but from my perspective Pumas seems to be quite successful and well received in Pharma, despite the conservatism.

ViralBShah 6y ago

That might be true about some pharma software, but that does not mean there's no appetite for innovation. A lot of the computational work is done with decades old Fortran libraries, plugged into R (yup, no Python in this world) and spreadsheets. It is very hard to upgrade these old Fortran libraries to leverage capabilities of modern numerical stacks. The reason why companies have to hold on to their SAS and COBOL is that once a drug is out in the real world, and there is a lawsuit, or a recall - you need to be able to have everything available for review. These things can happen 20 or 30 years after a drug becomes available. That would naturally induce risk aversion. New drugs today will use newer tools, and 30 years out, we'll be saying the same things about today's tools. In fact pharma companies have no choice but to adapt. Technology moves fast, and anything that gives an edge, if not adopted, will leave you behind.

fock 6y ago
What are these old Fortran libraries doing, which requires "new numerical stacks"? From my naive point of view, all what you are describing should be trivially done on a workstation. As I'm sure that's probably me underestimating the field: can you point me to some algorithm/technique which would benefit from new numerical stacks? Also is your stack open source? I somehow doubt the long-term advantage of one proprietary stack over another.
- DrScientist 6y ago
  Winnonlin is one such tool - been around forever, written in fortran. One of the reasons it is still popular is for regulatory use it's very well understood. I'm not enough of an expert to answer the second part of your question - on why new numerical stacks are beneficial. It may be around building more complex models of PK/PD but I'm not up to speed on what winnonlin can and can't do.
- ChrisRackauckas 6y ago
  Pharmacometrics is built around small stiff ODE models. You might think that this might mean it's "simple" computationally, but what it actually means is that every little detail matters a lot for performance. The way you do lu-factorization for 4 ODEs vs 6 ODEs needs to change in an architecture-dependent fashion, and BLAS libraries are not efficient in this range. Standard floating point exponentiation can be too expensive in some spots, while at the same time the standard stiff ODE solvers make assumptions about regularity which are regularly violated by pharmacometric models (due to how dosing works). There is more than a little bit of "by-hand SIMD" in this software stack. Specializing on all of these features is an interplay between algorithms and JIT compilation, where not utilizing statically compiled optimizations will hurt you at this size. We see this in benchmarks like the SciML Hires [1] which is able to outperform the classic Fortran libraries like LSODA with newly developed Rodas methods. As a less direct and more illustrative example, these benchmarks vs PyTorch [2] demonstrate what happens when you compare against an optimizing JIT compiler which doesn't specialize on small sized interactions (>30x performance difference!), showing how this kind of application is very much not in the regime of large kernels which a lot of recent compiler work tends to optimize for and instead heavily relies on being able to cross-compile and inline computations to remove every little overhead, all while improving the ODE algorithms themselves. We will have a paper that goes into more detail on this fairly soon. [1] https://benchmarks.sciml.ai/html/StiffODE/Hires.html [2] https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e0584a2bfd320

videep90 6y ago

Pumas-AI co-founder here. Indeed, Pharma is conservative due to the impact it can have on patient lives (good or bad). If something is working, it stays, and something failed them, it gets axed, both people and tools. Having said that, let's give them some credit, as along with the regulators at the FDA, pharma has come a long way from the days of SAS. Even the regulators at the FDA have started gravitating away from SAS into more R-based workflows. While it was a good switch away from SAS, scientists started realizing the bottlenecks with R and the requirement to learn multiple tools to get things working, especially when one has to scale up for the new age analytics. While it is good to have a swiss army knife of tools, there is more overhead in making them work together. I think that is why SAS had shined for the longest time. But as you rightly said, it is cost prohibitive and in the hands of very few. Pumas and the Julia ecosystem are about integrated modeling speak across multiple sectors that are composable and the best part is that one does not have to develop in one environment and switch to another for production. Your point about Salesforce may be right for trial operations, but someone has to do the anlaytics that connect back to the first principles of physiology and pharmacology. Pumas provides this connection to first principles from the core, and lets you build on it for both internal decision making or to serve the stakeholders with fancy UI's. A subtle but important point here is that compared to a 30-40 years ago since when SAS became a mainstay, I would argue that more of the younger generation are not averse to coding. In fact programming at the basic level is a fundamental expected skill set, and I personally see more and more life-science majors be efficient coders, thanks to the democratization of tools and education. Combined with this new workforce, and tools like Pumas and Julia, we are not far away from disrupting this sector.

DrScientist 6y ago
Why Julia? By choosing a language that a tiny minority know and use, haven't you crippled your adoption from the start?
- ViralBShah 6y ago
  I think Chris Rackauckas answers "Why Julia" below in https://news.ycombinator.com/item?id=24132382 https://news.ycombinator.com/item?id=24132382 This is a very clear case of a well-defined application. The field of pharma needs a combination of mechanistic models (based on differential equations), statistics, and ability to work with large datasets. The existing tools are far from sufficient. If anything, the need for high performance and scalable tools has been felt in the industry for a very long time. There are hundreds of thousands of Julia programmers worldwide now, and the user base is doubling every year. After all, new ideas do start with a tiny minority, but one that truly believes in those ideas and is enthusiastic in adopting them early.

jogagobburu 6y ago

You are right, a portion of pharma is conservative. There are two different needs in Pharma - one is more like assembly line (pure operations) and other other is "innovation". One has to decide what excites scientists - operations (eg calculating group means over and over where the answer has to be exact) OR innovation (get the questions right, designs right, analysis right, answers approximately accurate). I think there is a place for "operations", but they constitute "sustaining innovations". Meaning can I find the most economical solution for this problem? Operations is a separate discussion. Let us talk about "disruptive innovations". What if I told you I was part of the team that approved a treatment for H1N1 flu pandemic in children without ANY trial based on unapproved data in adults? What if I told you the development pathway for pulmonary arterial hypertension in pediatrics was driven by analysis performed by Pumas-like software to establish a reasonable endpoint? These are patients who cannot perform most daily functions we take for granted. There would have been no drugs approved otherwise. How can you, me and others contribute to the disruptive innovation which will transform how drugs are developed and patients are treated. Pumas is designed to disrupt drug development and precision medicine. There are plenty of opportunities for disruption in healthcare. these opportunities are the motivation for Pumas..Joga Gobburu (Co-Founder, Pumas-AI)

NicoJuicy 6y ago
Which country are you tlaking about? ( cfr. "H1N1 flu pandemic in children without ANY trial based on unapproved data in adults")
- jogagobburu 6y ago
  USA for emergency use.