13 ms·

This is why medtech has the FDA, and why the FDA has actual teeth. "move fast and break stuff" is bad PR if you're facebook. It's a game-over felony if you're

This is why medtech has the FDA, and why the FDA has actual teeth. "move fast and break stuff" is bad PR if you're facebook. It's a game-over felony if you're making medical devices.

ng12 9y ago
The FDA is necessary but I do think it could stand to adopt a little bit of the Silicon Valley ethos. Good example: the Freestyle Libre CGM is a recently approved glucose monitor for diabetics. The European version lasts 14 days and takes an hour to start up, the US version lasts 10 days and has a 12 hour (!) startup time. It's the exact same hardware with artificial restrictions added to the software for unclear reasons.
- tomca32 9y ago
  12 hour startup time? Wow, that's crazy. Any idea on the reasoning behind the startup delay? I understand the reasoning for disabling it after 10 days, dexcom does the same, but I can't find any reasoning for delaying startup. I'm in the US and my Dexcom takes 2 hours to startup, and that's really only because of the sensor temperature adjustment, otherwise it's pretty much instantaneous.
  - ng12 9y ago
    I was able to find this online: > In the US, FreeStyle Libre’s accuracy has improved a bit with these changes – the average measurement error relative to a reference value is 9.7%, an improvement from around 11% in Europe (and on par with 9.0% for Dexcom’s G5). (https://diatribe.org/abbotts-freestyle-libre-approved-us-replace-routine-fingersticks#warm-up%20time https://diatribe.org/abbotts-freestyle-libre-approved-us-rep...) Which is absolutely ridiculous. 12 hours without a CGM is much, much more dangerous than a 1.3% margin of error.
    - lambda 9y ago
      A 1.3% margin of error is the difference over the entire time of wearing it. It looks like on the first day, comparing blood glucose with FreeStyle Libre CGM, there was a significant difference in accuracy, with a lot more (~20% more) readings falling into "zone B", or "points that are outside of 20% but would not lead to inappropriate treatment" (chart https://www.liebertpub.com/action/showImage?doi=10.1089%2Fdia.2014.0378&iName=master.img-002.jpg&w=237&h=205 https://www.liebertpub.com/action/showImage?doi=10.1089%2Fdi..., from paper https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378 https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378). This is just one quick thing I found on Google Scholar. Given that this CGM can also act as a regular fingerprick blood glucose monitor during that 12 hour overlap, I think the FDA likely wanted to err on the safer side of using well known and well tested methods for monitoring to avoid any unexpected issues. Is there sufficient evidence of the benefit of CGM over fingerprick tests to believe that this caution could cause actual harm?
      - ng12 9y ago
        > Is there sufficient evidence of the benefit of CGM over fingerprick tests to believe that this caution could cause actual harm? This is a question you could only ask if you were looking only at the data without understanding the problem at hand. To answer your question: as a user I'm not sweating Zone B. That doesn't even register as a problem. Zone C is the only zone I care about, and Zone C situations can happen quite easily with fingersticks too. The entire game is imprecise -- every single day I make decisions based on incomplete data. You're going to have a very hard time convincing me that 11 hours of moderately imprecise data is more dangerous than having 11 hours of no data.
        lambda 9y ago
        You are on HN, and likely reasonably technically inclined. You would probably pay attention to instructions that indicate that it might not be accurate during the first 12 hours of use. You would know to be a bit skeptical of readings that show hypoglycemia during that time, and might either do a fingerstick to check or just ignore it. There are many users who would not. They might make a rash decision based on that data. If the data is in Zone B a larger percentage of the time in testing, in much wider general use there will likely be cases that weren't caught in clinical trials in which it could land in Zone C. During this 12 hour calibration window, you can still use fingersticks, which you have to do for calibration purposes with other CGMs anyhow. And it likely would be a fairly minor change for them to support doubling up on sensors, so that you could overlap them in order to avoid the 12 hour window; but that change would require further testing and approval, so for now the 12 hour calibration window is left in. It was only fairly recently approved with this 12 hour calibration window, so I wouldn't be surprised if the company is working on support for pairing two sensors to one device to support doubling up like this.
- bspn 9y ago
  > The FDA is necessary but I do think it could stand to adopt a little bit of the Silicon Valley ethos They're moving closer to the "move-fast" thinking (although not the "break things" part fortunately) under the new administration: https://www.economist.com/news/leaders/21739157-drug-regulator-america-innovating-rapidly-good-getting-medicines-market-faster https://www.economist.com/news/leaders/21739157-drug-regulat...
- lambda 9y ago
  > It's the exact same hardware with artificial restrictions added to the software for unclear reasons. Well, a quick search of Google Scholar turns up at least one citation in which the Freestyle Libre CGM was significantly less accurate in the first day of use than in later days, indicating that it needed more time for proper calibration: https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378 https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378 I expect that the differences are based on some actual findings, not just "unclear reasons," but you may have to do some research to find out. The FDA is notoriously cautious, possibly somewhat excessively so, but there is some good reason for that caution; medication and medical devices can have unexpected behavior and side effects and it can take a lot of long term study to find them. One example of the caution working out well is the Thalidomide case, which was approved in many other jurisdictions but not in the US, and which had significant side effects on fetuses.
  - crankylinuxuser 9y ago
    Indeed the FDA is cautious. But we can also show the other way that inhibiting these technologies also has fatal results too. What's the right regulation? I don't know.
  - ng12 9y ago
    What's unclear is not the data but the decisions they made based on the data. Why is that first-day margin of error acceptable for European diabetics but not American ones? Anecdotally, as a diabetic, the margin of error in that paper is much less scary than going 12 hours without a CGM. Poor data is much better than no data, especially if I know the data is going to be a little questionable the first day. I was completely ready to switch from Dexcom to the Libre until they announced that restriction.
    - deleted 9y ago
      [deleted]
    - daveFNbuck 9y ago
      > What's unclear is not the data but the decisions they made based on the data. Why is that first-day margin of error acceptable for European diabetics but not American ones? The decisions were being made by different people. It's not like the FDA decided that Americans require a 12-hour window but Europeans only need 1. The FDA is just more cautious.
- gumby 9y ago
  > The FDA is necessary but I do think it could stand to adopt a little bit of the Silicon Valley ethos. The FDA is necessarily conservative (in the practice, not political sense) but actively pursuing changes like you ask: look at their interest and experiments with adaptive trials. Out here in the valley they are considered by folks outside life sciences to be stodgy and bureaucratic. But they can move fast. In one (paper) submission I made we dropped one sheet of the 1600+ page document when copying them. Within a week of sending it to them they contacted us for that sheet and we sent it. My board of directors was very interested in how long it took us to reply (down to the minute) and when I said it was less than 10 minutes there was a big sigh of relief. Compare that to the months they gave 22andMe to reply to notices of gross violation that the company had ignored.
- deleted 9y ago
  [deleted]
- Sangermaine 9y ago
  Thalidomide.
  - ng12 9y ago
    Please do not put this in the same class as prenatal drug exposure. They are not in the same category for very obvious reasons.
    - Sangermaine 9y ago
      I'm merely pointing out a glaring instance where the FDA being "overcautious" prevented incredible harm. I'm far more inclined to stand with them erring on the side of caution and believing, as the thalidomide incident showed, that the European regulators are too lax.
      - ng12 9y ago
        Where does it follow that the caution has to be draconion in it's application?
Alex3917 9y ago
> It's a game-over felony if you're making medical devices. Making medical devices that give fake results usually isn’t even a crime, let alone a felony. Otherwise almost all of the makers of home glucose monitors would be in prison. Or walk into your local supermarket and see how accurate the cheapest food thermometers are, if you want an even more basic example.
- HeyLaughingBoy 9y ago
  There's a substantial difference between a result that is incorrect, but believed to be correct and one that is known to be incorrect, but presented as correct. The former is an error that probably requires the manufacturer to inform the FDA and send out a Customer Letter (a low-level form of product recall). The latter is fraud.
- gumby 9y ago
  Note: I have previously developed a combo product (device + drug) and have made submissions to and appeared before the FDA on both the pharma and device sides. > Making medical devices that give fake results usually isn’t even a crime, let alone a felony. It very much is a felony and people go to jail. The FDA even can and does make armed (!) raids on facilities when they believe there is fraud. > Otherwise almost all of the makers of home glucose monitors would be in prison. Home glucose monitors in the US only need an accuracy of 20% and the manufacturers have to prove it. And their labeling is regulated just like any other medical product. I don't know why you think that their results are fake. A distal finger stick of capillary blood will not give the same results as a venous draw analyzed with GLP, but nobody is claiming that it should. > Or walk into your local supermarket and see how accurate the cheapest food thermometers are, if you want an even more basic example. These are not regulated products, unless you are using them for commerce (in which case they are regulated by your State, not the FDA).
  - Alex3917 9y ago
    > These are not regulated products, unless you are using them for commerce So in other words a major category of medical devices, which are needed to prevent you from dying from eating undercooked poultry or whatever, aren't even regulated, then that seems to support what I was saying.

This is why medtech has the FDA, and why the FDA has actual teeth. "move fast and break stuff" is bad PR if you're facebook. It's a game-over felony if you're making medical devices.

ng12 9y ago

The FDA is necessary but I do think it could stand to adopt a little bit of the Silicon Valley ethos. Good example: the Freestyle Libre CGM is a recently approved glucose monitor for diabetics. The European version lasts 14 days and takes an hour to start up, the US version lasts 10 days and has a 12 hour (!) startup time. It's the exact same hardware with artificial restrictions added to the software for unclear reasons.

tomca32 9y ago
12 hour startup time? Wow, that's crazy. Any idea on the reasoning behind the startup delay? I understand the reasoning for disabling it after 10 days, dexcom does the same, but I can't find any reasoning for delaying startup. I'm in the US and my Dexcom takes 2 hours to startup, and that's really only because of the sensor temperature adjustment, otherwise it's pretty much instantaneous.
- ng12 9y ago
  I was able to find this online: > In the US, FreeStyle Libre’s accuracy has improved a bit with these changes – the average measurement error relative to a reference value is 9.7%, an improvement from around 11% in Europe (and on par with 9.0% for Dexcom’s G5). (https://diatribe.org/abbotts-freestyle-libre-approved-us-replace-routine-fingersticks#warm-up%20time https://diatribe.org/abbotts-freestyle-libre-approved-us-rep...) Which is absolutely ridiculous. 12 hours without a CGM is much, much more dangerous than a 1.3% margin of error.
  - lambda 9y ago
    A 1.3% margin of error is the difference over the entire time of wearing it. It looks like on the first day, comparing blood glucose with FreeStyle Libre CGM, there was a significant difference in accuracy, with a lot more (~20% more) readings falling into "zone B", or "points that are outside of 20% but would not lead to inappropriate treatment" (chart https://www.liebertpub.com/action/showImage?doi=10.1089%2Fdia.2014.0378&iName=master.img-002.jpg&w=237&h=205 https://www.liebertpub.com/action/showImage?doi=10.1089%2Fdi..., from paper https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378 https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378). This is just one quick thing I found on Google Scholar. Given that this CGM can also act as a regular fingerprick blood glucose monitor during that 12 hour overlap, I think the FDA likely wanted to err on the safer side of using well known and well tested methods for monitoring to avoid any unexpected issues. Is there sufficient evidence of the benefit of CGM over fingerprick tests to believe that this caution could cause actual harm?
    - ng12 9y ago
      > Is there sufficient evidence of the benefit of CGM over fingerprick tests to believe that this caution could cause actual harm? This is a question you could only ask if you were looking only at the data without understanding the problem at hand. To answer your question: as a user I'm not sweating Zone B. That doesn't even register as a problem. Zone C is the only zone I care about, and Zone C situations can happen quite easily with fingersticks too. The entire game is imprecise -- every single day I make decisions based on incomplete data. You're going to have a very hard time convincing me that 11 hours of moderately imprecise data is more dangerous than having 11 hours of no data.
      - lambda 9y ago
        You are on HN, and likely reasonably technically inclined. You would probably pay attention to instructions that indicate that it might not be accurate during the first 12 hours of use. You would know to be a bit skeptical of readings that show hypoglycemia during that time, and might either do a fingerstick to check or just ignore it. There are many users who would not. They might make a rash decision based on that data. If the data is in Zone B a larger percentage of the time in testing, in much wider general use there will likely be cases that weren't caught in clinical trials in which it could land in Zone C. During this 12 hour calibration window, you can still use fingersticks, which you have to do for calibration purposes with other CGMs anyhow. And it likely would be a fairly minor change for them to support doubling up on sensors, so that you could overlap them in order to avoid the 12 hour window; but that change would require further testing and approval, so for now the 12 hour calibration window is left in. It was only fairly recently approved with this 12 hour calibration window, so I wouldn't be surprised if the company is working on support for pairing two sensors to one device to support doubling up like this.
bspn 9y ago
> The FDA is necessary but I do think it could stand to adopt a little bit of the Silicon Valley ethos They're moving closer to the "move-fast" thinking (although not the "break things" part fortunately) under the new administration: https://www.economist.com/news/leaders/21739157-drug-regulator-america-innovating-rapidly-good-getting-medicines-market-faster https://www.economist.com/news/leaders/21739157-drug-regulat...
lambda 9y ago
> It's the exact same hardware with artificial restrictions added to the software for unclear reasons. Well, a quick search of Google Scholar turns up at least one citation in which the Freestyle Libre CGM was significantly less accurate in the first day of use than in later days, indicating that it needed more time for proper calibration: https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378 https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378 I expect that the differences are based on some actual findings, not just "unclear reasons," but you may have to do some research to find out. The FDA is notoriously cautious, possibly somewhat excessively so, but there is some good reason for that caution; medication and medical devices can have unexpected behavior and side effects and it can take a lot of long term study to find them. One example of the caution working out well is the Thalidomide case, which was approved in many other jurisdictions but not in the US, and which had significant side effects on fetuses.
- crankylinuxuser 9y ago
  Indeed the FDA is cautious. But we can also show the other way that inhibiting these technologies also has fatal results too. What's the right regulation? I don't know.
- ng12 9y ago
  What's unclear is not the data but the decisions they made based on the data. Why is that first-day margin of error acceptable for European diabetics but not American ones? Anecdotally, as a diabetic, the margin of error in that paper is much less scary than going 12 hours without a CGM. Poor data is much better than no data, especially if I know the data is going to be a little questionable the first day. I was completely ready to switch from Dexcom to the Libre until they announced that restriction.
  - deleted 9y ago
    [deleted]
  - daveFNbuck 9y ago
    > What's unclear is not the data but the decisions they made based on the data. Why is that first-day margin of error acceptable for European diabetics but not American ones? The decisions were being made by different people. It's not like the FDA decided that Americans require a 12-hour window but Europeans only need 1. The FDA is just more cautious.
gumby 9y ago
> The FDA is necessary but I do think it could stand to adopt a little bit of the Silicon Valley ethos. The FDA is necessarily conservative (in the practice, not political sense) but actively pursuing changes like you ask: look at their interest and experiments with adaptive trials. Out here in the valley they are considered by folks outside life sciences to be stodgy and bureaucratic. But they can move fast. In one (paper) submission I made we dropped one sheet of the 1600+ page document when copying them. Within a week of sending it to them they contacted us for that sheet and we sent it. My board of directors was very interested in how long it took us to reply (down to the minute) and when I said it was less than 10 minutes there was a big sigh of relief. Compare that to the months they gave 22andMe to reply to notices of gross violation that the company had ignored.
deleted 9y ago
[deleted]
Sangermaine 9y ago
Thalidomide.
- ng12 9y ago
  Please do not put this in the same class as prenatal drug exposure. They are not in the same category for very obvious reasons.
  - Sangermaine 9y ago
    I'm merely pointing out a glaring instance where the FDA being "overcautious" prevented incredible harm. I'm far more inclined to stand with them erring on the side of caution and believing, as the thalidomide incident showed, that the European regulators are too lax.
    - ng12 9y ago
      Where does it follow that the caution has to be draconion in it's application?

Alex3917 9y ago

> It's a game-over felony if you're making medical devices. Making medical devices that give fake results usually isn’t even a crime, let alone a felony. Otherwise almost all of the makers of home glucose monitors would be in prison. Or walk into your local supermarket and see how accurate the cheapest food thermometers are, if you want an even more basic example.

HeyLaughingBoy 9y ago
There's a substantial difference between a result that is incorrect, but believed to be correct and one that is known to be incorrect, but presented as correct. The former is an error that probably requires the manufacturer to inform the FDA and send out a Customer Letter (a low-level form of product recall). The latter is fraud.
gumby 9y ago
Note: I have previously developed a combo product (device + drug) and have made submissions to and appeared before the FDA on both the pharma and device sides. > Making medical devices that give fake results usually isn’t even a crime, let alone a felony. It very much is a felony and people go to jail. The FDA even can and does make armed (!) raids on facilities when they believe there is fraud. > Otherwise almost all of the makers of home glucose monitors would be in prison. Home glucose monitors in the US only need an accuracy of 20% and the manufacturers have to prove it. And their labeling is regulated just like any other medical product. I don't know why you think that their results are fake. A distal finger stick of capillary blood will not give the same results as a venous draw analyzed with GLP, but nobody is claiming that it should. > Or walk into your local supermarket and see how accurate the cheapest food thermometers are, if you want an even more basic example. These are not regulated products, unless you are using them for commerce (in which case they are regulated by your State, not the FDA).
- Alex3917 9y ago
  > These are not regulated products, unless you are using them for commerce So in other words a major category of medical devices, which are needed to prevent you from dying from eating undercooked poultry or whatever, aren't even regulated, then that seems to support what I was saying.