7 ms·

Theranos – Statement of Deficiency from CMMS [pdf]

137 points by jerryhuang100 11y ago

w1ntermute 11y ago
Corresponding article: Theranos Devices Often Failed Accuracy Requirements http://www.wsj.com/articles/theranos-devices-often-failed-accuracy-requirements-1459465578 http://www.wsj.com/articles/theranos-devices-often-failed-ac...
zavi 11y ago
WSJ sure does not like Theranos.
- URSpider94 11y ago
  It's a reporter with a juicy story. Just like with Woodward & Bernstein with Watergate, someone smells a Pulitzer coming if they can single-handedly trigger the collapse of a beloved unicorn.
  - nickgrosvenor 11y ago
    You're right, they're dead
    - URSpider94 11y ago
      That might be a little premature, but this team (primarily John Carreyrou and Christopher Weaver) are clearly not going to stop writing about Theranos any time soon. Additionally, they've pretty clearly provided an outlet for people who seem to be itching for this story to get some sunlight, so they're scooping the world on the blow-by-blow of Theranos' interactions with FDA, Walgreen's, Safeway, etc.
  - randycupertino 11y ago
    This reporting team already has a Pulitzer for their medicare fraud piece a few years ago. https://en.wikipedia.org/wiki/John_Carreyrou https://en.wikipedia.org/wiki/John_Carreyrou
smaili 11y ago
For those of us who aren't familiar, what exactly is the purpose of this document and what does it mean for Theranos?
godzillabrennus 11y ago
Theranos is getting some well deserved regulatory attention. We need better healthcare tech but we also need to know that it works.
niuzeta 11y ago
They're all dated November of 2015, which was right around when the WSJ investigation brought out. Is this the first time this document(s) was open publicly? What is the new information here?
tyre 11y ago
There is an accompanying article [1] that gives more background. The part on there internal quality control tests show the tech just isn't there. > The inspection report showed that 29% of the quality-control checks performed on the company’s proprietary blood-testing devices in October 2014 produced results outside the range considered acceptable by Theranos. > In February 2015, an Edison-run test to measure a hormone that affects testosterone levels failed 87% of quality-control checks, the report said. It doesn't look like they are using Edison to run most of their tests, so we can excuse the experimental tests if they aren't being used for actual patients. Chalk that up to work-in-progress. What is absolutely inexcusable is their treatment of patients, especially when they found out that results were incorrect. > Inspectors also found that Theranos sometimes released patients’ results even when the Edison devices used to run those tests produced erratic results in quality-control checks. and > The inspection report said Theranos didn’t notify doctors treating patients potentially affected by the erroneous results until mid-November, or seven weeks after the inspectors first identified the problems. Seven weeks to notify customers that you may have fucked up a medical test? We reach out to customers about potential software bugs within an hour. This is an instance where I am glad to have government slowing down innovation. "Move fast, break things" doesn't work when people's lives are at stake. [1]: http://www.wsj.com/articles/theranos-devices-often-failed-accuracy-requirements-1459465578 http://www.wsj.com/articles/theranos-devices-often-failed-ac...
- diskcat 11y ago
  >"Move fast, break things" doesn't work when people's lives are at stake. No matter what you do, people's lives are at stake. Even an app that suggests which restaurant to go for dinner could cause somebody to die by suggesting a restaurant in an area with higher rate of accidents. The problem is how little consumers have control of when it comes to healthcare. We like to hand wave and pretend that the rules of economics don't apply when it comes to healthcare so instead of having a market where a person can shop for medicine, doctor's service, equipment etc. it all is more or less just close your eyes go in the tunnel and you will be taken care of and any issue of choice with regards to money is occluded by 'universal healthcare is a human right'. If I could go to a hospital who would then give me a brochure of all the equipments they use and I find out one of their supplier is a dodgy companies that have dodgy equipments and terrible bug-handling practice I would go to another seller just as I would a smartphone that uses a dodgy processor that is known to break.
c-slice 11y ago
Although it seems like the acronym should be CMMS, it's actually CMS.
Animats 11y ago
What a mess. This has nothing to do with Theranos's "nano" technology. This is about Theranos botching standard tests performed with standard commercial equipment, and then not reporting their errors. Much medical lab practice includes running known test samples through the process regularly to check the test. They skipped a lot of that. Many things can potentially go wrong. They did. Expired reagents, freezers at wrong temperatures, failure to do regular maintenance, patient tests run on machines that were reporting errors... Some of the technicians hadn't completed medical technician training, but were doing complex tests. The summary: "Overall, over 21% of QC samples on all devices had values greater than 2 SDs". So many results were way off, and they weren't catching that. Those are for standard tests on standard machines. The results for Theranos' "Edison" machine were worse. As in more than 10x more variation than standard techniques. This isn't even the FDA. This is the Center for Medicare Services checking up on a company to which they paid a lot of money. WSJ: "“This is the first time that we’ve actually seen data from the Theranos instrument, and it’s as bad as one would have worried it would be,” said Stephen Master, associate professor of pathology at Weill Cornell Medical College in New York. “Based on this data, it’s hard for me to believe that they went live with this instrument and tested patient specimens on it.” "The inspection report said Theranos didn’t notify doctors treating patients potentially affected by the erroneous results until mid-November, or seven weeks after the inspectors first identified the problems." "The lab directors during the period of the survey no longer hold any position within the lab. The new lab director..." This looks like the "move fast and break things" crowd trying to run a medical lab.
- tomcorrigan 11y ago
  > This looks like the "move fast and break things" crowd trying to run a medical lab. That's a disingenuously kind interpretation. This was a bunch of fraudsters with failed technology making out that it worked.
  - officialchicken 11y ago
    Disagree, it is easy to find a Theranos article or Holmes quote with the word "disrupt". The enablers (VC's) of this trainwreck preferred that philosophy and not the traditional medtech/pharma/R&D that use Verify & Validate approach to development.
Aelinsaar 11y ago
This... this is real train wreck. When this finally goes down, it's going to happen so quickly, and I wouldn't be shocked if some people try to cut and run, fleeing the country.
cant_kant 11y ago
"In April and May 2015, a test to measure prostate-specific antigen, or PSA, failed quality-control checks 22% of the time. The PSA test is used to help detect prostate cancer." Ouch.... I see a string of class action suits in the future.
Fede_V 11y ago
Out of curiosity: has anyone ever tried to do a correlation about the performance of a hard-tech company vs the ratio of technical/political people in their board of directors? This is probably a strong prior on my part - but when I saw the composition of the board of directors of Theranos (secretary of states/generals/congressmen etc) I was immediately incredibly skeptical. It seems like the board was chosen to provide lucrative contacts instead of actually useful advice and supervision.
atonse 11y ago
This is one of those rare times where my procrastination in applying for a job at Theranos actually paid off. So grateful I avoided moving my family across the country to get entangled in this mess.
iskander 11y ago
This is actually a lot worse than the results recently reported by Mount Sinai in their comparison of a few dozen patients against two standard lab testing companies. I wonder if Theranos has been trying to clean up their standard tests.
brandon272 11y ago
Is there any chance that any of the company's directors will see jail time over this?
- jonnathanson 11y ago
  Doubtful, unless they've got any extant political enemies who want to make an example out of them. Theranos' BOD is basically a who's who of Washington power brokers.

137 points by jerryhuang100 11y ago

Corresponding article: Theranos Devices Often Failed Accuracy Requirements http://www.wsj.com/articles/theranos-devices-often-failed-accuracy-requirements-1459465578 http://www.wsj.com/articles/theranos-devices-often-failed-ac...

zavi 11y ago

WSJ sure does not like Theranos.

URSpider94 11y ago
It's a reporter with a juicy story. Just like with Woodward & Bernstein with Watergate, someone smells a Pulitzer coming if they can single-handedly trigger the collapse of a beloved unicorn.
- nickgrosvenor 11y ago
  You're right, they're dead
  - URSpider94 11y ago
    That might be a little premature, but this team (primarily John Carreyrou and Christopher Weaver) are clearly not going to stop writing about Theranos any time soon. Additionally, they've pretty clearly provided an outlet for people who seem to be itching for this story to get some sunlight, so they're scooping the world on the blow-by-blow of Theranos' interactions with FDA, Walgreen's, Safeway, etc.
- randycupertino 11y ago
  This reporting team already has a Pulitzer for their medicare fraud piece a few years ago. https://en.wikipedia.org/wiki/John_Carreyrou https://en.wikipedia.org/wiki/John_Carreyrou

smaili 11y ago

For those of us who aren't familiar, what exactly is the purpose of this document and what does it mean for Theranos?

godzillabrennus 11y ago

Theranos is getting some well deserved regulatory attention. We need better healthcare tech but we also need to know that it works.

niuzeta 11y ago

They're all dated November of 2015, which was right around when the WSJ investigation brought out. Is this the first time this document(s) was open publicly? What is the new information here?

tyre 11y ago

There is an accompanying article [1] that gives more background. The part on there internal quality control tests show the tech just isn't there. > The inspection report showed that 29% of the quality-control checks performed on the company’s proprietary blood-testing devices in October 2014 produced results outside the range considered acceptable by Theranos. > In February 2015, an Edison-run test to measure a hormone that affects testosterone levels failed 87% of quality-control checks, the report said. It doesn't look like they are using Edison to run most of their tests, so we can excuse the experimental tests if they aren't being used for actual patients. Chalk that up to work-in-progress. What is absolutely inexcusable is their treatment of patients, especially when they found out that results were incorrect. > Inspectors also found that Theranos sometimes released patients’ results even when the Edison devices used to run those tests produced erratic results in quality-control checks. and > The inspection report said Theranos didn’t notify doctors treating patients potentially affected by the erroneous results until mid-November, or seven weeks after the inspectors first identified the problems. Seven weeks to notify customers that you may have fucked up a medical test? We reach out to customers about potential software bugs within an hour. This is an instance where I am glad to have government slowing down innovation. "Move fast, break things" doesn't work when people's lives are at stake. [1]: http://www.wsj.com/articles/theranos-devices-often-failed-accuracy-requirements-1459465578 http://www.wsj.com/articles/theranos-devices-often-failed-ac...

diskcat 11y ago
>"Move fast, break things" doesn't work when people's lives are at stake. No matter what you do, people's lives are at stake. Even an app that suggests which restaurant to go for dinner could cause somebody to die by suggesting a restaurant in an area with higher rate of accidents. The problem is how little consumers have control of when it comes to healthcare. We like to hand wave and pretend that the rules of economics don't apply when it comes to healthcare so instead of having a market where a person can shop for medicine, doctor's service, equipment etc. it all is more or less just close your eyes go in the tunnel and you will be taken care of and any issue of choice with regards to money is occluded by 'universal healthcare is a human right'. If I could go to a hospital who would then give me a brochure of all the equipments they use and I find out one of their supplier is a dodgy companies that have dodgy equipments and terrible bug-handling practice I would go to another seller just as I would a smartphone that uses a dodgy processor that is known to break.

c-slice 11y ago

Although it seems like the acronym should be CMMS, it's actually CMS.

Animats 11y ago

What a mess. This has nothing to do with Theranos's "nano" technology. This is about Theranos botching standard tests performed with standard commercial equipment, and then not reporting their errors. Much medical lab practice includes running known test samples through the process regularly to check the test. They skipped a lot of that. Many things can potentially go wrong. They did. Expired reagents, freezers at wrong temperatures, failure to do regular maintenance, patient tests run on machines that were reporting errors... Some of the technicians hadn't completed medical technician training, but were doing complex tests. The summary: "Overall, over 21% of QC samples on all devices had values greater than 2 SDs". So many results were way off, and they weren't catching that. Those are for standard tests on standard machines. The results for Theranos' "Edison" machine were worse. As in more than 10x more variation than standard techniques. This isn't even the FDA. This is the Center for Medicare Services checking up on a company to which they paid a lot of money. WSJ: "“This is the first time that we’ve actually seen data from the Theranos instrument, and it’s as bad as one would have worried it would be,” said Stephen Master, associate professor of pathology at Weill Cornell Medical College in New York. “Based on this data, it’s hard for me to believe that they went live with this instrument and tested patient specimens on it.” "The inspection report said Theranos didn’t notify doctors treating patients potentially affected by the erroneous results until mid-November, or seven weeks after the inspectors first identified the problems." "The lab directors during the period of the survey no longer hold any position within the lab. The new lab director..." This looks like the "move fast and break things" crowd trying to run a medical lab.

tomcorrigan 11y ago
> This looks like the "move fast and break things" crowd trying to run a medical lab. That's a disingenuously kind interpretation. This was a bunch of fraudsters with failed technology making out that it worked.
- officialchicken 11y ago
  Disagree, it is easy to find a Theranos article or Holmes quote with the word "disrupt". The enablers (VC's) of this trainwreck preferred that philosophy and not the traditional medtech/pharma/R&D that use Verify & Validate approach to development.

Aelinsaar 11y ago

This... this is real train wreck. When this finally goes down, it's going to happen so quickly, and I wouldn't be shocked if some people try to cut and run, fleeing the country.

cant_kant 11y ago

"In April and May 2015, a test to measure prostate-specific antigen, or PSA, failed quality-control checks 22% of the time. The PSA test is used to help detect prostate cancer." Ouch.... I see a string of class action suits in the future.

Fede_V 11y ago

Out of curiosity: has anyone ever tried to do a correlation about the performance of a hard-tech company vs the ratio of technical/political people in their board of directors? This is probably a strong prior on my part - but when I saw the composition of the board of directors of Theranos (secretary of states/generals/congressmen etc) I was immediately incredibly skeptical. It seems like the board was chosen to provide lucrative contacts instead of actually useful advice and supervision.

atonse 11y ago

This is one of those rare times where my procrastination in applying for a job at Theranos actually paid off. So grateful I avoided moving my family across the country to get entangled in this mess.

iskander 11y ago

This is actually a lot worse than the results recently reported by Mount Sinai in their comparison of a few dozen patients against two standard lab testing companies. I wonder if Theranos has been trying to clean up their standard tests.

brandon272 11y ago

Is there any chance that any of the company's directors will see jail time over this?

jonnathanson 11y ago
Doubtful, unless they've got any extant political enemies who want to make an example out of them. Theranos' BOD is basically a who's who of Washington power brokers.