5 ms·

I like this approach, it doesn't necessarily need to be just the "market" performing the audits however. The FDA handles audits of medical software companies ju

I like this approach, it doesn't necessarily need to be just the "market" performing the audits however. The FDA handles audits of medical software companies just fine. Focusing on the Quality Management System and their Risk Assessment/Security practices seems like a solid approach, and of course centralize this data and make it easily searchable as much as possible, and provide API access to it in case vendors like Amazon want to integrate it and display certifications/grades for products/manufacturers automatically. All that being said, I have no idea how much manpower or money it would take to do audits at that scale, or even what the scale of IoT Devices vs. Medical Devices is.

CodeWriter23 3y ago
FDA: $450K per product. And they aren't doing very much more than asking the vendor to describe their protocols, then ensure the vendor complies with their protocols and any agency guidance. Source: I work at an FDA-regulated company.
- bick_nyers 3y ago
  Did not know it was $450k per product, my second responsibility outside of software engineering was being the risk manager at my previous company as well which is FDA-regulated. Still, many IoT companies that sell products don't even have protocols or a QMS at all, and need some kind of heat applied to them.
  - robertlagrant 3y ago
    I might be a bit cynical, but if you divide the world of IoT into companies that do things well (but charge more) and companies that do things badly (but charge less), then I think the following might happen if you mandate QMS and audits. The companies that do well already just add to their costs (and prices) as they need to employ people to maintain these systems, and companies that do badly will also have to hire those people, and increase their prices, but they will be creating a paper shield around the products, rather than a genuine product improvement.
    - bick_nyers 3y ago
      It's not just covered by hiring compliance people. You need to have an actual quality management system, e.g. a Jira (or whatever) instance that links from bug reports to documentation to code commit to feature deployment. Instead of just having an email address and sometimes letting the engineers know, and the engineers sometimes make a code commit with a message that makes any kind of sense, and engineers sometimes reviewing code, and engineers sometimes forgetting a region to deploy the update to. You might think these kinds of things are table stakes, and I would agree.
      - robertlagrant 3y ago
        Agree; I make software as a medical device. My point is you often don't have to do that. You just have to fling a lot of paper at an auditor, which can be generated well (as you describe, and as I would do, and the good companies in my example would already do) or badly (which the bad companies in my example would do) where it's basically generated post hoc in a hurry.
- convivialdingo 3y ago
  It depends... I just worked my part of certifying a product(security) and it was only ~$40k. Agreed on the current state of the FDA filings - there's a lot of paperwork and process auditing - but guidance is lacking and I'd like to have more clarity rather than just "industry best-practices." That said - things are much better and it seems like the trajectory is improving.

CodeWriter23 3y ago

FDA: $450K per product. And they aren't doing very much more than asking the vendor to describe their protocols, then ensure the vendor complies with their protocols and any agency guidance. Source: I work at an FDA-regulated company.

bick_nyers 3y ago
Did not know it was $450k per product, my second responsibility outside of software engineering was being the risk manager at my previous company as well which is FDA-regulated. Still, many IoT companies that sell products don't even have protocols or a QMS at all, and need some kind of heat applied to them.
- robertlagrant 3y ago
  I might be a bit cynical, but if you divide the world of IoT into companies that do things well (but charge more) and companies that do things badly (but charge less), then I think the following might happen if you mandate QMS and audits. The companies that do well already just add to their costs (and prices) as they need to employ people to maintain these systems, and companies that do badly will also have to hire those people, and increase their prices, but they will be creating a paper shield around the products, rather than a genuine product improvement.
  - bick_nyers 3y ago
    It's not just covered by hiring compliance people. You need to have an actual quality management system, e.g. a Jira (or whatever) instance that links from bug reports to documentation to code commit to feature deployment. Instead of just having an email address and sometimes letting the engineers know, and the engineers sometimes make a code commit with a message that makes any kind of sense, and engineers sometimes reviewing code, and engineers sometimes forgetting a region to deploy the update to. You might think these kinds of things are table stakes, and I would agree.
    - robertlagrant 3y ago
      Agree; I make software as a medical device. My point is you often don't have to do that. You just have to fling a lot of paper at an auditor, which can be generated well (as you describe, and as I would do, and the good companies in my example would already do) or badly (which the bad companies in my example would do) where it's basically generated post hoc in a hurry.
convivialdingo 3y ago
It depends... I just worked my part of certifying a product(security) and it was only ~$40k. Agreed on the current state of the FDA filings - there's a lot of paperwork and process auditing - but guidance is lacking and I'd like to have more clarity rather than just "industry best-practices." That said - things are much better and it seems like the trajectory is improving.