4 ms·

This point is extremely important, particularly in the healthcare field (which I happen to be in at the moment). We have to be very, very positive that our deid

by tedivm 7y ago

This point is extremely important, particularly in the healthcare field (which I happen to be in at the moment). We have to be very, very positive that our deidentification process is thorough and accurate to prevent a HIPAA violation from occurring.

jacquesm 7y ago
If you start from real data that's a very tough nut to crack. Depending on your purpose you may be better off with generating data that has the same statistical properties.
- nradov 7y ago
  That's true, but in healthcare generating data that matches the statistical properties of real data is also a very tough nut to crack. Essentially you need experienced clinicians who can document a fake patient's care journey through multiple years with thousands of concept codes and narrative text blocks. The level of effort is on the same order of magnitude as writing a novel. Of course if you don't need a complete longitudinal patient chart then generating realistic data for just one aspect can be a lot simpler.
  - jacquesm 7y ago
    https://www.statice.ai/ https://www.statice.ai/
    - nradov 7y ago
      Sure and Synthea is doing something similar, as well as a few others. Those can produce data good enough for certain use cases. But it's not quite good enough for realistic product demos involving complex conditions, or testing anything related to record linkage.
      - tedivm 7y ago
        We've tested numerous synthetic data companies and so far have not found any which can produce data that is useful for real world applications. There are some that are close enough where academic groups might be able to publish around it. There is a single startup we're working with that has come much closer, but still isn't quite there. I think that if they manage to get funding they'll be somewhere usable in the near future.
        jacquesm 7y ago
        Which one is that? Super interested for one of my customers...
missosoup 7y ago
> We have to be very, very positive that our deidentification process is thorough and accurate to prevent a HIPAA violation from occurring If your model is any more useful than something trained on coarse aggregates, then it can be used to reidentify individuals. This is a pretty hard dilemma in the entire industry, not just health. I hope my observations are skewed, but instead of trying to seriously address this issue I've seen an entire legal-loophole style data laundering industry emerge where highly identifiable information changes hands without it 'technically' changing hands in the legal sense. I'm talking about entities like datarepublic.
- tedivm 7y ago
  It really is a complicated topic, and one we spend a lot of time thinking about. We're using a peer reviewed method for removing PHI identifiers, and are combining that with an approach that involves using the least amount of data possible to get results. Our models will also never be released to the public, but instead will have an API where we can see abnormal behavior (such as sending lots of requests to try and tease out other information) and intervene.
  - angry_octet 7y ago
    Other than easily controlled walled garden type accesses it is hard to limit with throttling. Is there a way to determine how much information about a particular individual has been leaked out?

by tedivm 7y ago

jacquesm 7y ago

If you start from real data that's a very tough nut to crack. Depending on your purpose you may be better off with generating data that has the same statistical properties.

nradov 7y ago
That's true, but in healthcare generating data that matches the statistical properties of real data is also a very tough nut to crack. Essentially you need experienced clinicians who can document a fake patient's care journey through multiple years with thousands of concept codes and narrative text blocks. The level of effort is on the same order of magnitude as writing a novel. Of course if you don't need a complete longitudinal patient chart then generating realistic data for just one aspect can be a lot simpler.
- jacquesm 7y ago
  https://www.statice.ai/ https://www.statice.ai/
  - nradov 7y ago
    Sure and Synthea is doing something similar, as well as a few others. Those can produce data good enough for certain use cases. But it's not quite good enough for realistic product demos involving complex conditions, or testing anything related to record linkage.
    - tedivm 7y ago
      We've tested numerous synthetic data companies and so far have not found any which can produce data that is useful for real world applications. There are some that are close enough where academic groups might be able to publish around it. There is a single startup we're working with that has come much closer, but still isn't quite there. I think that if they manage to get funding they'll be somewhere usable in the near future.
      - jacquesm 7y ago
        Which one is that? Super interested for one of my customers...

missosoup 7y ago

> We have to be very, very positive that our deidentification process is thorough and accurate to prevent a HIPAA violation from occurring If your model is any more useful than something trained on coarse aggregates, then it can be used to reidentify individuals. This is a pretty hard dilemma in the entire industry, not just health. I hope my observations are skewed, but instead of trying to seriously address this issue I've seen an entire legal-loophole style data laundering industry emerge where highly identifiable information changes hands without it 'technically' changing hands in the legal sense. I'm talking about entities like datarepublic.

tedivm 7y ago
It really is a complicated topic, and one we spend a lot of time thinking about. We're using a peer reviewed method for removing PHI identifiers, and are combining that with an approach that involves using the least amount of data possible to get results. Our models will also never be released to the public, but instead will have an API where we can see abnormal behavior (such as sending lots of requests to try and tease out other information) and intervene.
- angry_octet 7y ago
  Other than easily controlled walled garden type accesses it is hard to limit with throttling. Is there a way to determine how much information about a particular individual has been leaked out?