8 ms·

I'm a medical researcher, and I've worked with exactly this sort of data. The number of hoops I had to jump through to even be allowed on the project was insane

I'm a medical researcher, and I've worked with exactly this sort of data. The number of hoops I had to jump through to even be allowed on the project was insane. Background checks, legal forms of all kinds, trainings on how to keep the data secure. Fingerprint locked rooms. Access control. And most importantly: the data themselves were air-gapped. If I fucked up on this scale, jail would be the least of my worries. I'd never work again. I'd be professionally shunned forever, with no hope of redemption once my name is attached to the incident in some Google search. No more grants, no more collaborations, no nothing. I'd be ruined. Frankly I'd be begging to be let into prison; where else am I gonna be able to eat? Nothing will happen to these people.

mvn9 6y ago
Somebody has to work for those who collect all the leaked data and use it for whatever is possible. You wouldn't earn fame but money for being a data pirate. Or you create a company and start with a fresh name. On the internet, nobody knows that you are a dog.
g_p 6y ago
In contrast here, this data likely came from an insurance company, which got access to the data from their claims. They don't inherently care about these people, because they're all seen as "costs" to the balance sheet profits for the year. Sending their data out for "AI" to try and learn how to predict which claims can be rejected is a rational thing to do, in their eyes, whatever the cost (to those concerned and their data), since it might help improve their bottom line by rejecting claims faster. At least there's a cynical take on this. They have no inherent interest in protecting this data, absent external pressures or legislation that imposes penalties. It strikes me the real issue is the lack of negative inventive for people and businesses who screw up with data. If you build something physical that falls down and hurts people, your investor money ends up paying compensation. In tech though, the goal seems to be to make the revenues without ever touching the responsibility. The story over the weekend of Amazon trying to avoid being deemed liable as a seller for goods they ship, (often) deliver, warehouse, market, dispatch and receive seems to sum this up. A race to the bottom when it comes to protections - Amazon effectively is a shop selling goods from others, as it all happens on their terms. We see similar in physical retail where the manufacturer or reseller has a retention of title over goods in the store, and that doesn't affect the store being responsible. It seems like the "being responsible for what you build" is very much lacking in tech companies, due to lack of external regulation to achieve it. Same here in this case.
- harha 6y ago
  We've tried this for a while (companies using customers data) and I see a strong case for governments to start protecting users more strongly, because the current way this is being treated isn't working (medical data needs to be handled with greatest care everywhere else, why not here?). The user already pays for the insurance (or hardware or service in other cases), so the data collected should only be used for those purposes, e.g., handling the claim and not building AI models without the necessary care. The customer should have the right to sell their own data to a company they see fit to take care of it appropriately and that compensates them for it.
  - pc86 6y ago
    The problem is with ownership I think. I'm not German so I'm really only speaking from what I've seen online (mostly here), but my understanding is that for Germans in particular (maybe EU writ large), you own data about you regardless of how it is collected or who has custody of it. In the US, it seems like the data about people is owned by whoever creates that data (in this case, an insurance company), so they are free to do with that data whatever they like provided it doesn't violate some other non-data-specific law.
mola 6y ago
When the mantra of a huge swathe of business is "disruption" "move fast and break things" etc, well, we end up with a broken disrupted system. Innovation is good, but stability is also good. Instead of optimizing only for dynamism we need to optimize for both. The reason you'll get yourself in trouble and these people won't is because you are operating in different moral systems. Where they operate, the moral thing to do is whatever it takes to succeed, it's entirely moral to "fake it until you make it" to buy media articles promoting your product, to sell media space your readers believe is actually curated, to the highest bidder. You operate in the "ivory tower" they hate, because it's slow and puts up boundaries that limit people from succeeding at all cost, that makes you think about the consequences of your actions. They call it slow and inefficient, I call efficient in the long run.
- thinkharderdev 6y ago
  I'm not sure that framing makes a lot of sense here. This was a SaaS product for process automation which seems like a perfectly reasonable product. I would argue in fact that having tech companies (meaning companies which have specific expertise in building software products as opposed to companies that have a expertise in something else and also have an "IT department" bolted on) manage sensitive data is likely to make it more secure rather than less. Companies and other institutions which don't have any particular expertise in software engineering and security aren't likely to do a good job. Of course when those companies do screw up they should be held liable though, up to an including criminal liability is the leak is bad enough and the exploited vulnerability is egregious enough.
- unishark 6y ago
  Universities can be punished by NIH in addition to the feds if they violate privacy rules in clinical studies. Columbia was threatened with losing all their research funding if they didn't get their act together after a big breach a few years ago. But I know plenty academic researchers with a "move fast and break things" and "ignore all rules" attitude. In any highly-competitive system there will be those that gain an advantage by being amoral while everyone else follows the rules. Healthcare is probably especially vulnerable to this as the rules (and legitimate companies following them) has resulted in healthcare lagging far behind other industries in terms of new technologies.
sillysaurusx 6y ago
This is a wild exaggeration. First of all, everyone is insecure. Work as a pentester, and you'll see that the whole world is insecure. After the 30th project with an SQL injection vuln or misconfiguration, you'll never be phased again. Secondly, this seems to be a straightforward database misconfiguration. The database was set to public with no restrictions. It likely wasn't accessible to anyone who wasn't explicitly looking for it, i.e. security researchers or black hats. Yes, this was horrible, but nothing bad happened here. The database was simply set to private. Third, my friend, chillax a bit. Yes, it's extraordinarily important to protect people's medical data. But no, you're not personally liable if a fuckup happens. Everyone fucks up. That's why the fuckups are covered by legal protections. You take your job seriously. Good! But you also take your job way too seriously. Bad. You're gonna burn yourself out within a decade with this mindset. None of this is to say that what happened didn't matter. Of course it matters. But you are literally saying you'd be begging to be let into prison. That's not proportionate, and feels like a reflection of the current social climate of retribution-as-forgiveness.
- pc86 6y ago
  What's more, it's important to take in the context of the data. It's not mental health, or HIV status, or sexual history, or really of any nature that can do damage other than the privacy aspect. It's people who hurt their neck after a car accident. Not the most sensitive stuff in the world. Just to be explicit about it, I'm not suggesting the release of this data isn't a problem. Just that as medical data goes this is about an innocuous as it could have been. Even day-to-day visits to your primary care doc would probably have references to much more sensitive health data.
  - nitrogen 6y ago
    It's more sensitive than you'd think if any claims or settlements were involved. Or employers might want to snoop on their injured employees to find a reason to deny leave.
  - deleted 6y ago
    [deleted]
- 6y ago
thecleaner 6y ago
Maybe that is why these kinds of projects are picked up by private companies ? I believe currently in the USA a pursuit of profit is allows you make far less mistakes than a pursuit of public good. Sometimes the two overlap but when they dont we frequently see the former being allowed far more mistakes.
dekhn 6y ago
I was a medical researcher and also a data engineer for a biotech and I've never seen a dataset that required background checks, or "fingerprint locked rooms" (fingerprints are not biometric security). Instead, a partner company (a well-known, multibillion dollar biotech) handed us a pile of hard drives that were encrypted by trivial keys. I sat down with the CEO and discussed our plans (to upload the data to our S3 buckets with ACLs) along with all our other security plans, pointed out the two of us were professionally and legally on the hook (her more than me, as I was not an executive and had limited liability), and what I was doing to protect the clinical data we had. We looked at our plans, and ultimatley concluded we could both sleep happily at night with what we were doing. It was definitely a lot of work to maintain the ACLs and monitoring and checking to make sure we weren't leaking data, and a lot of education for our scientists. Even if something negative had happened, we would have been able to foresically detect it, and work with legal authorities with confidence we had done a good-faith, better-than-average practices implementation.
ryandrake 6y ago
> Nothing will happen to these people. Not true. They may end up putting out a press release explaining how "Here at Company X, we take security very seriously. Securing data is very important, and we will investigate ourselves to understand if we did anything wrong." So yea, consequences!

mvn9 6y ago

Somebody has to work for those who collect all the leaked data and use it for whatever is possible. You wouldn't earn fame but money for being a data pirate. Or you create a company and start with a fresh name. On the internet, nobody knows that you are a dog.

g_p 6y ago

In contrast here, this data likely came from an insurance company, which got access to the data from their claims. They don't inherently care about these people, because they're all seen as "costs" to the balance sheet profits for the year. Sending their data out for "AI" to try and learn how to predict which claims can be rejected is a rational thing to do, in their eyes, whatever the cost (to those concerned and their data), since it might help improve their bottom line by rejecting claims faster. At least there's a cynical take on this. They have no inherent interest in protecting this data, absent external pressures or legislation that imposes penalties. It strikes me the real issue is the lack of negative inventive for people and businesses who screw up with data. If you build something physical that falls down and hurts people, your investor money ends up paying compensation. In tech though, the goal seems to be to make the revenues without ever touching the responsibility. The story over the weekend of Amazon trying to avoid being deemed liable as a seller for goods they ship, (often) deliver, warehouse, market, dispatch and receive seems to sum this up. A race to the bottom when it comes to protections - Amazon effectively is a shop selling goods from others, as it all happens on their terms. We see similar in physical retail where the manufacturer or reseller has a retention of title over goods in the store, and that doesn't affect the store being responsible. It seems like the "being responsible for what you build" is very much lacking in tech companies, due to lack of external regulation to achieve it. Same here in this case.

harha 6y ago
We've tried this for a while (companies using customers data) and I see a strong case for governments to start protecting users more strongly, because the current way this is being treated isn't working (medical data needs to be handled with greatest care everywhere else, why not here?). The user already pays for the insurance (or hardware or service in other cases), so the data collected should only be used for those purposes, e.g., handling the claim and not building AI models without the necessary care. The customer should have the right to sell their own data to a company they see fit to take care of it appropriately and that compensates them for it.
- pc86 6y ago
  The problem is with ownership I think. I'm not German so I'm really only speaking from what I've seen online (mostly here), but my understanding is that for Germans in particular (maybe EU writ large), you own data about you regardless of how it is collected or who has custody of it. In the US, it seems like the data about people is owned by whoever creates that data (in this case, an insurance company), so they are free to do with that data whatever they like provided it doesn't violate some other non-data-specific law.

mola 6y ago

When the mantra of a huge swathe of business is "disruption" "move fast and break things" etc, well, we end up with a broken disrupted system. Innovation is good, but stability is also good. Instead of optimizing only for dynamism we need to optimize for both. The reason you'll get yourself in trouble and these people won't is because you are operating in different moral systems. Where they operate, the moral thing to do is whatever it takes to succeed, it's entirely moral to "fake it until you make it" to buy media articles promoting your product, to sell media space your readers believe is actually curated, to the highest bidder. You operate in the "ivory tower" they hate, because it's slow and puts up boundaries that limit people from succeeding at all cost, that makes you think about the consequences of your actions. They call it slow and inefficient, I call efficient in the long run.

thinkharderdev 6y ago
I'm not sure that framing makes a lot of sense here. This was a SaaS product for process automation which seems like a perfectly reasonable product. I would argue in fact that having tech companies (meaning companies which have specific expertise in building software products as opposed to companies that have a expertise in something else and also have an "IT department" bolted on) manage sensitive data is likely to make it more secure rather than less. Companies and other institutions which don't have any particular expertise in software engineering and security aren't likely to do a good job. Of course when those companies do screw up they should be held liable though, up to an including criminal liability is the leak is bad enough and the exploited vulnerability is egregious enough.
unishark 6y ago
Universities can be punished by NIH in addition to the feds if they violate privacy rules in clinical studies. Columbia was threatened with losing all their research funding if they didn't get their act together after a big breach a few years ago. But I know plenty academic researchers with a "move fast and break things" and "ignore all rules" attitude. In any highly-competitive system there will be those that gain an advantage by being amoral while everyone else follows the rules. Healthcare is probably especially vulnerable to this as the rules (and legitimate companies following them) has resulted in healthcare lagging far behind other industries in terms of new technologies.

sillysaurusx 6y ago

This is a wild exaggeration. First of all, everyone is insecure. Work as a pentester, and you'll see that the whole world is insecure. After the 30th project with an SQL injection vuln or misconfiguration, you'll never be phased again. Secondly, this seems to be a straightforward database misconfiguration. The database was set to public with no restrictions. It likely wasn't accessible to anyone who wasn't explicitly looking for it, i.e. security researchers or black hats. Yes, this was horrible, but nothing bad happened here. The database was simply set to private. Third, my friend, chillax a bit. Yes, it's extraordinarily important to protect people's medical data. But no, you're not personally liable if a fuckup happens. Everyone fucks up. That's why the fuckups are covered by legal protections. You take your job seriously. Good! But you also take your job way too seriously. Bad. You're gonna burn yourself out within a decade with this mindset. None of this is to say that what happened didn't matter. Of course it matters. But you are literally saying you'd be begging to be let into prison. That's not proportionate, and feels like a reflection of the current social climate of retribution-as-forgiveness.

pc86 6y ago
What's more, it's important to take in the context of the data. It's not mental health, or HIV status, or sexual history, or really of any nature that can do damage other than the privacy aspect. It's people who hurt their neck after a car accident. Not the most sensitive stuff in the world. Just to be explicit about it, I'm not suggesting the release of this data isn't a problem. Just that as medical data goes this is about an innocuous as it could have been. Even day-to-day visits to your primary care doc would probably have references to much more sensitive health data.
- nitrogen 6y ago
  It's more sensitive than you'd think if any claims or settlements were involved. Or employers might want to snoop on their injured employees to find a reason to deny leave.
- deleted 6y ago
  [deleted]
6y ago

thecleaner 6y ago

Maybe that is why these kinds of projects are picked up by private companies ? I believe currently in the USA a pursuit of profit is allows you make far less mistakes than a pursuit of public good. Sometimes the two overlap but when they dont we frequently see the former being allowed far more mistakes.

dekhn 6y ago

I was a medical researcher and also a data engineer for a biotech and I've never seen a dataset that required background checks, or "fingerprint locked rooms" (fingerprints are not biometric security). Instead, a partner company (a well-known, multibillion dollar biotech) handed us a pile of hard drives that were encrypted by trivial keys. I sat down with the CEO and discussed our plans (to upload the data to our S3 buckets with ACLs) along with all our other security plans, pointed out the two of us were professionally and legally on the hook (her more than me, as I was not an executive and had limited liability), and what I was doing to protect the clinical data we had. We looked at our plans, and ultimatley concluded we could both sleep happily at night with what we were doing. It was definitely a lot of work to maintain the ACLs and monitoring and checking to make sure we weren't leaking data, and a lot of education for our scientists. Even if something negative had happened, we would have been able to foresically detect it, and work with legal authorities with confidence we had done a good-faith, better-than-average practices implementation.

ryandrake 6y ago

> Nothing will happen to these people. Not true. They may end up putting out a press release explaining how "Here at Company X, we take security very seriously. Securing data is very important, and we will investigate ourselves to understand if we did anything wrong." So yea, consequences!