5 ms·

According to them, they got approval for doing that: > The alternative was to upload it to the cloud using tools such as Google Storage and use BigQuery to ext

by ZoFreX 13y ago

According to them, they got approval for doing that: > The alternative was to upload it to the cloud using tools such as Google Storage and use BigQuery to extract data from it. As PA has an existing relationship with Google, we pursued this route (with appropriate approval). This showed that it is possible to get even sensitive data in the cloud and apply proper safeguards.

deleted 13y ago
[deleted]
- icebraining 13y ago
  They didn't say they had approval from Google. That wouldn't make much sense, you don't really need approval from Google to use BigQuery, you just need to open a Google account and create a new BigQuery project on their Developers Console
  - tszming 13y ago
    Given that the scale of the data, it is likely that they need to contact Google to get rid of the default quotas: https://developers.google.com/bigquery/quota-policy https://developers.google.com/bigquery/quota-policy
    - icebraining 13y ago
      Why? I don't see any limit being hit by loading 27 DVDs worth of data.
atmosx 13y ago
So apparently they don't have money to spend on building a secure, state-owned cloud storage service for patient records right? I wonder, does the GCHQ use private cloud storage contractors or for spying material the government found money to create an appropriate database? :-)
- boomzilla 13y ago
  This is just crazy. According to the article, this "big data" fits in 27 DVDs, which is roughly 1-2 TB of data. Do you really need Big Query or whatever for this?
  - Zirro 13y ago
    According to Wikipedia, DVDs of the largest capacity can hold 17.08 GB of data, but those are rare. If the article is correct about these details, the data could be ~460 GB at most, but is likely less than half of that if the DVDs in question were of normal capacity.
  - yapcguy 13y ago
    50 x 50GB blu-ray discs, for over 2TB of storage, only costs $126 at Amazon. A 2TB portable USB drive costs $100. But no, let's just upload everything to Google Drive.
    - fixermark 13y ago
      I would assume it's not about storage; it's about processing. The bold byline of BigQuery (https://developers.google.com/bigquery/ https://developers.google.com/bigquery/) is "Analyze terabytes of data in seconds." Store the data locally? Easily. Set up the necessary back-end infrastructure to sift, twist, and churn the data efficiently? Now you're talking hiring some people to set up an infrastructure for that... Or you could upload it to Google and use their infrastructure.
Silhouette 13y ago
And what "appropriate approval" was that, exactly? In general, exporting personal data outside of the EEA requires the explicit notification of the data subject under UK data protection law (among other consequences of the first Principle[1]). Moreover, the rules for even processing sensitive personal information, which includes health-related information, are significantly stronger than the general case. They should never had been given that data in the first place, of course, and giving it to them should clearly be illegal on the part of whoever disclosed it. If it turns out not to have been, that will be a compelling case for dramatically strengthening the legal data protection and privacy framework in the UK. But I don't see how either the original source or PA Consulting can get around the basic conditions for processing sensitive personal data[2]. In particular, the most likely condition they might appeal to here in the absence of explicit consent reads: "The processing is necessary for medical purposes, and is undertaken by a health professional or by someone who is subject to an equivalent duty of confidentiality." [Emphasis added] Even once they had it, that still doesn't give them a free pass on exporting the data outside the EEA without notification (see [3]), or actually processing the data themselves for that matter. [1] http://ico.org.uk/for_organisations/data_protection/the_guide/principle_1 http://ico.org.uk/for_organisations/data_protection/the_guid... [2] http://ico.org.uk/for_organisations/data_protection/the_guide/conditions_for_processing#conditions-sensitive-data http://ico.org.uk/for_organisations/data_protection/the_guid... [3] http://ico.org.uk/for_organisations/data_protection/the_guide/principle_8 http://ico.org.uk/for_organisations/data_protection/the_guid...
- nolok 13y ago
  Doesn't the "safe harbor" clause apply to US companies ? You know, that joke of a clause which says that US companies fit the needs of our data protection law as long as they claim to fit it (and they only have to claim it) ? Part of the new data protection law that was supposed to be voted in the EU following/during the PRISM scandal was revoking that stupid clause but I'm not sure what happened to that reform.
  - Silhouette 13y ago
    Safe Harbor avoids some of the issues with exporting data from the EEA at all. It doesn't cover exporting the data without notifying the subjects appropriately. It isn't even close to covering exporting sensitive personal data (which is a technical term explicitly including health-related information), for which much stronger rules apply under UK data protection legislation.

by ZoFreX 13y ago

deleted 13y ago

[deleted]

icebraining 13y ago
They didn't say they had approval from Google. That wouldn't make much sense, you don't really need approval from Google to use BigQuery, you just need to open a Google account and create a new BigQuery project on their Developers Console
- tszming 13y ago
  Given that the scale of the data, it is likely that they need to contact Google to get rid of the default quotas: https://developers.google.com/bigquery/quota-policy https://developers.google.com/bigquery/quota-policy
  - icebraining 13y ago
    Why? I don't see any limit being hit by loading 27 DVDs worth of data.

atmosx 13y ago

So apparently they don't have money to spend on building a secure, state-owned cloud storage service for patient records right? I wonder, does the GCHQ use private cloud storage contractors or for spying material the government found money to create an appropriate database? :-)

boomzilla 13y ago
This is just crazy. According to the article, this "big data" fits in 27 DVDs, which is roughly 1-2 TB of data. Do you really need Big Query or whatever for this?
- Zirro 13y ago
  According to Wikipedia, DVDs of the largest capacity can hold 17.08 GB of data, but those are rare. If the article is correct about these details, the data could be ~460 GB at most, but is likely less than half of that if the DVDs in question were of normal capacity.
- yapcguy 13y ago
  50 x 50GB blu-ray discs, for over 2TB of storage, only costs $126 at Amazon. A 2TB portable USB drive costs $100. But no, let's just upload everything to Google Drive.
  - fixermark 13y ago
    I would assume it's not about storage; it's about processing. The bold byline of BigQuery (https://developers.google.com/bigquery/ https://developers.google.com/bigquery/) is "Analyze terabytes of data in seconds." Store the data locally? Easily. Set up the necessary back-end infrastructure to sift, twist, and churn the data efficiently? Now you're talking hiring some people to set up an infrastructure for that... Or you could upload it to Google and use their infrastructure.

Silhouette 13y ago

And what "appropriate approval" was that, exactly? In general, exporting personal data outside of the EEA requires the explicit notification of the data subject under UK data protection law (among other consequences of the first Principle[1]). Moreover, the rules for even processing sensitive personal information, which includes health-related information, are significantly stronger than the general case. They should never had been given that data in the first place, of course, and giving it to them should clearly be illegal on the part of whoever disclosed it. If it turns out not to have been, that will be a compelling case for dramatically strengthening the legal data protection and privacy framework in the UK. But I don't see how either the original source or PA Consulting can get around the basic conditions for processing sensitive personal data[2]. In particular, the most likely condition they might appeal to here in the absence of explicit consent reads: "The processing is necessary for medical purposes, and is undertaken by a health professional or by someone who is subject to an equivalent duty of confidentiality." [Emphasis added] Even once they had it, that still doesn't give them a free pass on exporting the data outside the EEA without notification (see [3]), or actually processing the data themselves for that matter. [1] http://ico.org.uk/for_organisations/data_protection/the_guide/principle_1 http://ico.org.uk/for_organisations/data_protection/the_guid... [2] http://ico.org.uk/for_organisations/data_protection/the_guide/conditions_for_processing#conditions-sensitive-data http://ico.org.uk/for_organisations/data_protection/the_guid... [3] http://ico.org.uk/for_organisations/data_protection/the_guide/principle_8 http://ico.org.uk/for_organisations/data_protection/the_guid...

nolok 13y ago
Doesn't the "safe harbor" clause apply to US companies ? You know, that joke of a clause which says that US companies fit the needs of our data protection law as long as they claim to fit it (and they only have to claim it) ? Part of the new data protection law that was supposed to be voted in the EU following/during the PRISM scandal was revoking that stupid clause but I'm not sure what happened to that reform.
- Silhouette 13y ago
  Safe Harbor avoids some of the issues with exporting data from the EEA at all. It doesn't cover exporting the data without notifying the subjects appropriately. It isn't even close to covering exporting sensitive personal data (which is a technical term explicitly including health-related information), for which much stronger rules apply under UK data protection legislation.