5 ms·

I work at an academic institution that generates and analyzes large amounts of genomic data (clinical sequencing of cancer patients). We were really impressed w

by i000 11y ago

I work at an academic institution that generates and analyzes large amounts of genomic data (clinical sequencing of cancer patients). We were really impressed with the Google cloud and executed several large proof-of-concept projects on it. Honestly, being able to spin up 1000 machines to process 10s of thousands of samples within hours (using cheap preemptible machines) is a huge step forward. The biggest challenge we face is data protection. Sequencing data is not considered protected health information so theoretically HIPAA regulations do not apply, but this may change in future so nobody is willing to take responsibility. If sequencing data is considered PHI, HIPAA requires a BAA between the institution and Google. The process of getting these in place was so convoluted and difficult that it was just easier to use another provider more closely associated with our institution.

timeu 11y ago
I am wondering if you did a cost analysis. Last time I checked a dedicated on-premise HPC was still cheaper than using the cloud. Actually the CPU hours are cheaper in the cloud but data-transfer (bandwith) and storage make it more expensive (might not apply to storage anymore but AFAIK data transfer costs are still an issue). What is your experience ?
- i000 11y ago
  Of course we did this, but cost is not the only consideration. Frankly, CPU-hours and TB-years are still within 5-10% of the total cost providing sequencing results to the patients. What is really important is that the cloud allows us to determine exactly our cost-per-analysis without paying for excess capacity or maintenance. What matters also is reliability, redundancy, and peak capacity, where the cloud (both Google and the other provider) win hands down.
- boulos 11y ago
  Just a note, you don't pay for sending data to any provider (ingress) only for pulling data out / serving it (egress). You can put your GCS buckets behind Cloudflare to take advantage of "interconnect" pricing of $.04/GB. Disclaimer: I work on Compute Engine.
  - timeu 11y ago
    good to know. Thanks for the info !
- laurencerowe 11y ago
  You can use https://aws.amazon.com/directconnect/ https://aws.amazon.com/directconnect/ to reduce egress costs from AWS down to $0.02/GB.
- jsolson 11y ago
  As boulos points out ingress is free -- the other question is how much you actually need to egress. Transfer within a GCP region is generally free as well, so unless you really need your giant result set in its raw form, you can potentially just leave it in GCP and do any analysis/summarizing work you might otherwise have done externally within GCP. Taking this to the extreme, you might even fire up a Windows VM so you could run some analysis or reporting tool there (say, Tableau), within GCP and then only need to egress the Dashboard or Report that it generated. You'd still have the raw data, but you would only be paying to store it. Especially if you're only keeping it around for archival purposes you could put it in a Nearline bucket and really minimize your costs. (note: I work on GCE)
boulos 11y ago
Glad to hear this worked out for you (particularly preemptible VMs which I worked on). Sorry about the HIPAA/ BAA issues, which institution are you with so I can have someone do the work so you can have a better experience next time. Disclaimer: I work on Compute Engine.
- brohoolio 11y ago
  I'd love to do more with Google at my university but we couldn't get Google to even start talking about signing our BAA or other agreements. It was a non-starter for our legal department. We ended up moving towards AWS because they were willing to sign. It sucked from my perspective because we already had such a tie in with Google with our email/ collab platform. In the couple years since then our agreements have only gotten stronger. The best way for Google to address this is to sign something with Internet2 and then the institutions can use that contract instead of negotiating separately.
  - i000 11y ago
    In our case Google was very proactive and willing to sign a BAA. But the terms of those agreements have to satisfy both partners which for some reason ultimately did not happen.
- davidglazer 11y ago
  Yes please -- if you're able to share which institution you're with, I'd like to follow up on the BAA hassles. (We do regularly sign BAAs with institutions, but I know that, as with many things legal, simple things don't always stay simple.) Disclaimer: I work on Google Genomics.
pbnjay 11y ago
Just curious, was this primarily hotspot or other smaller panels? We looked at this internally as well but the available machine types have very limited ram for things like the exome panels we're starting to work with.
- boulos 11y ago
  Please take another look; we introduced custom machine types back in November: http://googlecloudplatform.blogspot.com/2015/11/introducing-Custom-Machine-Types-the-freedom-to-configure-the-best-VM-shape-for-your-workload.html Which would let you adjust your RAM to vCPU ratio (as low as ~1 GiB per vCPU and up to 6.5). Disclaimer: I work on Compute Engine.
kajecounterhack 11y ago
For the layman, is a BAA this? https://en.wikipedia.org/wiki/Broad_Agency_Announcement https://en.wikipedia.org/wiki/Broad_Agency_Announcement (The other terms were a bit easier to google) Here are the other terms: HIPAA: the acronym for the Health Insurance Portability and Accountability Act that was passed by Congress in 1996 PHI: Protected health information
- jrowley 11y ago
  BAA = business associate agreement. Contract between an organization that handles personal health information (e.g. a research lab in a university ) and a business that does work with that data (e.g. Google Genomics). Edit: link to relevant definition http://searchhealthit.techtarget.com/definition/HIPAA-business-associate-agreement-BAA http://searchhealthit.techtarget.com/definition/HIPAA-busine...
- ljoshua 11y ago
  No, it's a Business Associate Agreement, part of HIPAA. (See a sample here: http://www.hhs.gov/hipaa/for-professionals/covered-entities/sample-business-associate-agreement-provisions/index.html http://www.hhs.gov/hipaa/for-professionals/covered-entities/...) Basically an agreement between organizations that there is an understanding that PHI (protected health information) is at stake and that both parties are aware of their obligations to protect it.
tw04 11y ago
Sounds like exactly what NPS (netapp private storage) was designed for. I'm confident a Google version could be spun up if there were the appropriate demand.
imaginenore 11y ago
You could do the same before on AWS. And AWS spot instances are way way cheaper than Google's offerings. I bet your university has hundreds of computers idling at night - why not use them?

by i000 11y ago

timeu 11y ago

I am wondering if you did a cost analysis. Last time I checked a dedicated on-premise HPC was still cheaper than using the cloud. Actually the CPU hours are cheaper in the cloud but data-transfer (bandwith) and storage make it more expensive (might not apply to storage anymore but AFAIK data transfer costs are still an issue). What is your experience ?

i000 11y ago
Of course we did this, but cost is not the only consideration. Frankly, CPU-hours and TB-years are still within 5-10% of the total cost providing sequencing results to the patients. What is really important is that the cloud allows us to determine exactly our cost-per-analysis without paying for excess capacity or maintenance. What matters also is reliability, redundancy, and peak capacity, where the cloud (both Google and the other provider) win hands down.
boulos 11y ago
Just a note, you don't pay for sending data to any provider (ingress) only for pulling data out / serving it (egress). You can put your GCS buckets behind Cloudflare to take advantage of "interconnect" pricing of $.04/GB. Disclaimer: I work on Compute Engine.
- timeu 11y ago
  good to know. Thanks for the info !
laurencerowe 11y ago
You can use https://aws.amazon.com/directconnect/ https://aws.amazon.com/directconnect/ to reduce egress costs from AWS down to $0.02/GB.
jsolson 11y ago
As boulos points out ingress is free -- the other question is how much you actually need to egress. Transfer within a GCP region is generally free as well, so unless you really need your giant result set in its raw form, you can potentially just leave it in GCP and do any analysis/summarizing work you might otherwise have done externally within GCP. Taking this to the extreme, you might even fire up a Windows VM so you could run some analysis or reporting tool there (say, Tableau), within GCP and then only need to egress the Dashboard or Report that it generated. You'd still have the raw data, but you would only be paying to store it. Especially if you're only keeping it around for archival purposes you could put it in a Nearline bucket and really minimize your costs. (note: I work on GCE)

boulos 11y ago

Glad to hear this worked out for you (particularly preemptible VMs which I worked on). Sorry about the HIPAA/ BAA issues, which institution are you with so I can have someone do the work so you can have a better experience next time. Disclaimer: I work on Compute Engine.

brohoolio 11y ago
I'd love to do more with Google at my university but we couldn't get Google to even start talking about signing our BAA or other agreements. It was a non-starter for our legal department. We ended up moving towards AWS because they were willing to sign. It sucked from my perspective because we already had such a tie in with Google with our email/ collab platform. In the couple years since then our agreements have only gotten stronger. The best way for Google to address this is to sign something with Internet2 and then the institutions can use that contract instead of negotiating separately.
- i000 11y ago
  In our case Google was very proactive and willing to sign a BAA. But the terms of those agreements have to satisfy both partners which for some reason ultimately did not happen.
davidglazer 11y ago
Yes please -- if you're able to share which institution you're with, I'd like to follow up on the BAA hassles. (We do regularly sign BAAs with institutions, but I know that, as with many things legal, simple things don't always stay simple.) Disclaimer: I work on Google Genomics.

pbnjay 11y ago

Just curious, was this primarily hotspot or other smaller panels? We looked at this internally as well but the available machine types have very limited ram for things like the exome panels we're starting to work with.

boulos 11y ago
Please take another look; we introduced custom machine types back in November: http://googlecloudplatform.blogspot.com/2015/11/introducing-Custom-Machine-Types-the-freedom-to-configure-the-best-VM-shape-for-your-workload.html Which would let you adjust your RAM to vCPU ratio (as low as ~1 GiB per vCPU and up to 6.5). Disclaimer: I work on Compute Engine.

kajecounterhack 11y ago

For the layman, is a BAA this? https://en.wikipedia.org/wiki/Broad_Agency_Announcement https://en.wikipedia.org/wiki/Broad_Agency_Announcement (The other terms were a bit easier to google) Here are the other terms: HIPAA: the acronym for the Health Insurance Portability and Accountability Act that was passed by Congress in 1996 PHI: Protected health information

jrowley 11y ago
BAA = business associate agreement. Contract between an organization that handles personal health information (e.g. a research lab in a university ) and a business that does work with that data (e.g. Google Genomics). Edit: link to relevant definition http://searchhealthit.techtarget.com/definition/HIPAA-business-associate-agreement-BAA http://searchhealthit.techtarget.com/definition/HIPAA-busine...
ljoshua 11y ago
No, it's a Business Associate Agreement, part of HIPAA. (See a sample here: http://www.hhs.gov/hipaa/for-professionals/covered-entities/sample-business-associate-agreement-provisions/index.html http://www.hhs.gov/hipaa/for-professionals/covered-entities/...) Basically an agreement between organizations that there is an understanding that PHI (protected health information) is at stake and that both parties are aware of their obligations to protect it.

tw04 11y ago

Sounds like exactly what NPS (netapp private storage) was designed for. I'm confident a Google version could be spun up if there were the appropriate demand.

imaginenore 11y ago

You could do the same before on AWS. And AWS spot instances are way way cheaper than Google's offerings. I bet your university has hundreds of computers idling at night - why not use them?