4 ms·
As a followup to this. We have now successfully run complex statistical models across all 2.5 million snps on a single AWS instance in less than 3 hours just by
by nstrayer 7y ago
As a followup to this. We have now successfully run complex statistical models across all 2.5 million snps on a single AWS instance in less than 3 hours just by writing R code using the package I describe at the end of the article.
- wikibob 7y agoHave you considered using AWS EC2 Spot Instances? The price can be 50 to 70% cheaper. You would have to add some additional reliability to your pipeline so it could continue processing when instances are unexpectedly terminated. But this might be well worth it as it sounds like your research group is cost-constrained. AWS made some changes this year so the spot prices are more stable and instances don't get shut down as frequently.
- nstrayer 7y agoI did use spot instances for most of the clusters and a few of the processing jobs! I got out of the habit of using them earlier due to loosing them but now that they have the 'pay up to the on-demand price' option they're great!