5 ms·

Anyone know of open datasets one could play around with?

acomjean 10y ago
for genetic data ensemble.org has a lot. http://useast.ensembl.org/info/about/species.html http://useast.ensembl.org/info/about/species.html for example Humans: You end up here ftp://ftp.ensembl.org/pub/release-86/fasta/homo_sapiens/dna/ there is a read me which helps figure out what this stuff is, or ask your friendly neighborhood biologist. ftp://ftp.ensembl.org/pub/release-86/fasta/homo_sapiens/dna/README
- jerven 10y ago
  To be honest, you should probably look at their RDF dumps if you are from a CS background or you will end up in Ensembls Perl/MySQL swamp.
searine 10y ago
All US publicly funded sequencing data is required to be posted on the NIH's Sequence Read Archive within a few months of it's generation. There are petabytes of open data there for you pick from every kind of organism. Go nuts. https://www.ncbi.nlm.nih.gov/sra https://www.ncbi.nlm.nih.gov/sra
abetusk 10y ago
The Harvard Personal Genome Project Over 200 whole genomes and I think over 500 genotyping data (23andMe and the like) released under a CC0 license with sporadic phenotype data [1]. Open Humans [2] has a bunch of data with a convenient API [3]. OpenSNP has a lot of genotyping data (23andMe etc.) available for download [4]. For a more comprehensive list check out one of the many "Awesome Public Datasets" [5] (biology section). [1] https://my.pgp-hms.org/public_genetic_data https://my.pgp-hms.org/public_genetic_data [2] https://www.openhumans.org/members/ https://www.openhumans.org/members/ [3] https://www.openhumans.org/public-data-api/ https://www.openhumans.org/public-data-api/ [4] https://opensnp.org/ https://opensnp.org/ [5] https://github.com/caesar0301/awesome-public-datasets#biology https://github.com/caesar0301/awesome-public-datasets#biolog...
- tetron 10y ago
  Incidentally, Arvados (http://arvados.org http://arvados.org) is the software used to host the Harvard PGP data, and is a free software platform for managing large scale storage and analysis aimed at scientific workloads.

for genetic data ensemble.org has a lot. http://useast.ensembl.org/info/about/species.html http://useast.ensembl.org/info/about/species.html for example Humans: You end up here ftp://ftp.ensembl.org/pub/release-86/fasta/homo_sapiens/dna/ there is a read me which helps figure out what this stuff is, or ask your friendly neighborhood biologist. ftp://ftp.ensembl.org/pub/release-86/fasta/homo_sapiens/dna/README

jerven 10y ago
To be honest, you should probably look at their RDF dumps if you are from a CS background or you will end up in Ensembls Perl/MySQL swamp.

searine 10y ago

All US publicly funded sequencing data is required to be posted on the NIH's Sequence Read Archive within a few months of it's generation. There are petabytes of open data there for you pick from every kind of organism. Go nuts. https://www.ncbi.nlm.nih.gov/sra https://www.ncbi.nlm.nih.gov/sra

abetusk 10y ago

The Harvard Personal Genome Project Over 200 whole genomes and I think over 500 genotyping data (23andMe and the like) released under a CC0 license with sporadic phenotype data [1]. Open Humans [2] has a bunch of data with a convenient API [3]. OpenSNP has a lot of genotyping data (23andMe etc.) available for download [4]. For a more comprehensive list check out one of the many "Awesome Public Datasets" [5] (biology section). [1] https://my.pgp-hms.org/public_genetic_data https://my.pgp-hms.org/public_genetic_data [2] https://www.openhumans.org/members/ https://www.openhumans.org/members/ [3] https://www.openhumans.org/public-data-api/ https://www.openhumans.org/public-data-api/ [4] https://opensnp.org/ https://opensnp.org/ [5] https://github.com/caesar0301/awesome-public-datasets#biology https://github.com/caesar0301/awesome-public-datasets#biolog...

tetron 10y ago
Incidentally, Arvados (http://arvados.org http://arvados.org) is the software used to host the Harvard PGP data, and is a free software platform for managing large scale storage and analysis aimed at scientific workloads.