38 ms·

Open Biology Data

34 points by elkingtowa 12y ago

folli 12y ago
I consider myself an enthusiastic bioinformatician, but one thing that drives me up to wall on a regular basis are the thousands of "standardized" formats to archive and share data. A small example: given a genome which is commonly saved as FASTA format (a simple text file containing the nucleotide sequence having a header denoted by a greater-than symbol; one of the few cases where there is only one format currently used, maybe just because no one was creative enough to invent something different), the annotation of the genes (position, direction, name, function etc. of genes) is most commonly stored as either GFF (where in turn there are several versions such as GTF, GFF2 and GFF3 still used), BED files and GenBank format. Those formats only describe the tabular setup regarding how the information is stored. Whether the actual name of a gene is stored in the identifier "ID" or "name" or "gene_name" or whatever is still up to whichever undergrad got the honor to submit the annotation on that day.
- rpedela 12y ago
  Yeah that can be frustrating especially considering most of the data seems like it can be stored as CSV, JSON, etc based on my very quick look. I think the naming issue will likely always be there since I don't know of any highly successful data format which dictates the specific names to use. Sometimes there are name formatting conventions, but that is a little different.
- davecap1 12y ago
  Hey folli, I'm the CTO of SolveBio (https://www.solvebio.com https://www.solvebio.com). We're a startup that's trying to make it super easy to access biological data (specifically for clinical genetic test pipelines). We're well aware of the "file format problem", and we're trying to tackle it by building easy to use APIs into our datasets. This allows for "format-less" retrieval of data (basically JSON) which can be exported into basically any format through a set of helper tools. We're looking for frustrated bioinformaticians to work with us and help develop our platform, so feel free to shoot me an email if you're interested.
- jerven 12y ago
  You should have a lookat rdf and json-ld. More and more bio databases are supporting these serialisations of their data. Quite a few even allow sparql access (sql over http for semantic graphs) e.g. rd.ebi.ac.uk and beta.sparql.uniprot.org among others. there is also work to make data stored in existing databases and files available by sparql without transforming the data first.

34 points by elkingtowa 12y ago

folli 12y ago

I consider myself an enthusiastic bioinformatician, but one thing that drives me up to wall on a regular basis are the thousands of "standardized" formats to archive and share data. A small example: given a genome which is commonly saved as FASTA format (a simple text file containing the nucleotide sequence having a header denoted by a greater-than symbol; one of the few cases where there is only one format currently used, maybe just because no one was creative enough to invent something different), the annotation of the genes (position, direction, name, function etc. of genes) is most commonly stored as either GFF (where in turn there are several versions such as GTF, GFF2 and GFF3 still used), BED files and GenBank format. Those formats only describe the tabular setup regarding how the information is stored. Whether the actual name of a gene is stored in the identifier "ID" or "name" or "gene_name" or whatever is still up to whichever undergrad got the honor to submit the annotation on that day.

rpedela 12y ago
Yeah that can be frustrating especially considering most of the data seems like it can be stored as CSV, JSON, etc based on my very quick look. I think the naming issue will likely always be there since I don't know of any highly successful data format which dictates the specific names to use. Sometimes there are name formatting conventions, but that is a little different.
davecap1 12y ago
Hey folli, I'm the CTO of SolveBio (https://www.solvebio.com https://www.solvebio.com). We're a startup that's trying to make it super easy to access biological data (specifically for clinical genetic test pipelines). We're well aware of the "file format problem", and we're trying to tackle it by building easy to use APIs into our datasets. This allows for "format-less" retrieval of data (basically JSON) which can be exported into basically any format through a set of helper tools. We're looking for frustrated bioinformaticians to work with us and help develop our platform, so feel free to shoot me an email if you're interested.
jerven 12y ago
You should have a lookat rdf and json-ld. More and more bio databases are supporting these serialisations of their data. Quite a few even allow sparql access (sql over http for semantic graphs) e.g. rd.ebi.ac.uk and beta.sparql.uniprot.org among others. there is also work to make data stored in existing databases and files available by sparql without transforming the data first.