5 ms·
Two questions: 1. What do people use (currently) for data standardization? 2. What do you do if it's not in tabular format, but is something like JSON/noSQL?
by arxpoetica 8y ago
Two questions:
1. What do people use (currently) for data standardization?
2. What do you do if it's not in tabular format, but is something like JSON/noSQL?
- meej 8y agoI use RDF and standard open schemas like schema.org or VIVO-ISF, with local schema extensions as needed. The datasets I need to standardize tend to be things like scholarly research networking data or data about healthcare clinics and providers, which tend to be highly connected with some sparsely populated fields, both of which lend themselves well to a graph-based model. Right now I use the Karma Data Integration tool (http://usc-isi-i2.github.io/karma/ http://usc-isi-i2.github.io/karma/) to transform my source data to RDF triples. It can handle data in a variety of shapes, not just tabular, though it does struggle with say, highly-nested XML data. I want to try LinkedPipes ETL (https://etl.linkedpipes.com/ https://etl.linkedpipes.com/) on my next project, whatever it might be.
- alexyaseen 8y agoAt risk of shamelessly promoting, we built Parabola (https://parabola.io https://parabola.io) with many of these same best practices in mind. HN discussion: https://news.ycombinator.com/item?id=17147272 https://news.ycombinator.com/item?id=17147272
- arxpoetica 8y agoAh, yes, I remember Parabola.