4 ms·
To start with, we can add stars (pay with prestige). Getting more into science fiction--but very possible science fiction--we can put data on the blockchain and
by akarve 9y ago
To start with, we can add stars (pay with prestige). Getting more into science fiction--but very possible science fiction--we can put data on the blockchain and let people transact. The data owner would get the lion's share of the transaction.
- RangerScience 9y agoHmm. I can see some ways for this to work. Awhile ago I started a project converting government voting records (both elections and congressional) into a database. Would that be interesting to you? Here's an idea: You could also host a data bounty program, and/or start a grant program for the production of these data sets. Still missing an answer to "what are you requirements"? How do you verify data quality, etc? What format(s)?
- kevinemoore 9y agoCurrently, we support 2 "targets" a Pandas DataFrame and a file. Files can be any format. The Quilt build logic uses Pandas to read files into DataFrames so any format Pandas can read should work in Quilt to create a DataFrame node.
- akarve 9y agoWe support arbitrary data formats in that Quilt falls back to a raw copy if it can't parse the file. On the columnar side (things we convert to Parquet) we support XLS, CSV, TSV, and actually anything that `pandas.read_csv` can parse. We use pandas and pyarrow for column type inference. We want to add a "data linter" that checks data against user-provided rules, and welcome such feature requests on GitHub or in our Slack Channel.