4 ms·
You can use AWS to host open datasets: https://aws.amazon.com/opendata/public-datasets/ https://aws.amazon.com/opendata/public-datasets/ These are some other p
by eindiran 8y ago
You can use AWS to host open datasets: https://aws.amazon.com/opendata/public-datasets/ https://aws.amazon.com/opendata/public-datasets/
These are some other people working in roughly the same space:
http://datproject.org/ http://datproject.org/
http://www.pachyderm.io/ http://www.pachyderm.io/
But it does seem like Quilt is a go-to, if you are looking for a "Github for data" host.
- jmaxfield 8y agoI use Quilt pretty much daily and while I like AWS open datasets I don't think it is as actively developed on as Quilt is. DAT project on the other hand I really do like as a way to simply transfer large amounts of data between contributors, that said, if you are just trying to get data out there and have people use it freely for their own work I think Quilt presents the solution due to searchable and easily understood python (and I think an R repo) usage of datasets.
- jdoliner 8y agoPachyderm founder here. We're not really a data hosting provider, although we may offer that in the future. Right now Pachyderm is more like intranet data hosting for companies. You have to spin up your own Kubernetes cluster and deploy Pachyderm on it. It's also not normally used to download data onto your local machine for processing because it has its own computation layer which allows you to run code at scale and tracks the provenance of the data to keep things reproducible.