4 ms·
> If someone wants to host the raw files to allow others to download it let me know. It is a 83 GB tar.gz file which uncompressed is just over 1 TB in size. So
by robbya 7y ago
> If someone wants to host the raw files to allow others to download it let me know. It is a 83 GB tar.gz file which uncompressed is just over 1 TB in size.
Some of the cloud providers have free hosting for public data sets (people who use the data incur cost to download/process the data). I'm not sure if this would qualify.
* https://aws.amazon.com/opendata/public-datasets/ https://aws.amazon.com/opendata/public-datasets/
* https://azure.microsoft.com/en-us/services/open-datasets/ https://azure.microsoft.com/en-us/services/open-datasets/
- lunixbochs 7y agoI similarly have a 127GB tar.xz of about 2TB of code/files from the top GitHub repositories (filtered by highest starred repositories per primary language) as of around August 2017 if anyone wants it or wants to host it.
- z3t4 7y agoMake a bit torrent
- trevyn 7y agoYou could also use an S3 or GCS “requester pays” bucket, which means you never get charged for data transfer. Would cost about $2/month for the 83GB storage, but that’s it. https://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html https://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPay... https://cloud.google.com/storage/docs/requester-pays https://cloud.google.com/storage/docs/requester-pays
- bauruine 7y agoOr you could just put it on some Hetzner box. I would do that but I'm not sure how to contact him.
- polygot 7y agoI'm very interested in this data; if there is a torrent, I would be able to seed it if needed.