3 ms·
How do you even get the petabytes of data into S3 without incurring massive charges? Previous estimations of getting the initial dataset into cloud was.. prohi
by r00fus 5y ago
How do you even get the petabytes of data into S3 without incurring massive charges? Previous estimations of getting the initial dataset into cloud was.. prohibitive (note this was 3-4 years ago).
- laurencerowe 5y agoWhat charges in particular are you referring to? The cloud services only charge data egress fees. Data ingress is free. The ENCODE Project data were a hundred terabytes or so at the point I migrated it into S3 (it's now over a petabyte.) We were on a fast academic network with a 100GB interconnect so could just `aws s3 sync` rather than worry about using AWS Snowballs or anything.
- nicornk 5y agoUploading data into AWS is free.