3 ms·
Noob question: when to use S3 vs HDFS?
by The_rationalist 6y ago
Noob question: when to use S3 vs HDFS?
- jamesblonde 6y agoFor cost on AWS, use S3. AWS price local storage on VMs to make HDFS not competitive. If you want the HDFS API but S3 storage costs on AWS, use HopsFS-S3 - https://www.logicalclocks.com/blog/hopsfs-100x-times-faster-than-aws-s3 https://www.logicalclocks.com/blog/hopsfs-100x-times-faster-... (disclosure: work on HopsFS). HDFS provides POSIX-like API, and now has atomic metadata operations that S3 doesn't (mv/rename, chown, chmod) and append for files.