4 ms·
Integrating NVMe Disks in HopsFS (HDFS)
- SirOibaf 8y ago"HDFS and S3 are designed around large blocks (optimized to overcome slow random I/O on disks), while new NVMe hardware supports fast random disk I/O (and potentially small blocks sizes). " Interesting solution presented here - keep the block size constant, but put the small files on NVMe disks.
- senorcarbone 8y agosuper cool. The world needs more success stories for NVM
- jamesblonde 8y agoDisclaimer, one of the authors here. You can see a video with discussion of the NVMe support in HopsFS in this Spark Summit talk: https://www.youtube.com/watch?v=tx6HyoUYGL0 https://www.youtube.com/watch?v=tx6HyoUYGL0