4 ms·

Regarding small files, HopsFS can store small files in the metadata layer for improved performance. https://kth.diva-portal.org/smash/record.jsf?pid=diva2:1260

• jamesblonde 6y ago
  Yes, this paper did not compare small file performance in HopsFS with EMRFS/S3. But HopsFS on S3 also supports storing small files on NVMe disks in the metadata layer. Previously, we have shown that we can get up to 66X higher throughput for writes for small files versus HDFS (peer reviewed, of course) - https://www.logicalclocks.com/blog/millions-and-millions-of-files-for-deep-learning-with-hopsfs https://www.logicalclocks.com/blog/millions-and-millions-of-...