5 ms·
What's a small write in your use case? How many KBs is the write op, and how many ops/sec? We can store small files on NVMe disks in metadata, so that you can
by jamesblonde 6y ago
What's a small write in your use case? How many KBs is the write op, and how many ops/sec?
We can store small files on NVMe disks in metadata, so that you can read them with a latency of just a couple of ms, and can scale to 10s of thousands of concurrent reads/writes per second with small files.
https://www.logicalclocks.com/blog/millions-and-millions-of-files-for-deep-learning-with-hopsfs https://www.logicalclocks.com/blog/millions-and-millions-of-...