4 ms·
What's the relevance of NVMe? Is that I/O latency still relevant in the presence of networking and scheduling overheads?
by jey 8y ago
What's the relevance of NVMe? Is that I/O latency still relevant in the presence of networking and scheduling overheads?
- jamesblonde 8y agoIf your V-100 GPU can process 150 images/sec, can S3 keep up? What if you now have 10 GPUs or 100 GPUs? Now, HDFS can't keep up. But HopsFS can....
- dekhn 8y agoMany people on these forums train directly from S3 and know how to get that kind of throughput. Like any blobject store, the key is to use multithreaded readers and high concurrency and set up your storage properly.
- jey 8y agoInteresting. What's the bottleneck for HDFS, is it that the namenode has high latency under load?