3 ms·
If your V-100 GPU can process 150 images/sec, can S3 keep up? What if you now have 10 GPUs or 100 GPUs? Now, HDFS can't keep up. But HopsFS can....
by jamesblonde 8y ago
If your V-100 GPU can process 150 images/sec, can S3 keep up? What if you now have 10 GPUs or 100 GPUs? Now, HDFS can't keep up. But HopsFS can....
- dekhn 8y agoMany people on these forums train directly from S3 and know how to get that kind of throughput. Like any blobject store, the key is to use multithreaded readers and high concurrency and set up your storage properly.
- jey 8y agoInteresting. What's the bottleneck for HDFS, is it that the namenode has high latency under load?