3 ms·
No first hand experience here but my understanding is that HDFS and Hadoop in general are geared towards high-throughput and not necessarily low latency. You'll
by skorgu 17y ago
No first hand experience here but my understanding is that HDFS and Hadoop in general are geared towards high-throughput and not necessarily low latency. You'll likely have to put a caching layer in front of it if this is still [1] the case. There is already some [2] HTTP accessibility for HDFS, I'm again not 100% clear on its status but if it works well enough a nice caching proxy (varnish) with plenty of memory should help with latency on heavily-accessed files.
[1] http://www.nabble.com/Using-HDFS-to-serve-www-requests-td22725659.html http://www.nabble.com/Using-HDFS-to-serve-www-requests-td227...
[2] https://issues.apache.org/jira/browse/HADOOP-5010 https://issues.apache.org/jira/browse/HADOOP-5010