3 ms·
It was useful at the time though. Both network and disk were substantially slower at the time. And servers had far fewer cores. So which server stored the data
by mbreese 3y ago
It was useful at the time though. Both network and disk were substantially slower at the time. And servers had far fewer cores. So which server stored the data could make your processing faster. And you’d have less risk for IO contention within the same chassis.
Fast forward a few years and compute servers can have hundreds of cores and the network is significantly faster.
I think it was a “good enough” design for the time and those assumptions. And batch processing. Now? I think there are different assumptions to be made.
(And I think there is a good argument that by the time Hadoop/HDFS had stabilized the assumptions had already shifted)