2 ms·
The way HBase/HDFS does this is perfectly backwards and counterproductive. You don't want to put a program and its data on one local disk, because that one loca
by jeffbee 6y ago
The way HBase/HDFS does this is perfectly backwards and counterproductive. You don't want to put a program and its data on one local disk, because that one local disk has almost no resources. The later-period thinking at Google is the opposite of shipping the program to the data. Instead, you allocate the data randomly over as many spindles as you can find, then when you read it all back out you very briefly use all of those spindles at once. Really, it's something to behold in action.
That's batch processing: mapreduce and flume and whatnot. Search is still very much an exercise in getting the queries out to where the index shards live.
- Xorlev 6y agoDid you mean Hadoop/HDFS? HBase works pretty analogously to Bigtable (though, having run HBase in production at $dayjob-1, I'd take the latter any day).
- jeffbee 6y agoHBase doesn't work analogously to bigtable. HBase intentionally tries to move region files onto the same node as the region server. That is ass-backwards. HBase devs think of this as "good locality" but all you're really getting from it is terrible performance.
- ithkuil 6y agoThe secret sauce for that is networking. Once you have a domain with full clos connectivity and very low latencies you can start disaggregating storage from compute again
- ddorian43 6y agoAssuming you have a 1GB file. According to Colossus paper, that gets split into 1MB parts, so 1000. And they probably use erasure-coding for replication. How does it work ? Do they EC each 1000 part separately, or ?
- jeffbee 6y agoAccording to these slides, each stripe is a separate replication group. Although the figure depicts replicated rather than erasure coded files, I think it's safe to imagine that the idea holds for both. http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Keynote.pdf http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke...