4 ms·

Wasn't this the aim of Hadoop as well? Also, to be fair, IBM wasn't able to do much with any of the companies they acquired in the same timeframe as Blekko. I

by 2sk21 6y ago

Wasn't this the aim of Hadoop as well? Also, to be fair, IBM wasn't able to do much with any of the companies they acquired in the same timeframe as Blekko. I was working at IBM at that time and witnessed this first hand.

ChuckMcM 6y ago
I kind of gathered that IBM had this sort of love/hate relationship with acquisitions. They loved the "new ideas" but many of the "old guard" hated the idea that something from outside of IBM was novel and worth pursuing. I watched first hand as things that had been done by a company that was acquired were buried while the exact same kind of thing done by IBM research was given lots of funding. I understood not having two efforts, but suggested they be blended to get the best of both teams, alas that was shot down. At one time IBM research had a pretty awesome storage group, perhaps they will will have a computational storage fabric offering at some point.
- 2sk21 6y ago
  I know exactly the IBM Research group you are referring to but won't name names to avoid embarrassment :-) Politics in large organizations can be pretty nasty.
jeffbee 6y ago
The way HBase/HDFS does this is perfectly backwards and counterproductive. You don't want to put a program and its data on one local disk, because that one local disk has almost no resources. The later-period thinking at Google is the opposite of shipping the program to the data. Instead, you allocate the data randomly over as many spindles as you can find, then when you read it all back out you very briefly use all of those spindles at once. Really, it's something to behold in action. That's batch processing: mapreduce and flume and whatnot. Search is still very much an exercise in getting the queries out to where the index shards live.
- Xorlev 6y ago
  Did you mean Hadoop/HDFS? HBase works pretty analogously to Bigtable (though, having run HBase in production at $dayjob-1, I'd take the latter any day).
  - jeffbee 6y ago
    HBase doesn't work analogously to bigtable. HBase intentionally tries to move region files onto the same node as the region server. That is ass-backwards. HBase devs think of this as "good locality" but all you're really getting from it is terrible performance.
- ithkuil 6y ago
  The secret sauce for that is networking. Once you have a domain with full clos connectivity and very low latencies you can start disaggregating storage from compute again
- ddorian43 6y ago
  Assuming you have a 1GB file. According to Colossus paper, that gets split into 1MB parts, so 1000. And they probably use erasure-coding for replication. How does it work ? Do they EC each 1000 part separately, or ?
  - jeffbee 6y ago
    According to these slides, each stripe is a separate replication group. Although the figure depicts replicated rather than erasure coded files, I think it's safe to imagine that the idea holds for both. http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Keynote.pdf http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke...

by 2sk21 6y ago

ChuckMcM 6y ago

I kind of gathered that IBM had this sort of love/hate relationship with acquisitions. They loved the "new ideas" but many of the "old guard" hated the idea that something from outside of IBM was novel and worth pursuing. I watched first hand as things that had been done by a company that was acquired were buried while the exact same kind of thing done by IBM research was given lots of funding. I understood not having two efforts, but suggested they be blended to get the best of both teams, alas that was shot down. At one time IBM research had a pretty awesome storage group, perhaps they will will have a computational storage fabric offering at some point.

2sk21 6y ago
I know exactly the IBM Research group you are referring to but won't name names to avoid embarrassment :-) Politics in large organizations can be pretty nasty.

jeffbee 6y ago

The way HBase/HDFS does this is perfectly backwards and counterproductive. You don't want to put a program and its data on one local disk, because that one local disk has almost no resources. The later-period thinking at Google is the opposite of shipping the program to the data. Instead, you allocate the data randomly over as many spindles as you can find, then when you read it all back out you very briefly use all of those spindles at once. Really, it's something to behold in action. That's batch processing: mapreduce and flume and whatnot. Search is still very much an exercise in getting the queries out to where the index shards live.

Xorlev 6y ago
Did you mean Hadoop/HDFS? HBase works pretty analogously to Bigtable (though, having run HBase in production at $dayjob-1, I'd take the latter any day).
- jeffbee 6y ago
  HBase doesn't work analogously to bigtable. HBase intentionally tries to move region files onto the same node as the region server. That is ass-backwards. HBase devs think of this as "good locality" but all you're really getting from it is terrible performance.
ithkuil 6y ago
The secret sauce for that is networking. Once you have a domain with full clos connectivity and very low latencies you can start disaggregating storage from compute again
ddorian43 6y ago
Assuming you have a 1GB file. According to Colossus paper, that gets split into 1MB parts, so 1000. And they probably use erasure-coding for replication. How does it work ? Do they EC each 1000 part separately, or ?
- jeffbee 6y ago
  According to these slides, each stripe is a separate replication group. Although the figure depicts replicated rather than erasure coded files, I think it's safe to imagine that the idea holds for both. http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Keynote.pdf http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke...