3 ms·

Probably fairly similar to Hadoop. From what I've learned thus far, dealing with outages pretty much boils down to replication. http://sector.sourceforge.net/te

• jacquesm 17y ago
  > Sector provides automatic failover for Map style data processing. When a slave fails during the processing of a data segment, another slave will be assigned for the same data segment to continue processing. But that does not say whether the 'read' or 'write' that failed will be restarted or whether it starts the whole segment from scratch. Which is what's interesting in Hadoop, it apparently (I haven't tried that) keeps chugging along is if literally nothing happened, so from the applications point of view the fault never occurred.
  • lzimm 17y ago
    Hm. I see your point. I guess what it boils down to is how frequently the nodes report their status back to the master (or whatever its called) so that it can determine which operations have succeeded and which are still pending/failed. I wonder how much of a tax it takes to report the result of each operation sequentially vs. doing it in a batch at the end of the segment. If the difference in performance is because Hadoop reports sequential while sector does it batched, be nice to have something that gives you a sliding bar to chose somewhere along the gradient between failover optimization and optimistic optimization where your failovers are far and few in between.