9 ms·

Great answer. Thank you. I considered trying to answer myself, but (a) I was too lazy and (b) it wouldn't have been as good as your answer anyway. To make amend

Great answer. Thank you. I considered trying to answer myself, but (a) I was too lazy and (b) it wouldn't have been as good as your answer anyway. To make amends, here are two additional points about the relationship between the two. (1) One of the statements in the CRUSH paper introduction is flat-out false. Specifically, it says that "RUSH [a predecessor] remains the only existing set of algorithms in the literature that utilizes a mapping function in place of explicit metadata" etc. By 2006, other approaches - including Kademlia, but more commonly circular consistent hashing - were extremely well known. Just as Ceph itself grew from this paper, so did the tendency to make exaggerated claims on its behalf (e.g. "it's a filesystem" years before that was actually true). (2) One of the key problems with any "deterministic placement" approach like CRUSH, which relies on clients later duplicating the process by which data had been placed, is what to do when the map changes. Section 3.3 addresses this in terms of how much data has to move, but that process is hardly instantaneous. What to do in the interim? And how does the new map get propagated? These issues are not addressed in the paper, which is fine given its mathematical focus, but anyone who hopes to build an actual system based on an algorithm like this needs to think through the answers, and those answers have a large effect on how well the system really works.

pritambaral 6y ago
> One of the statements in the CRUSH paper introduction is flat-out false. Specifically, it says that "RUSH [a predecessor] remains the only existing set of algorithms in the literature that utilizes a mapping function in place of explicit metadata" etc. Looking at the rest of the sentence: " ... that utilizes a mapping function in place of explicit metadata and supports the efficient addition and removal of weighted devices.", I feel like calling the statement is "flat-out false" is incorrect.
- notacoward 6y ago
  Those other well known methods also support efficient addition and removal. Supporting weighted devices is a tweak that should be obvious to any competent engineer, which I know because I've done it myself in a similar system. The paper might reasonably claim their way is better, but to represent it as a qualitative change granting never-before-seen capabilities is to misrepresent the state of the art at the time. The only comparison where that works is to Lustre's "MDS controls all" approach, which most people working in the field at the time (but apparently not the buffoons working on HDFS) already knew was bad.

pritambaral 6y ago

> One of the statements in the CRUSH paper introduction is flat-out false. Specifically, it says that "RUSH [a predecessor] remains the only existing set of algorithms in the literature that utilizes a mapping function in place of explicit metadata" etc. Looking at the rest of the sentence: " ... that utilizes a mapping function in place of explicit metadata and supports the efficient addition and removal of weighted devices.", I feel like calling the statement is "flat-out false" is incorrect.

notacoward 6y ago
Those other well known methods also support efficient addition and removal. Supporting weighted devices is a tweak that should be obvious to any competent engineer, which I know because I've done it myself in a similar system. The paper might reasonably claim their way is better, but to represent it as a qualitative change granting never-before-seen capabilities is to misrepresent the state of the art at the time. The only comparison where that works is to Lustre's "MDS controls all" approach, which most people working in the field at the time (but apparently not the buffoons working on HDFS) already knew was bad.