3 ms·

The original maths is correct. You have neglected the fact that the 4 failed drives can be any of the 20. Your calculations are for 4 specific drives failing.

mrb 8y ago
Oh you are right. The OP still has a slight error though. C(20,4) = 4845 possible combinations of failing drives (4 out of 20.) Therefore the chance a file is irrecoverable is: .97^16 × .03^4 × 4845 = 0.241% (not 0.3%) But the OP's conclusion is largely correct: every customer will have some irrecoverable files.
- majewsky 8y ago
  I think you're not considering that failed drives will be replaced and the data on them reconstructed from the other shards. This failure mode requires 4 of 20 drives to fail in such a short amount of time that reconstruction cannot be completed.
  - mrb 8y ago
    Yes, but this was the OP's scenario: what happens when 3% of drives all fail at the same time when the DC is powered back on. Edit: actually the math is still wrong. The chance any 4 out of 20 drive is failing is: .03^4 × C(20,4) = .03^4 × 4845 = 0.392% — There is no need to multiply by .97^16 as the status of the other 16 drives is irrelevant.
    - mrb 8y ago
      Decidedly, statistics is hard. Everything above is wrong. Let's label the twenty drives D0 through D19. There are 2^20 possible scenarios, which can be represented as a string of 20 bits: • 00000000000000000000 = all 20 drives are working • 00000000000000000001 = D19-D1 working, D0 failing • 00000000000000000010 = D19-D2 working, D1 failing, D0 working • 00000000000000000011 = D19-D2 working, D1-D0 failing • etc The probability of each of these scenarios is: • 00000000000000000000: .97^20 • 00000000000000000001: .97^19 × .03 • 00000000000000000010: .97^19 × .03 • 00000000000000000011: .97^18 × .03^2 • etc There are C(20,4) = 4845 scenarios with exactly four failing drives (four "1" bits.) The probability of each scenario is .97^16 × .03^4. Therefore the probability of 4 failing drives (any drive) is the sum of the probability of each scenario: .97^16 × .03^4 × C(20,4) like I said 3 comments above. However the probability of a file being irrecoverable is P(4 failing drives) + P(5 failing drives) + ... + P(20 failing drives): .97^16 × .03^4 × C(20,4) + .97^15 × .03^5 × C(20,5) + ... + .97^0 × .03^20 × C(20,20) = 0.267%

mrb 8y ago

Oh you are right. The OP still has a slight error though. C(20,4) = 4845 possible combinations of failing drives (4 out of 20.) Therefore the chance a file is irrecoverable is: .97^16 × .03^4 × 4845 = 0.241% (not 0.3%) But the OP's conclusion is largely correct: every customer will have some irrecoverable files.

majewsky 8y ago
I think you're not considering that failed drives will be replaced and the data on them reconstructed from the other shards. This failure mode requires 4 of 20 drives to fail in such a short amount of time that reconstruction cannot be completed.
- mrb 8y ago
  Yes, but this was the OP's scenario: what happens when 3% of drives all fail at the same time when the DC is powered back on. Edit: actually the math is still wrong. The chance any 4 out of 20 drive is failing is: .03^4 × C(20,4) = .03^4 × 4845 = 0.392% — There is no need to multiply by .97^16 as the status of the other 16 drives is irrelevant.
  - mrb 8y ago
    Decidedly, statistics is hard. Everything above is wrong. Let's label the twenty drives D0 through D19. There are 2^20 possible scenarios, which can be represented as a string of 20 bits: • 00000000000000000000 = all 20 drives are working • 00000000000000000001 = D19-D1 working, D0 failing • 00000000000000000010 = D19-D2 working, D1 failing, D0 working • 00000000000000000011 = D19-D2 working, D1-D0 failing • etc The probability of each of these scenarios is: • 00000000000000000000: .97^20 • 00000000000000000001: .97^19 × .03 • 00000000000000000010: .97^19 × .03 • 00000000000000000011: .97^18 × .03^2 • etc There are C(20,4) = 4845 scenarios with exactly four failing drives (four "1" bits.) The probability of each scenario is .97^16 × .03^4. Therefore the probability of 4 failing drives (any drive) is the sum of the probability of each scenario: .97^16 × .03^4 × C(20,4) like I said 3 comments above. However the probability of a file being irrecoverable is P(4 failing drives) + P(5 failing drives) + ... + P(20 failing drives): .97^16 × .03^4 × C(20,4) + .97^15 × .03^5 × C(20,5) + ... + .97^0 × .03^20 × C(20,20) = 0.267%