4 ms·
By the 'end to end principle', redundancy should probably be concentrated somewhere in the stack, and the rest of the stack should be concerned merely with vali
by gwern 7y ago
By the 'end to end principle', redundancy should probably be concentrated somewhere in the stack, and the rest of the stack should be concerned merely with validating integrity. It's unlikely that the optimum balance of resources and loss probability will entail redundancy at every level of the stack, from raw HDD bytes up to the global system level.
- mixmastamyk 7y agoImagine files being moved from tape to disk to optical disc to NAS over the years. What now? https://news.ycombinator.com/item?id=16886607 https://news.ycombinator.com/item?id=16886607
- gwern 7y agoYou move them, then you verify the integrity of the new copy, then you can get rid of the old one. You don't need to build integrity checks and extra FEC at every level of the system redundantly. Just like in the end-to-end principle when applied to networking: you have a single strong integrity check at the very furthest endpoint possible, and then you don't build in integrity & ECC at every level of the stack, you devote those resources to higher performance, and just do retransmission from the other endpoint when a file occasionally gets corrupted and the integrity check catches it.
- mixmastamyk 7y agoAppealing to "more manual work" is not very compelling to be honest.
- gwern 7y ago-_- I never appealed to 'more manual work', nor did I say that. If you refuse to understand my point and want to make up things I did not say, so be it.
- mixmastamyk 7y ago> you verify the integrity
- gwern 7y agoDon't be a moron. You knew perfectly well what I meant. (Did I also mean that 'you', a human, should be checking hashsums and FEC by hand for every network packet...?)