3 ms·
Tangential: why does MapReduce use /dev/random as its entropy source? "After a long and tricky debugging process, we found that a big MapReduce job was firing
by throwaway_yy2Di 13y ago
Tangential: why does MapReduce use /dev/random as its entropy source?
"After a long and tricky debugging process, we found that a big MapReduce job was firing up every few hours and, as a part of its normal functioning, it was reading from /dev/random. When too many of the MapReduce workers landed on a machine, they were able read enough to deplete the randomness available on the entire machine. It was on these machines that our serving binaries were becoming unresponsive: they were blocking on reads of /dev/random!"
http://www.reddit.com/r/IAmA/comments/1w1y5m/we_are_the_google_site_reliability_engineering/cexz5yy http://www.reddit.com/r/IAmA/comments/1w1y5m/we_are_the_goog...
- packetslave 13y agoIt sounds like it wasn't MapReduce itself, but rather the specific MR user job that was being run.