4 ms·
That is not difficult to handle in bash. The expectation going in should be that individual computations and even computers might fail in ways that keep them fr
by saurik 16y ago
That is not difficult to handle in bash. The expectation going in should be that individual computations and even computers might fail in ways that keep them from reporting failure, at which point the fact that it is written in a language running on a VM with a poor memory manager (as it really wasn't designed for this kind of stress, and rapidly dominates RAM and fragments its heap to hell) is not a problem. (This happens to be a sore point of mine, as I had a rather unenlightened professor in grad school who refused to believe I could effectively solve this one massive distributed problem that came up with a "simple shell script for doing the distribution", giving me a bad grade in his class and never forgiving me for having made the claim. This being despite the fact that I then sent him the finished result three days later, having narrowed the search space with some light intelligence enough to have gotten the same answer his system took a week to do on better hardware, and his beloved not-a-shell-script. sigh ;P)
- gfodor 16y agoIt seems easier said than done, unless you think Hadoop's architecture and fault tolerance logic is over-engineered.