4 ms·
Mincemeat.py actually does do fault tolerance and preemptive duplication. I'm trying to get it to handle all of the MapReduce logic (and reliability, security,
by michaelfairley 16y ago
Mincemeat.py actually does do fault tolerance and preemptive duplication. I'm trying to get it to handle all of the MapReduce logic (and reliability, security, etc.) without the extra burdens of setting up a distributed file system (the machines I've been running it on have a hefty NFS share, which is good enough for most purposes), preallocation of the machines that will be part of the cluster (my university uses Condor, which spawns processes randomly across a large cluster), etc.
- mayank 16y agoIn that case, kudos! One of my pet gripes about Hadoop is that it really used to be unusable without HDFS (don't know if that's changed). If you truly manage to develop it to a stable release with all the features of Hadoop, I definitely see it making some inroads in scientific computation.