3 ms·
Sector/Sphere is what the Sloan Digital Sky Survey uses. Instead of supplying map and reduce routines, you implement generic "user defined functions". This gi
by bunderbunder 14y ago
Sector/Sphere is what the Sloan Digital Sky Survey uses.
Instead of supplying map and reduce routines, you implement generic "user defined functions". This gives you some more flexibility about how the work is handled, though if you want to just implement map and reduce UDFs, it supposedly gets better performance than Hadoop.
It's also designed to support distributing work over WANs. I think Hadoop really wants every compute node to be on the same LAN.
- heretohelp 14y ago>I think Hadoop really wants every compute node to be on the same LAN. Fucking a-right it does. You should see the labyrinthine depths people descend to in order to scale Hadoop. Sub-clusters of sub-clusters, rack-local clusters, Zookeeper nodes all over the place. It's like fuckin' 'Nam all over again man.