4 ms·
The Twitter Engineering Blog post is way more interesting http://engineering.twitter.com/2011/08/storm-is-coming-more-details-and-plans.html http://engineering.
by jackowayed 15y ago
The Twitter Engineering Blog post is way more interesting http://engineering.twitter.com/2011/08/storm-is-coming-more-details-and-plans.html http://engineering.twitter.com/2011/08/storm-is-coming-more-...
Despite having a "master" node, it sounds like this actually has no single points of failure. Since all the state for the master is in ZooKeeper, I think you could just fail over to another server running the master if your first main one gets messed up. Pretty cool. (I may be totally wrong here ... All my distributed systems knowledge has come from being around people who know about distributed systems.)
Though one thing that's not especially satisfying is that if his answer for when a Bolt needs to store state is "use a database". I guess the Hadoop answer is that if your reducer fails, it just runs it again, and there's no real analog to doing that when your bolts are meant to run infinitely
- nathanmarz 15y agoThat's correct, the design will make it easy to cluster the master node later on.