4 ms·
Wow. Good work Storm people! Some nice features here. Anyone privy to what changes they made to yield the performance improvements?
by therealdrag0 11y ago
Wow. Good work Storm people! Some nice features here.
Anyone privy to what changes they made to yield the performance improvements?
- lovelearning 11y agoFrom the issue log : "Profiling a simple speed of light topology, shows that a good chunk of time of the SpoutOutputCollector.emit() is spent in the clojure reduce() function.. which is part of the ACK-ing logic. Re-implementing this reduce() logic in java gives a big performance boost in both in the Spout.nextTuple() and Bolt.execute()" [1]: https://issues.apache.org/jira/browse/STORM-1539 https://issues.apache.org/jira/browse/STORM-1539
- d3fun 11y agoMore coming in the roadmap. We are also working performance improvements around internal queuing.
- kabhwan 11y agoApache Storm 1.0.0 introduces batching internal disruptor queue which adds latencies slightly (but with timeout set to 1ms per each queue so it doesn't hurt) but gives huge throughput improvement. https://github.com/apache/storm/pull/765 https://github.com/apache/storm/pull/765