3 ms·
They have dropped clojure completely. All the critical path messaging in Storm was done using clojure. Dropped netty too. The actual messaging ("stream manager"
by lovelearning 10y ago
They have dropped clojure completely. All the critical path messaging in Storm was done using clojure. Dropped netty too. The actual messaging ("stream manager") seems to be C++. Perhaps that explains the latency and CPU improvements. The architectural changes mentioned account for better cluster utilization, fault tolerance and back pressure implementation, but don't explain why raw streaming performance is so much better in Heron.
Edit: Not so sure what they are using for networking. Seemed like cpp-netlib at first, but I don't think so now.
- JonathonW 10y agoLooks like a homegrown networking framework: https://github.com/twitter/heron/tree/master/heron/common/src/cpp/network https://github.com/twitter/heron/tree/master/heron/common/sr... Their default event loop implementation uses libevent, and they're using protobuf in some of their higher-level networking classes, but the networking code itself seems to pretty much be plain sockets (with a thin portability layer on top in a few places).