4 ms·

These numbers are great, but they are also highly dependant on hardware architecture and a strong coupling with network architecture. Another big contributor t

These numbers are great, but they are also highly dependant on hardware architecture and a strong coupling with network architecture. Another big contributor to messaging is in the message size. One can claim 10M messages/sec but for 1 byte messages, 10m messages/s is hardly impressive. I can say that on today's hardware, achieving 3-7M msgs/sec in a single-stream, guaranteed, in-order messaging is possible. This us also with 120 byte messages, which amounts to about 3-7Gbps if sustained messaging in clusters of 2 to 200+ machines. What do you need to achieve this? * Kernel bypass network architecture e.g. Solarflare 10/40gbe * 10gbe switches * decoupling of transport layer into a dedicated processor cores * busy-polling processes * business logic placed on dedicated cores. * use of shared memory and/or memory mapping to transport messaging between processes. It all sounds fairly convoluted to throw up processes and burn a core on rx/tx, but if you need to process millions of messages a second at fairly low latency, this is about the limit of where it goes. Using traditional application and socket tied to a single process will get you to around 1 million messages/sec with some very large buffers and efficient business logic, but thst eill be about it. The next step would be to start sharding streams, but that comes with its own complexities.

zhenjl 12y ago
Author here. The message size tested is 1024 bytes.
- eternalban 12y ago
  Are you cpu bound or (net) io bound?
  - zhenjl 12y ago
    When I ran this I gave the server and client 2 cores each. Neither were maxing out the cores. At about 856K MPS (sent + received) @ 1024 bytes/msg, that's less than 1 GBps. So I can't imagine it being io(net) bound either. So inconclusive...unfortunately. Need to profile a bit more to know. Edit: Tho this is TCP based so it's possible that it's netio bound given the protocol overhead. At this message rate, each packet is almost guaranteed to be full size. So we are doing about 600K pps, approximately.
    - itchyouch 12y ago
      Are you passing messages on the same host? When doing message passing between processes on the same machine, memory mapping and shared memory will be your friends. Check this architecture out for intra-host message passing: https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/ If you are doing tcp over loopback, the MTU is 64K, IIRC. You should be able to pack more messages per packet. Since you aren't CPU bound, you are probably memory bound. Also if you on a non-kernelbypass accelerated host, wire-to-application memory takes about 10 micros, so wire-to-memory-to-wire will probably land somewhere around 20 microseconds.
      - zhenjl 12y ago
        lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 Mac OS Yosemite..
    - eternalban 12y ago
      > When I ran this I gave the server and client 2 cores each. Neither were maxing out the cores. grep for GOMAXPROCS didn't turn up anything so not sure what you mean by that. This may help with core utilization. func init() { // runtime.GOMAXPROCS(runtime.NumCPU()) // … }
      - zhenjl 12y ago
        I ran with GOMAXPROCS=2 on the command line. The post showed the actual commands that I used in each case.
        eternalban 12y ago
        Ah, missed that. In any event, that sets the number of Go Processors per Go runtime. It has no bearing on the number of "cores" utilized.
jandrewrogers 12y ago
I think you are overstating the difficulty of achieving good network throughput on modern hardware. Architecture definitely matters but saturating 10GbE is pretty achievable without going to extraordinary lengths. While doing kernel bypass for networking is more efficient (and preferable for other reasons), it is not necessary for pushing a huge number of messages in my experience. The biggest bottleneck tends to be the computational cost of parsing the protocol format. Nonetheless, I've seen GeoJSON documents moved over networks at per node messaging rates similar to those in the article.
- itchyouch 12y ago
  In my experience, just passing messages around w/ dummy logic should be yielding numbers about 10x greater at around 10x the reduced latency. With architectural tuning, the messages/sec in the various scenarios can be increased 5x (~4M msgs/s) while decreasing the average latency of processing a message to the theoretical minimums of around 5-6 microseconds to get from wire to application memory and back out to wire. Even with efficient protocol formats (mold64), one reaches ceilings too soon. The fact that the author mentions that they are neither CPU bound or network IO bound leads me to believe that they are probably bound by memory IO and lost efficiencies to context switching.

zhenjl 12y ago

Author here. The message size tested is 1024 bytes.

eternalban 12y ago
Are you cpu bound or (net) io bound?
- zhenjl 12y ago
  When I ran this I gave the server and client 2 cores each. Neither were maxing out the cores. At about 856K MPS (sent + received) @ 1024 bytes/msg, that's less than 1 GBps. So I can't imagine it being io(net) bound either. So inconclusive...unfortunately. Need to profile a bit more to know. Edit: Tho this is TCP based so it's possible that it's netio bound given the protocol overhead. At this message rate, each packet is almost guaranteed to be full size. So we are doing about 600K pps, approximately.
  - itchyouch 12y ago
    Are you passing messages on the same host? When doing message passing between processes on the same machine, memory mapping and shared memory will be your friends. Check this architecture out for intra-host message passing: https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/ If you are doing tcp over loopback, the MTU is 64K, IIRC. You should be able to pack more messages per packet. Since you aren't CPU bound, you are probably memory bound. Also if you on a non-kernelbypass accelerated host, wire-to-application memory takes about 10 micros, so wire-to-memory-to-wire will probably land somewhere around 20 microseconds.
    - zhenjl 12y ago
      lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 Mac OS Yosemite..
  - eternalban 12y ago
    > When I ran this I gave the server and client 2 cores each. Neither were maxing out the cores. grep for GOMAXPROCS didn't turn up anything so not sure what you mean by that. This may help with core utilization. func init() { // runtime.GOMAXPROCS(runtime.NumCPU()) // … }
    - zhenjl 12y ago
      I ran with GOMAXPROCS=2 on the command line. The post showed the actual commands that I used in each case.
      - eternalban 12y ago
        Ah, missed that. In any event, that sets the number of Go Processors per Go runtime. It has no bearing on the number of "cores" utilized.

jandrewrogers 12y ago

I think you are overstating the difficulty of achieving good network throughput on modern hardware. Architecture definitely matters but saturating 10GbE is pretty achievable without going to extraordinary lengths. While doing kernel bypass for networking is more efficient (and preferable for other reasons), it is not necessary for pushing a huge number of messages in my experience. The biggest bottleneck tends to be the computational cost of parsing the protocol format. Nonetheless, I've seen GeoJSON documents moved over networks at per node messaging rates similar to those in the article.

itchyouch 12y ago
In my experience, just passing messages around w/ dummy logic should be yielding numbers about 10x greater at around 10x the reduced latency. With architectural tuning, the messages/sec in the various scenarios can be increased 5x (~4M msgs/s) while decreasing the average latency of processing a message to the theoretical minimums of around 5-6 microseconds to get from wire to application memory and back out to wire. Even with efficient protocol formats (mold64), one reaches ceilings too soon. The fact that the author mentions that they are neither CPU bound or network IO bound leads me to believe that they are probably bound by memory IO and lost efficiencies to context switching.