4 ms·

Thanks for this. I used this multiproducer multiconsumer ringbuffer https://www.linuxjournal.com/content/lock-free-multi-producer-multi-consumer-queue-ring-bu

Thanks for this. I used this multiproducer multiconsumer ringbuffer https://www.linuxjournal.com/content/lock-free-multi-producer-multi-consumer-queue-ring-buffer https://www.linuxjournal.com/content/lock-free-multi-produce... I am inspired by LMAX Disruptor. I like the idea of splitting all IO into two halves: submit and reply. At one point I was looking slides of some company who used LMAX Disruptor pattern for their IO and had high performance. My extremely naive Java benchmark to test locks I get 35,871,639 locks per second on 12 threads with contention. I am trying to work out how to improve the performance of the multiproducer multiconsumer ringbuffer which gets 10 million requests per second it should be higher. https://github.com/samsquire/multiversion-concurrency-control https://github.com/samsquire/multiversion-concurrency-contro... I want to create a message queue that uses ringbuffers to communicate between threads efficiently and elegantly and handles the splitting of IO in half. I want to combine coroutines and threads together.

bob1029 3y ago
LMAX Disruptor was very inspirational to me as well. I've found a lot of problem spaces it fits into perfectly. For example, any sort of MPSC arrangement. Literally any. The most obvious being if you want to DIY your own database. If performance is of interest to you, you may want to check out the latest variants of Disruptor on .NET. Figures nearing 500 million per second range are present here, and will likely be exceeded in future versions of .NET: https://medium.com/@ocoanet/improving-net-disruptor-performance-part-3-introducing-the-valuedisruptor-5b467730bbe https://medium.com/@ocoanet/improving-net-disruptor-performa... The thing .NET has to its advantage (not sure if this is true anymore) is how structs vs classes work in memory. This is why there is a speedup with ValueDisruptor<T>. > how to improve the performance of the multiproducer multiconsumer > multiconsumer Fundamentally, I think multi-consumer is going to chop a whole order of magnitude off no matter how you slice it. Having just one consumer (aka a 'writer') is a foundational aspect of the performance games played in this arena.
- dist1ll 3y ago
  If you're interested in queues, you might want to check out the USENIX BBQ paper from last year: https://www.usenix.org/conference/atc22/presentation/wang-jiawei https://www.usenix.org/conference/atc22/presentation/wang-ji... They've gone with a block-based approach for SPSC/MPSC/MPMC that blows the linux implementation, DPDK and the LMAX Disruptor out of the water.
  - bob1029 3y ago
    This is very interesting... I've never thought to divide the buffer into multiple blocks. Are they requiring some low-level OS abstraction to pull this off or are we looking at a novel technique at a higher level (i.e. could I patch an existing implementation)?
- samsquire 3y ago
  Thanks for your reply and thanks for your insight and wisdom. 500 million requests per second. I think that's awesome for _intra thread communication_. I have a sharded bank simulation which naively sends money between accounts and it gets 700 million transactions a second which is the aggregate of 12 threads all transacting local to a thread, so 500 million across threads seems good (I split money up rather than accounts per thread). I see that some web frameworks can handle up to 600,000 requests per second on techempower benchmarks. So we need to combine the best of each! Especially the part about one writer. I have a journal entry called "Golden concurrency" inspired by someone's comment on HN (I really should have linked it!) talking about writing to a database from 3 processes and they decided to split the database into 3 databases rather than suffer contention from concurrency control. "Golden concurrency" is my term or desire that degradation caused by multiple producers/multiple consumers in different configurations could be optimised for and could be a scenario that is handled without performance degradation. I suspect you need a different design if you have 1 producer:1 consumer producers to consumers, or 1 producer:N consumers or N producers:1 consumer or N producers:N consumers. I enjoyed the following benchmarks with Kafka and RedPanda which made me think of this too. https://jack-vanlightly.com/blog/2023/5/15/kafka-vs-redpanda-performance-do-the-claims-add-up https://jack-vanlightly.com/blog/2023/5/15/kafka-vs-redpanda...
  - agallego 3y ago
    That benchmark was comparing apples to oranges. Redpanda fsynced to disk and kafka saved to memory with deferred writes. Here is a response https://redpanda.com/blog/why-fsync-is-needed-for-data-safety-in-kafka-or-non-byzantine-protocols https://redpanda.com/blog/why-fsync-is-needed-for-data-safet...

bob1029 3y ago

LMAX Disruptor was very inspirational to me as well. I've found a lot of problem spaces it fits into perfectly. For example, any sort of MPSC arrangement. Literally any. The most obvious being if you want to DIY your own database. If performance is of interest to you, you may want to check out the latest variants of Disruptor on .NET. Figures nearing 500 million per second range are present here, and will likely be exceeded in future versions of .NET: https://medium.com/@ocoanet/improving-net-disruptor-performance-part-3-introducing-the-valuedisruptor-5b467730bbe https://medium.com/@ocoanet/improving-net-disruptor-performa... The thing .NET has to its advantage (not sure if this is true anymore) is how structs vs classes work in memory. This is why there is a speedup with ValueDisruptor<T>. > how to improve the performance of the multiproducer multiconsumer > multiconsumer Fundamentally, I think multi-consumer is going to chop a whole order of magnitude off no matter how you slice it. Having just one consumer (aka a 'writer') is a foundational aspect of the performance games played in this arena.

dist1ll 3y ago
If you're interested in queues, you might want to check out the USENIX BBQ paper from last year: https://www.usenix.org/conference/atc22/presentation/wang-jiawei https://www.usenix.org/conference/atc22/presentation/wang-ji... They've gone with a block-based approach for SPSC/MPSC/MPMC that blows the linux implementation, DPDK and the LMAX Disruptor out of the water.
- bob1029 3y ago
  This is very interesting... I've never thought to divide the buffer into multiple blocks. Are they requiring some low-level OS abstraction to pull this off or are we looking at a novel technique at a higher level (i.e. could I patch an existing implementation)?
samsquire 3y ago
Thanks for your reply and thanks for your insight and wisdom. 500 million requests per second. I think that's awesome for _intra thread communication_. I have a sharded bank simulation which naively sends money between accounts and it gets 700 million transactions a second which is the aggregate of 12 threads all transacting local to a thread, so 500 million across threads seems good (I split money up rather than accounts per thread). I see that some web frameworks can handle up to 600,000 requests per second on techempower benchmarks. So we need to combine the best of each! Especially the part about one writer. I have a journal entry called "Golden concurrency" inspired by someone's comment on HN (I really should have linked it!) talking about writing to a database from 3 processes and they decided to split the database into 3 databases rather than suffer contention from concurrency control. "Golden concurrency" is my term or desire that degradation caused by multiple producers/multiple consumers in different configurations could be optimised for and could be a scenario that is handled without performance degradation. I suspect you need a different design if you have 1 producer:1 consumer producers to consumers, or 1 producer:N consumers or N producers:1 consumer or N producers:N consumers. I enjoyed the following benchmarks with Kafka and RedPanda which made me think of this too. https://jack-vanlightly.com/blog/2023/5/15/kafka-vs-redpanda-performance-do-the-claims-add-up https://jack-vanlightly.com/blog/2023/5/15/kafka-vs-redpanda...
- agallego 3y ago
  That benchmark was comparing apples to oranges. Redpanda fsynced to disk and kafka saved to memory with deferred writes. Here is a response https://redpanda.com/blog/why-fsync-is-needed-for-data-safety-in-kafka-or-non-byzantine-protocols https://redpanda.com/blog/why-fsync-is-needed-for-data-safet...