4 ms·
I would look at adding LMAX Disruptor to this list. It can run circles around stuff scattered across TPL usages. Doesnt fit every use case, but its really incre
by bob1029 6y ago
I would look at adding LMAX Disruptor to this list. It can run circles around stuff scattered across TPL usages. Doesnt fit every use case, but its really incredible when it does fit. I was able to build a toy project that handles millions of user events per second on a single thread using this.
Getting your application aligned with the NUMA model makes way more difference in performance than anything else.
https://github.com/disruptor-net/Disruptor-net https://github.com/disruptor-net/Disruptor-net
- fnord123 6y ago> Getting your application aligned with the NUMA model makes way more difference in performance than anything else. Choose a better data format (binary format rather than json). Compress data. Remove repeated work (lift out of loops). Get all the data from the DB up front. Index. Cache. Parallelise. cpusets (aligned with the NUMA model)
- TeMPOraL 6y ago> Remove repeated work (lift out of loops). > Get all the data from the DB up front. Between those two, I'd add: Move computation to the data where appropriate. Let the data layer (particularly the database itself) do the heavy lifting.
- UK-Al05 6y agoConsidering the databases often end up being the bottleneck. It may not always be wise. So its a small amount of data, but large computation it might be tempted to do it on the application. But its a large amount of data, I/O maybe a problem in doing it on the app.
- TeMPOraL 6y agoThat's why I hedged my bets with "where appropriate" there :). You're of course right, and it could be that DB being a compute bottleneck is the most common case - but I've seen or been involved in projects where the bottleneck was DB I/O and processing unnecessary queries, because the application was doing most of the calculations.
- fnord123 6y agoI would also add 'use specialised hardware (simd, gpgpu)'
- boyter 6y agoDo you or anyone else know how this compares in performance to the Java version it's based on?
- bob1029 6y agohttps://medium.com/@ocoanet/improving-net-disruptor-performance-part-3-introducing-the-valuedisruptor-5b467730bbe https://medium.com/@ocoanet/improving-net-disruptor-performa... In this article, with some optimizations, the author was able to achieve on the order of ~70m operations per second using the .NET port. I think .NET might actually be faster than Java in some places here.
- coldtea 6y ago>I would look at adding LMAX Disruptor to this list. Isn't that a Java thing? So, this one is a port to .NET
- bob1029 6y agoCorrect - this was originally a Java library. The above link is for the .NET port.