3 ms·
I did a rewrite. The old code worked, but was slow. Adding features would make it slower. Lock-free queues and threads everywhere, packet buffers bouncing from
by ncmncm 7y ago
I did a rewrite.
The old code worked, but was slow. Adding features would make it slower. Lock-free queues and threads everywhere, packet buffers bouncing from input queues to delivery queues to free queues to free lists, threads manfully shuttling them around, with a bit of actual work done at one stage.
Replaced it all with one big-ass ring buffer and one writer process per NIC interface. Readers in separate processes map and watch the ring buffer, and can be killed and started anytime. Packets are all processed in place, not copied, not freed, just overwritten in due time.
It took a few months. Now a single 2U server and a disk array captures all New York and Chicago market activity (commodity futures excepted).
I kept the part that did the little work, scrapped the rest.
C++, mmap, hugepages FTW.