4 ms·
Latency. Latency. Latency! It's hard to measure, so nobody does. Throughput is easy to measure, so everybody does. Latency is hard to buy, so few people try.
by jiggawatts 5y ago
Latency. Latency. Latency!
It's hard to measure, so nobody does.
Throughput is easy to measure, so everybody does.
Latency is hard to buy, so few people try.
Throughput is easy to buy, so everybody does.
Latency is what matters to every user.
Throughput matters only to a few people.
Turn on SR-IOV. Disable ACPI C-states. Stop tunnelling internal traffic through virtual firewalls. Use binary protocols instead of JSON over HTTPS.
I've seen just those alone improve end-user experience tenfold.
- nomel 5y ago> Disable ACPI C-states. Related, I worked on an embedded linux device that bit-banged an unusual communication protocol. I wanted to offload this bit banging to some dedicated microcontrollers built into the SOC, to help free up the main CPU for some other use. I spent some time writing a nice little device driver that used DMA to send the transaction to the bit banger. The main CPU usage went from 60% down to 2%! Success! Unfortunately, the transactions/second was now a fraction of what it was before. After some digging, it turned out the improved CPU usage was low enough that the CPU scaling governor was staying at the lowest frequency. Disabling the governor and forcing the CPU to max frequency resulted in a pretty huge transactions/second improvements. Obvious in retrospect, of course.