3 ms·
DDIO is a bit of a pet peeve of mine. DDIO applies to ANY I/O device, which makes it a horrible idea on servers with even a moderate amount of I/O. With DDIO
by drewg123 5y ago
DDIO is a bit of a pet peeve of mine.
DDIO applies to ANY I/O device, which makes it a horrible idea on servers with even a moderate amount of I/O. With DDIO, EVERY I/O device's DMA writes (eg, transfer from device to host) allocates space in cache. This means it works great on toy benchmarks with a single thread polling for I/O from a NIC. But scale it up to a 32c/64t server that is reading a few gigabytes/sec from disk and doing few tens of gigabits of network I/O, and you thrash your caches.
Its precursor, DCA, was limited to Intel (and a few other vendors) NICs, and was far better. With DCA, the NIC decides what, if any, DMA write should be pushed to cache and tags that DMA write with a PCIe TLP steering tag. The best choices are typically RX descriptors, and protocol headers. Other data is not tagged, and hence does not pollute the cache.
- jeffbee 5y agoIt sounds like what you want is RDT+CAT, so you can allocate your LLC as you like. And that means you have to pick and choose your platform since Intel keeps waffling over whether CAT should exist on every SKU or not.
- drewg123 5y agoWell, yes and no. I don't see how CAT can differentiate between a packet header and packet data, but being able to control the number of ways dedicated to DDIO would be nice. In terms of features like CAT being available on different SKUs.. Intel needs to remember that they're not just competing with themselves these days. Their competitors generally don't go fusing off features on low-end SKUs to drive sales of higher margin SKUs. Intel needs to differentiate chips solely on on cores count, clock speed, cache size, etc.
- jeffbee 5y agoCAT, at least, has trickled down to desktop-class SKUs. The real problem is you can't use it in EC2.