4 ms·
We've observed FDP to make a surprisingly big difference in drive internal WA. If you can meaningfully tag different lifetime/stream data from your workloads,
by loeg 1y ago
We've observed FDP to make a surprisingly big difference in drive internal WA. If you can meaningfully tag different lifetime/stream data from your workloads, and you can expect hardware that supports it, it's very helpful. We some something like WAF reduction from ~1.60 to ~1.04 (on a synthetic but vaguely plausible workload).
- jeffbee 1y agoWith rocks, I assume?
- loeg 1y agoMost of our writes (>99%) aren't rocks.
- jeffbee 1y agoHrmm. Still guessing about your workloads, but isn't it possible that workload A could cause a disproportionate amount of amplification, while still being much smaller in aggregate than workload B?
- loeg 1y agoSeems possible, though we only incorporated FDP changes to the non-rocks writes and saw large WAF reduction. (We're approximately an object store as a service with rocksdb for metadata and direct disk writes for data; typical object size is hundreds of kB to single-digit MBs, and rocks updates are batched. Many of our users can accurately predict how long their objects will live, so we can segregate FDP streams by lifetime.)