4 ms·

AWS docs and blogs describe the Nitro SSD architecture, which is locally attached with custom firmware. > The Nitro Cards are physically connected to the syste

AWS docs and blogs describe the Nitro SSD architecture, which is locally attached with custom firmware. > The Nitro Cards are physically connected to the system main board and its processors via PCIe, but are otherwise logically isolated from the system main board that runs customer workloads. https://docs.aws.amazon.com/whitepapers/latest/security-design-of-aws-nitro-system/the-components-of-the-nitro-system.html https://docs.aws.amazon.com/whitepapers/latest/security-desi... > In order to make the [SSD] devices last as long as possible, the firmware is responsible for a process known as wear leveling.... There’s some housekeeping (a form of garbage collection) involved in this process, and garden-variety SSDs can slow down (creating latency spikes) at unpredictable times when dealing with a barrage of writes. We also took advantage of our database expertise and built a very sophisticated, power-fail-safe journal-based database into the SSD firmware. https://aws.amazon.com/blogs/aws/aws-nitro-ssd-high-performance-storage-for-your-i-o-intensive-applications/ https://aws.amazon.com/blogs/aws/aws-nitro-ssd-high-performa... This firmware layer seems like a good candidate for the slowdown.

someguydave 3y ago
Yeah I wonder how Nitro balances the latency & bandwidth demands of multiple VMs while also minimizing memory cache misses on the CPU (I am assuming it uses DMA to talk to the main CPU cores)
dan-robertson 3y ago
Yeah, I’m curious how they would respond to the claims in the article. In [1], they talk about aiming for low latency, for consistent performance (apparently other SSDs could stall at inopportune times), and support on-disk encryption. Latency is often in direct conflict with throughput (eg batching usually trades one for the other), and also matters a lot for plenty of filesystem or database tasks (indeed the OP links to a paper showing that popular databases, even column stores, struggle to use the full disk throughput, though I didn’t read why). Encryption is probably not the reason – dedicated hardware on modern chips can do AES at 50GB/s, though maybe it is if it increases latency? So maybe there’s something else to it like sharing between many vms on one host [1] https://m.youtube.com/watch?v=Cxie0FgLogg https://m.youtube.com/watch?v=Cxie0FgLogg
- kwillets 3y ago
  The Nitro chipset claims 100 GB/s encryption, so that doesn't seem to be the reason.
  - deleted 3y ago
    [deleted]
akira2501 3y ago
I've noticed the firmware programming positions have been more common in their job listings lately.

someguydave 3y ago

Yeah I wonder how Nitro balances the latency & bandwidth demands of multiple VMs while also minimizing memory cache misses on the CPU (I am assuming it uses DMA to talk to the main CPU cores)

dan-robertson 3y ago

Yeah, I’m curious how they would respond to the claims in the article. In [1], they talk about aiming for low latency, for consistent performance (apparently other SSDs could stall at inopportune times), and support on-disk encryption. Latency is often in direct conflict with throughput (eg batching usually trades one for the other), and also matters a lot for plenty of filesystem or database tasks (indeed the OP links to a paper showing that popular databases, even column stores, struggle to use the full disk throughput, though I didn’t read why). Encryption is probably not the reason – dedicated hardware on modern chips can do AES at 50GB/s, though maybe it is if it increases latency? So maybe there’s something else to it like sharing between many vms on one host [1] https://m.youtube.com/watch?v=Cxie0FgLogg https://m.youtube.com/watch?v=Cxie0FgLogg

kwillets 3y ago
The Nitro chipset claims 100 GB/s encryption, so that doesn't seem to be the reason.
- deleted 3y ago
  [deleted]

akira2501 3y ago

I've noticed the firmware programming positions have been more common in their job listings lately.