4 ms·

I don’t know, I have observed in my workloads, booting, game load, and building programs, that super fast ssds make almost no difference compared to cheap slow

I don’t know, I have observed in my workloads, booting, game load, and building programs, that super fast ssds make almost no difference compared to cheap slow ssds. But any ssd is miraculous compared to a spinny drive Presumably video editing or something might get more of a win but I don’t know.

noizejoy 5y ago
When I got my first NVMe SSD, I was disappointed that it wasn’t significantly faster than my SATA SSD. But soon I realized that it was Samsung’s Magician software that made the SATA SSD competitive with an NVMe SSD via RAM caching.
lamontcg 5y ago
I'm talking about back in the day when I worked at Amazon with thousands of servers and in the 2001-2006 era we had basically no SSDs and it was all spinny rust (set the slider to 2006 and that is about my mental model). Spinny rust was always shit, so it was all about RAM and network and then the long tail siting on disk. I'm wondering how datacenter SSDs impact architectural decisions with e.g. 'microservices' by having that new latency layer which is cheaper than RAM and faster than network.
- jasonwatkinspdx 5y ago
  Oh, SSDs have definitely had a big impact. On a cost per iop basis, they've been faster for a while, so any workload that's iop limited is on SSDs these days (assuming it doesn't fit in ram). Re your prior comment about Redis style caches, it's worth noting Memcached now supports an SSD optimized external store. It identifies values where it'd be beneficial to spill them to SSD and does so, holding metadata in ram such that the object can be served in a single io (size permitting), and that there's no io required at all to delete/expire an item. It can saturate just about any SSD array you can build right now assuming your network can keep up. It'll take a while, but if you skim the last decade or so of VLDB papers, you can see lots of compelling research into databases that are SSD optimized. It'll take some time for this to coalesce into proven industry solutions but it'll definitely happen. And even with existing databases, I think a lot of people still underestimate what can be done with a sql database on a machine with a beefy SSD array. With more modest scale systems lots of people cargo culting an architecture built around say a cassandra cluster when something much more straightforward would easily suffice. In any case, having millions of ios/sec be totally affordable in a server is a huge shift, and I feel like we still have a lot of layers and inertia in the way of fully utilizing that.
  - lamontcg 5y ago
    Yeah what you're talking about with memcached spilling to the SSD for an extended cache is sort of what I was thinking about. Bit surprised that isn't already much more common of a pattern. And yeah, I'm a fan of sqlite whenever it can be used, then postgresql when you need some beefier. A frightening about of Amazon ran on pre-chewed-up BDBs that were pushed out to every server that needed them (originally to literally every webserver, then pushed out to every server in a microservice cluster), effectively as caches in front of SQL databases. Using sqlite these days should be much better than that since BDBs were buggy and awful.

noizejoy 5y ago

When I got my first NVMe SSD, I was disappointed that it wasn’t significantly faster than my SATA SSD. But soon I realized that it was Samsung’s Magician software that made the SATA SSD competitive with an NVMe SSD via RAM caching.

lamontcg 5y ago

I'm talking about back in the day when I worked at Amazon with thousands of servers and in the 2001-2006 era we had basically no SSDs and it was all spinny rust (set the slider to 2006 and that is about my mental model). Spinny rust was always shit, so it was all about RAM and network and then the long tail siting on disk. I'm wondering how datacenter SSDs impact architectural decisions with e.g. 'microservices' by having that new latency layer which is cheaper than RAM and faster than network.

jasonwatkinspdx 5y ago
Oh, SSDs have definitely had a big impact. On a cost per iop basis, they've been faster for a while, so any workload that's iop limited is on SSDs these days (assuming it doesn't fit in ram). Re your prior comment about Redis style caches, it's worth noting Memcached now supports an SSD optimized external store. It identifies values where it'd be beneficial to spill them to SSD and does so, holding metadata in ram such that the object can be served in a single io (size permitting), and that there's no io required at all to delete/expire an item. It can saturate just about any SSD array you can build right now assuming your network can keep up. It'll take a while, but if you skim the last decade or so of VLDB papers, you can see lots of compelling research into databases that are SSD optimized. It'll take some time for this to coalesce into proven industry solutions but it'll definitely happen. And even with existing databases, I think a lot of people still underestimate what can be done with a sql database on a machine with a beefy SSD array. With more modest scale systems lots of people cargo culting an architecture built around say a cassandra cluster when something much more straightforward would easily suffice. In any case, having millions of ios/sec be totally affordable in a server is a huge shift, and I feel like we still have a lot of layers and inertia in the way of fully utilizing that.
- lamontcg 5y ago
  Yeah what you're talking about with memcached spilling to the SSD for an extended cache is sort of what I was thinking about. Bit surprised that isn't already much more common of a pattern. And yeah, I'm a fan of sqlite whenever it can be used, then postgresql when you need some beefier. A frightening about of Amazon ran on pre-chewed-up BDBs that were pushed out to every server that needed them (originally to literally every webserver, then pushed out to every server in a microservice cluster), effectively as caches in front of SQL databases. Using sqlite these days should be much better than that since BDBs were buggy and awful.