4 ms·
Oh, SSDs have definitely had a big impact. On a cost per iop basis, they've been faster for a while, so any workload that's iop limited is on SSDs these days (a
by jasonwatkinspdx 5y ago
Oh, SSDs have definitely had a big impact. On a cost per iop basis, they've been faster for a while, so any workload that's iop limited is on SSDs these days (assuming it doesn't fit in ram).
Re your prior comment about Redis style caches, it's worth noting Memcached now supports an SSD optimized external store. It identifies values where it'd be beneficial to spill them to SSD and does so, holding metadata in ram such that the object can be served in a single io (size permitting), and that there's no io required at all to delete/expire an item. It can saturate just about any SSD array you can build right now assuming your network can keep up.
It'll take a while, but if you skim the last decade or so of VLDB papers, you can see lots of compelling research into databases that are SSD optimized. It'll take some time for this to coalesce into proven industry solutions but it'll definitely happen.
And even with existing databases, I think a lot of people still underestimate what can be done with a sql database on a machine with a beefy SSD array. With more modest scale systems lots of people cargo culting an architecture built around say a cassandra cluster when something much more straightforward would easily suffice.
In any case, having millions of ios/sec be totally affordable in a server is a huge shift, and I feel like we still have a lot of layers and inertia in the way of fully utilizing that.
- lamontcg 5y agoYeah what you're talking about with memcached spilling to the SSD for an extended cache is sort of what I was thinking about. Bit surprised that isn't already much more common of a pattern. And yeah, I'm a fan of sqlite whenever it can be used, then postgresql when you need some beefier. A frightening about of Amazon ran on pre-chewed-up BDBs that were pushed out to every server that needed them (originally to literally every webserver, then pushed out to every server in a microservice cluster), effectively as caches in front of SQL databases. Using sqlite these days should be much better than that since BDBs were buggy and awful.