2 ms·
No, Stonebraker is making many architectural assumptions that do not necessarily hold. Some observations about the design of good disk-based systems: - Typical
by jandrewrogers 12y ago
No, Stonebraker is making many architectural assumptions that do not necessarily hold. Some observations about the design of good disk-based systems:
- Typical inexpensive server and disk systems today are quite a bit different than even five years ago. A database internals design heuristic that would be valid several years ago no longer applies in many cases. An "optimal" design can vary widely due to relatively small changes to the assumed constraints.
- The disk bandwidth typically exceeds network bandwidth by a significant degree. Even if you are saturating the network with inserts and similar, in theory you should be able to drive that through storage if you use the available IOPS efficiently and proactively.
- RAM is RAM. Whether backed by disk or not, the amount of data it can hold is approximately the same. Any significant differences in what you can put in RAM between in-memory and on-disk architectures has more to do with internal implementation and design choices, it is not intrinsic.
- In modern database kernels, no thread is waiting for I/O operations to complete. There is also few or no tree structures, locking, secondary indexing, context switching, and similar which are frequently the source of poor throughput. It tends to be much more linear, pipelined, and parallel.
All of the above aside, regardless of design, a database engine only needs enough throughput to saturate the network interface under load. Once the network is saturated, nothing you do in the database engine will improve the effective throughput of the database.
It turns out that it is pretty straightforward to design a disk-backed database kernel that can saturate 10GbE full-duplex for a wide range of workloads. Consequently, the supposed performance benefits of in-memory are largely moot. It simplifies implementation but does not address a real performance problem on modern hardware relative to a modern database kernel design.