2 ms·
Postgres and Sql Server are both similar generations of database architecture, so I would expect them to perform similarly in some aggregate sense. For Postgres
by jandrewrogers 4y ago
Postgres and Sql Server are both similar generations of database architecture, so I would expect them to perform similarly in some aggregate sense. For Postgres, I would argue that the major architectural limitations that impact relative performance are in three areas:
- The storage engine architecture is obsolete and highly suboptimal. It has no way to accommodate modern high-density or high-bandwidth storage, both of which are the norm now, nor is it possible to implement any of the (very successful) I/O schedule optimization concepts that have emerged in recent years. The net effect is a gross underutilization of the capabilities of modern storage hardware.
- The internal page layout and table architecture in Postgres is a textbook classic but not appropriate for most workloads on modern hardware. Modern page engines need great performance across diverse categories of workloads and data models -- we use databases for a lot more than accounting systems these days -- while also being amenable to aggressive use of hardware optimization like SIMD. Thinking in terms of "row-oriented" and "column-oriented" is a false dichotomy; optimized page engines commonly have many elements of both but are not identifiable as pure expressions of either nor even a simple hybrid (e.g. PAX layouts). This is one of the easiest changes to backport into old database architectures.
- Indexing in some modern systems have been completely reimagined to great effect. Tables are effectively index-organized with no secondary indexing structures but without loss of high query selectivity across diverse columns; there is a separation of the concerns regarding fast search and key constraint enforcement; major increases in index write throughput concurrent with major decreases in storage and memory footprint (no B-tree bloat). For small tables, this optimization has no effect and may even be a mild pessimization depending on the workload. As tables become larger the performance starts to diverge greatly, becoming multiple orders of magnitude faster. The Postgres architecture can't be modified to support this.
If you built a database engine that was approximately state-of-the-art in these three areas, I would expect a 100x difference in performance relative to Postgres for many large data model workloads using server hardware like an AWS i3en.24xlarge. Obviously building such a database would be a hell of a lot of work, it isn't something you can reasonably do for fun on nights and weekends.
- zasdffaa 4y agoEdit: this comes across rude (I suppose it is), and maybe I'm wrong to doubt you but I'm no DB n00b and pretty well everything about your post just doesn't make sense, given my knowledge. What? [deleted] A quick browse then... > The storage engine architecture is obsolete and highly suboptimal cite? > It has no way to accommodate modern high-density or high-bandwidth storage WTF does this even mean? Storage is a hardware issue, nothing to do with the DB engine. And it has mmap and huge pages <https://www.postgresql.org/docs/current/runtime-config-resource.html https://www.postgresql.org/docs/current/runtime-config-resou...> so what's it missing? > nor is it possible to implement any of the (very successful) I/O schedule optimization concepts that have emerged in recent years. The net effect is a gross underutilization of the capabilities of modern storage hardware. No evidence given. > Modern page engines need great performance across diverse categories of workloads and data models It's chunks in memory that's all, how is it failing then? > Indexing in some modern systems have been completely reimagined to great effect. Tables are effectively index-organized with no secondary indexing structures just what? > but without loss of high query selectivity across diverse columns selectivity is a feature of data, not how it is organised, at all > there is a separation of the concerns regarding fast search and key constraint enforcement what? what? what? what? what? what? what? what? Search is a read issue, key constraints are a write issue...? > no B-tree bloat seriously WTF? > As tables become larger the performance starts to diverge greatly, becoming multiple orders of magnitude faster "multiple orders of magnitude" - Wuuuut? cite please
- jandrewrogers 4y agoI'm on a long road trip, so can't spend a lot of time responding right now (gotta hit the road). But some quick points: The page layout is important because many workloads are memory-bandwidth bound. Effective selectivity mechanisms outside the page reduce the need for intra-page selectivity optimization in terms of delivering performance but you still need to preserve memory bandwidth. This biases designs with good page-external selectivity mechanisms to optimize for widening the set of workloads they perform well on instead of squeezing out slightly more selectivity for narrow workloads. Some of these assertions are self-evident and non-controversial, such as the poor storage performance of Postgres and the issue of B-tree bloat generally. Using mmap() for storage is the low-performance option, articles regarding which are regularly posted on HN, so the fact that it "has mmap" is a good example of why it is expected to perform poorly (though it isn't the only reason in the case of Postgres). I believe there are plans in the works to potentially redesign the Postgres storage layer in future versions, so this may improve at some point. But more broadly, you seem to be a bit confused about the theory of database kernel design and the tradeoffs that can be made there? You are questioning things about how actual, real, databases, including most open source ones, are designed.
- atombender 4y agoDo you have any good sources to seek out for learning about these improvements, such as optimized page engines and better indexing?