Sigh. Databases seems to be one of the last bastions of ivorytower-ism in software engineering. Databases aren't magic. They are software. And as you say, ther

Sigh. Databases seems to be one of the last bastions of ivorytower-ism in software engineering. Databases aren't magic. They are software. And as you say, there is a deep well of literature, and almost all interesting modern databases are open source. So it's possible for anyone with the interest and motivation to learn how they work, and yes, improve them. > With all due respect, you seem to be only loosely familiar with database architecture, both theory and historical practice. Nothing you've described is actually novel. That you are unfamiliar with why no one builds things this way, despite many attempts, does not lend confidence. You claim that there is all this information out there saying why this won't work, and that it has been tried, but don't point to any of it. > Tangentially, the "prolly tree" is intrinsically not a scalable data structure. First: Dolt publishes industry standard performance benchmarks and they are an average of 4.4x slower than MySQL: https://docs.dolthub.com/sql-reference/benchmarks/latency https://docs.dolthub.com/sql-reference/benchmarks/latency This is using the original storage format from noms which wasn't written for oltp workloads. A new storage format is coming which is and will dramatically improve this: https://www.dolthub.com/blog/2022-05-20-new-format-alpha/ https://www.dolthub.com/blog/2022-05-20-new-format-alpha/ In any case 5x slower already shows, experimentally, that this approach works for database-style problems. === Second: In a purely algorithmic sense, prolly trees are the definition of scalable. They are log(n) (with a large base) for inserts, deletes, and seeks and they have efficient ordered scans. They have basically the same algorithmic complexity as B Trees, B+ Trees, LSM trees, etc -- very similar properties to the data structures used by other databases. The problem with prolly trees is actually the reverse: they are scalable, but have large constant factors due to the overhead of hashing. But a single-digit constant factor slower performance than MySQL but with versioning seems like a great product for many applications. If anyone reading this is interested, the Dolt team did a great job writing up the how prolly trees work and how they compare to classic databases here: https://www.dolthub.com/blog/2020-04-01-how-dolt-stores-table-data/ https://www.dolthub.com/blog/2020-04-01-how-dolt-stores-tabl...