7 ms·

Are you aware of any efforts by existing databaes or entirely new databases built with this realization in mind?

jchrisa 7y ago
NoSQL is based on this realization. Rather than optimize for IO with normalization, denormalize and pipe data more or less directly from storage to clients, simplifying the compute and consistency model to make distributed data easier.
- mike_ivanov 7y ago
  The point of normalization is logical consistency, not IO optimization.
  - jchrisa 7y ago
    Sure. But the point of (early) NoSQL was IO optimization, not logical consistency.
    - bogomipz 7y ago
      I was curious about the parenthetical "early" in your comment. Is this optimization no longer the case then? Could you elaborate on recent developments regarding this? I've been out of the NoSQL loop for some time and am genuinely curious.
      - tracker1 7y ago
        The problem is, it varies... you have what are effectively key/value stores to document databases, column stores and everything in between. You have systems built on other systems. RethinkDB and Cockroach have different approaches than Redis, Mongo, Cassandra or others. CockroachDB gives an SQL interface over the top of a distributed data store with better consistency and relations. Cassandra has no real relations over a BigTable/Column-Store solution. It really just depends. I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments.
        dorlaor 7y ago
        We at Scylla tune the kernel, drives, stride, clock source automatically and benchmark the disk to figure out what's its maximum parallelism so it won't queue (we can saturate 8 NVMe) and we'll do the queuing within the DB's userspace so we can prioritize real time queries over background tasks: https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/ https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/
        bogomipz 7y ago
        >"I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments." I think this is already happening. I think it was Aerospike that used the FTL of of NANd/NVMe drives for a direct key value store and I think another vendor maybe Fusion had an SDK for this as well. The Optane stuff looks really interesting, are there any server vendors shipping with those?
        Rafuino 7y ago
        Not OP but, yes, server vendors are shipping with Optane DC Persistent Memory, like Lenovo [1] and Supermicro [2], among others. Google also has a testing program [3, 4] with access to instances with 7TB of total memory. [1] https://lenovopress.com/lp1066-intel-optane-dc-persistent-memory https://lenovopress.com/lp1066-intel-optane-dc-persistent-me... [2] https://www.storagereview.com/supermicro_superserver_with_intel_optane_dc_persistent_memory_first_look_review https://www.storagereview.com/supermicro_superserver_with_in... [3] https://cloud.google.com/blog/topics/partners/available-first-on-google-cloud-intel-optane-dc-persistent-memory https://cloud.google.com/blog/topics/partners/available-firs... [4] https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVioClFX5N-061jqO46vzpHzAPHkjwzVw/viewform https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVi...
      - jchanimal 7y ago
        By early NoSQL, I mean the projects that are basically an API wrapped around a data structure, that push the understanding of the data structure onto the developer. Newer products (like my employer FaunaDB) prioritize correctness and offer abstract query interfaces like GraphQL.
    - bunderbunder 7y ago
      This one gets hard to pick into, given how nebulous the term NoSQL is. One could probably argue, quite convincingly, that the point of truly early NoSQL was that SQL hadn't been invented yet.
    - acdha 7y ago
      I never saw it portrayed that way: it was always higher-level programmer ease of use pitches along the lines of “isn’t this easier than doing a JOIN?” pitch shortly before the speaker had a valuable learning experience about the value of data integrity and consistency.
      - virgilp 7y ago
        You really don't remember the "web scale" memes about Mongo? It was always primarily about "performance".
        acdha 7y ago
        I don't remember seeing that predicated as a low-level I/O savings — it was always things about being easy to cluster or avoiding the performance overheads of ACID. The pitch was usually “it's easier and you don't need a skilled DBA”.
        adrianhel 7y ago
        What I heard was "You can practically scale indefinitely". Which I guess is more true of NoSQL. As is I/O savings. But it's not something I've ever heard it advertised for.
        rhizome 7y ago
        Performance was an imposed callsign of Mongo, the reason everybody started using it was the declarative schema stuff and so the dev didn't have to know how to start a terminal, which was a not-insignificant hurdle. It doesn't have significant performance differences over other simple KVs. "Faster" was primarily a measurement of learning curve.
  - sieabahlpark 7y ago
    It's both.
    - mike_ivanov 7y ago
      No, it is not. Normalization is an artefact of Relational Algebra (a set theory being applied to tabular data). It has nothing to do with concrete implementations and their concerns (such as IO). http://wiki.c2.com/?RelationalModel http://wiki.c2.com/?RelationalModel. Moreover, ALL existing relational databases do denormalize data behind the scenes (think about caches and indexes) for the sake of IO optimization, which means that normalization does not achieve the IO optimization goal by itself (otherwise, why bother?)
      - bunderbunder 7y ago
        It really is. Originally, schema normalization was just about logical consistency. But that was before query optimizers were invented, and long before they grew to be so very optimized for working with normalized schemata. Nowadays, even someone who cares 100% about performance and 0% about data integrity (And is somehow still using a relational database, yeah. So what? It's MY hypothetical.) still has good reason to default to something like 3NF. Insert famous Knuth quote here. Caches and indexes and all that stuff may technically count as denormalizations, but, insofar as the definition of 3NF doesn't mention them, that's a tangential issue.
        crazygringo 7y ago
        > Nowadays, even someone who cares 100% about performance and 0% about data integrity (And is somehow still using a relational database, yeah. So what? It's MY hypothetical.) still has good reason to default to something like 3NF That's just... wrong. Best practices with database design are to start with a totally normalized schema because it reduces scope for errors/inconsistencies. And then to intentionally denormalize where required to increase performance, in order to reduce lookups from joins or computing aggregate functions -- at the cost of having to manually ensure duplicated data remains consistent. Query optimizers can only do so much. (If it weren't to increase performance, I can't really think of any reason why you ever would denormalize on a single database server, unless you have some extremely complicated calculations that SQL isn't capable of expressing?)
- marshray 7y ago
  I see your analogy, but (NoSQL and its clients) are conceptually much higher up on the stack. Interestingly, the paper does say: [eBPF] enables applications to perform up to L7 protocol (e.g., HTTP or Memcached) processing. Furthermore, some programmable NICs are able to execute eBPF programs directly on the NIC hardware.
dustingetz 7y ago
http://www.datomic.com http://www.datomic.com
- sdegutis 7y ago
  Datomic is just a Prolog logical layer on top of traditional RDBMS (e.g. Postgres, H2, etc).
  - dustingetz 7y ago
    No, Datomic storage is unbundled, prod configurations target basically DynamoDB.
  - lame88 7y ago
    I'd agree it probably doesn't quite apply to this case, but there are similarities from what I've researched, and I think that description doesn't quite do it justice. First being that datomic is temporal with full on-demand history, which I think is a big deal; and has a serialized writer, n-reader architecture. As to how it may relate to this problem, I believe peer applications (in-process) for on-prem are able to exploit locality with the full indexes in-memory without round-trip queries - also the case for ions with datomic cloud. I'll let anyone more experienced clarify or correct me. The concept at least is really cool; I hope to give it a real test drive some time soon.
ignoramous 7y ago
CMU's Andy Pavlo is probably at the forefront in this area, here's what they were working on (they're re-doing it all over again now as a yet to be announced project): https://news.ycombinator.com/item?id=14391523 https://news.ycombinator.com/item?id=14391523 Also, I'd encourage you to go through their Advanced Database class: https://www.youtube.com/playlist?list=PLSE8ODhjZXjYutVzTeAds8xUt1rcmyT7x https://www.youtube.com/playlist?list=PLSE8ODhjZXjYutVzTeAds... Michael Stonebreaker, a Turing awardee, called for complete destruction of old database order in 2008: https://www.dbms2.com/2008/02/18/mike-stonebraker-calls-for-the-complete-destruction-of-the-old-dbms-order/ https://www.dbms2.com/2008/02/18/mike-stonebraker-calls-for-... post which, I think, they created VoltDB (H-Store) and SciDB. Interesting to note that Andy Pavlo worked on Stonebreaker's H-Store.
- hueving 7y ago
  Why do you suppose nothing useful has come out of the CMU group for actual use? Is it just a case of limited payoff for marginal gain?
  - ignoramous 7y ago
    I'm in no position to comment on that, unfortunately. Not sure if Andy's group has detailed why their first-take didn't come off apart from this terse note on GitHub which notes that building an autonomous-db is what they're main goal is (rather than a nvme-first db, which seems like a sub-goal): > The Peloton project is dead. We have abandoned this repository and moved on to build a new DBMS. There are a several engineering techniques and designs that we learned from this first system on how to support autonomous operations that we are doing a much better job at implementing in the second system. Here's their alleged unannounced second-take: https://github.com/cmu-db/terrier https://github.com/cmu-db/terrier
Jach 7y ago
ScyllaDB is probably in the right direction. It at least bypasses the kernel for networking (uses Intel's DPDK).
- PeterCorless 7y ago
  Yes. Pekka works at ScyllaDB, and the paper references Avi Kivity's work at Scylla twice in the endnotes, as well as the underlying Seastar engine that Scylla is built upon (that's where DPDK is embedded).
farazbabar 7y ago
Aerospike
jandrewrogers 7y ago
Yes, there are efforts in this area. I started a first-principles database kernel design 4-5 years ago which has a couple different licensees and implementations, and should be showing up in a broad platform relatively soon. In open source, ScyllaDB is a credible implementation that goes half-way to addressing this (as an intentional reimplementation of Cassandra, it inherits some of the old style architectural features of Cassandra for better and worse). I am aware of a few other efforts at tech companies but in my opinion these are a mixed bag because they are borrowing too heavily from existing classic architectures for the sake of expediency to fully realize the benefits. The archetypical differentiating features, relative to classic designs, of these future-looking (for lack of a better term) kernel designs is the prodigious exploitation of user space I/O, dearth of multithreaded synchronization, and the paucity of classic tree-like data structures (not necessarily hash tables, just not trees). The industry will get here eventually, some emerging tech sectors absolutely require it for their data models.
AtlasBarfed 7y ago
IMO the biggest problem is that SQL joins don't scale on commodity hardware networks due to the CAP theorem, assuming you want availability. Network IO is a lot better these days, and per Scylla it can get to a pretty small instruction window, but still it isn't what XPoint and SSD crunch. Database rearchitecture at the single node level can help optimize this, and maybe improve the megamachine SQL node to scale up to fatter sizes, but really big data is still a network IO and reliability challenge. And it may be true we can pump a single fat node up to really huge throughput once some db rearchitecture fully leverages SSDs and I/O, but that just exposes more data to downtime if there is a network partition.

NoSQL is based on this realization. Rather than optimize for IO with normalization, denormalize and pipe data more or less directly from storage to clients, simplifying the compute and consistency model to make distributed data easier.

mike_ivanov 7y ago
The point of normalization is logical consistency, not IO optimization.
- jchrisa 7y ago
  Sure. But the point of (early) NoSQL was IO optimization, not logical consistency.
  - bogomipz 7y ago
    I was curious about the parenthetical "early" in your comment. Is this optimization no longer the case then? Could you elaborate on recent developments regarding this? I've been out of the NoSQL loop for some time and am genuinely curious.
    - tracker1 7y ago
      The problem is, it varies... you have what are effectively key/value stores to document databases, column stores and everything in between. You have systems built on other systems. RethinkDB and Cockroach have different approaches than Redis, Mongo, Cassandra or others. CockroachDB gives an SQL interface over the top of a distributed data store with better consistency and relations. Cassandra has no real relations over a BigTable/Column-Store solution. It really just depends. I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments.
      - dorlaor 7y ago
        We at Scylla tune the kernel, drives, stride, clock source automatically and benchmark the disk to figure out what's its maximum parallelism so it won't queue (we can saturate 8 NVMe) and we'll do the queuing within the DB's userspace so we can prioritize real time queries over background tasks: https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/ https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/
      - bogomipz 7y ago
        >"I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments." I think this is already happening. I think it was Aerospike that used the FTL of of NANd/NVMe drives for a direct key value store and I think another vendor maybe Fusion had an SDK for this as well. The Optane stuff looks really interesting, are there any server vendors shipping with those?
        Rafuino 7y ago
        Not OP but, yes, server vendors are shipping with Optane DC Persistent Memory, like Lenovo [1] and Supermicro [2], among others. Google also has a testing program [3, 4] with access to instances with 7TB of total memory. [1] https://lenovopress.com/lp1066-intel-optane-dc-persistent-memory https://lenovopress.com/lp1066-intel-optane-dc-persistent-me... [2] https://www.storagereview.com/supermicro_superserver_with_intel_optane_dc_persistent_memory_first_look_review https://www.storagereview.com/supermicro_superserver_with_in... [3] https://cloud.google.com/blog/topics/partners/available-first-on-google-cloud-intel-optane-dc-persistent-memory https://cloud.google.com/blog/topics/partners/available-firs... [4] https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVioClFX5N-061jqO46vzpHzAPHkjwzVw/viewform https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVi...
    - jchanimal 7y ago
      By early NoSQL, I mean the projects that are basically an API wrapped around a data structure, that push the understanding of the data structure onto the developer. Newer products (like my employer FaunaDB) prioritize correctness and offer abstract query interfaces like GraphQL.
  - bunderbunder 7y ago
    This one gets hard to pick into, given how nebulous the term NoSQL is. One could probably argue, quite convincingly, that the point of truly early NoSQL was that SQL hadn't been invented yet.
  - acdha 7y ago
    I never saw it portrayed that way: it was always higher-level programmer ease of use pitches along the lines of “isn’t this easier than doing a JOIN?” pitch shortly before the speaker had a valuable learning experience about the value of data integrity and consistency.
    - virgilp 7y ago
      You really don't remember the "web scale" memes about Mongo? It was always primarily about "performance".
      - acdha 7y ago
        I don't remember seeing that predicated as a low-level I/O savings — it was always things about being easy to cluster or avoiding the performance overheads of ACID. The pitch was usually “it's easier and you don't need a skilled DBA”.
        adrianhel 7y ago
        What I heard was "You can practically scale indefinitely". Which I guess is more true of NoSQL. As is I/O savings. But it's not something I've ever heard it advertised for.
      - rhizome 7y ago
        Performance was an imposed callsign of Mongo, the reason everybody started using it was the declarative schema stuff and so the dev didn't have to know how to start a terminal, which was a not-insignificant hurdle. It doesn't have significant performance differences over other simple KVs. "Faster" was primarily a measurement of learning curve.
- sieabahlpark 7y ago
  It's both.
  - mike_ivanov 7y ago
    No, it is not. Normalization is an artefact of Relational Algebra (a set theory being applied to tabular data). It has nothing to do with concrete implementations and their concerns (such as IO). http://wiki.c2.com/?RelationalModel http://wiki.c2.com/?RelationalModel. Moreover, ALL existing relational databases do denormalize data behind the scenes (think about caches and indexes) for the sake of IO optimization, which means that normalization does not achieve the IO optimization goal by itself (otherwise, why bother?)
    - bunderbunder 7y ago
      It really is. Originally, schema normalization was just about logical consistency. But that was before query optimizers were invented, and long before they grew to be so very optimized for working with normalized schemata. Nowadays, even someone who cares 100% about performance and 0% about data integrity (And is somehow still using a relational database, yeah. So what? It's MY hypothetical.) still has good reason to default to something like 3NF. Insert famous Knuth quote here. Caches and indexes and all that stuff may technically count as denormalizations, but, insofar as the definition of 3NF doesn't mention them, that's a tangential issue.
      - crazygringo 7y ago
        > Nowadays, even someone who cares 100% about performance and 0% about data integrity (And is somehow still using a relational database, yeah. So what? It's MY hypothetical.) still has good reason to default to something like 3NF That's just... wrong. Best practices with database design are to start with a totally normalized schema because it reduces scope for errors/inconsistencies. And then to intentionally denormalize where required to increase performance, in order to reduce lookups from joins or computing aggregate functions -- at the cost of having to manually ensure duplicated data remains consistent. Query optimizers can only do so much. (If it weren't to increase performance, I can't really think of any reason why you ever would denormalize on a single database server, unless you have some extremely complicated calculations that SQL isn't capable of expressing?)
marshray 7y ago
I see your analogy, but (NoSQL and its clients) are conceptually much higher up on the stack. Interestingly, the paper does say: [eBPF] enables applications to perform up to L7 protocol (e.g., HTTP or Memcached) processing. Furthermore, some programmable NICs are able to execute eBPF programs directly on the NIC hardware.

dustingetz 7y ago

http://www.datomic.com http://www.datomic.com

sdegutis 7y ago
Datomic is just a Prolog logical layer on top of traditional RDBMS (e.g. Postgres, H2, etc).
- dustingetz 7y ago
  No, Datomic storage is unbundled, prod configurations target basically DynamoDB.
- lame88 7y ago
  I'd agree it probably doesn't quite apply to this case, but there are similarities from what I've researched, and I think that description doesn't quite do it justice. First being that datomic is temporal with full on-demand history, which I think is a big deal; and has a serialized writer, n-reader architecture. As to how it may relate to this problem, I believe peer applications (in-process) for on-prem are able to exploit locality with the full indexes in-memory without round-trip queries - also the case for ions with datomic cloud. I'll let anyone more experienced clarify or correct me. The concept at least is really cool; I hope to give it a real test drive some time soon.

ignoramous 7y ago

CMU's Andy Pavlo is probably at the forefront in this area, here's what they were working on (they're re-doing it all over again now as a yet to be announced project): https://news.ycombinator.com/item?id=14391523 https://news.ycombinator.com/item?id=14391523 Also, I'd encourage you to go through their Advanced Database class: https://www.youtube.com/playlist?list=PLSE8ODhjZXjYutVzTeAds8xUt1rcmyT7x https://www.youtube.com/playlist?list=PLSE8ODhjZXjYutVzTeAds... Michael Stonebreaker, a Turing awardee, called for complete destruction of old database order in 2008: https://www.dbms2.com/2008/02/18/mike-stonebraker-calls-for-the-complete-destruction-of-the-old-dbms-order/ https://www.dbms2.com/2008/02/18/mike-stonebraker-calls-for-... post which, I think, they created VoltDB (H-Store) and SciDB. Interesting to note that Andy Pavlo worked on Stonebreaker's H-Store.

hueving 7y ago
Why do you suppose nothing useful has come out of the CMU group for actual use? Is it just a case of limited payoff for marginal gain?
- ignoramous 7y ago
  I'm in no position to comment on that, unfortunately. Not sure if Andy's group has detailed why their first-take didn't come off apart from this terse note on GitHub which notes that building an autonomous-db is what they're main goal is (rather than a nvme-first db, which seems like a sub-goal): > The Peloton project is dead. We have abandoned this repository and moved on to build a new DBMS. There are a several engineering techniques and designs that we learned from this first system on how to support autonomous operations that we are doing a much better job at implementing in the second system. Here's their alleged unannounced second-take: https://github.com/cmu-db/terrier https://github.com/cmu-db/terrier

Jach 7y ago

ScyllaDB is probably in the right direction. It at least bypasses the kernel for networking (uses Intel's DPDK).

PeterCorless 7y ago
Yes. Pekka works at ScyllaDB, and the paper references Avi Kivity's work at Scylla twice in the endnotes, as well as the underlying Seastar engine that Scylla is built upon (that's where DPDK is embedded).

farazbabar 7y ago

Aerospike

jandrewrogers 7y ago

Yes, there are efforts in this area. I started a first-principles database kernel design 4-5 years ago which has a couple different licensees and implementations, and should be showing up in a broad platform relatively soon. In open source, ScyllaDB is a credible implementation that goes half-way to addressing this (as an intentional reimplementation of Cassandra, it inherits some of the old style architectural features of Cassandra for better and worse). I am aware of a few other efforts at tech companies but in my opinion these are a mixed bag because they are borrowing too heavily from existing classic architectures for the sake of expediency to fully realize the benefits. The archetypical differentiating features, relative to classic designs, of these future-looking (for lack of a better term) kernel designs is the prodigious exploitation of user space I/O, dearth of multithreaded synchronization, and the paucity of classic tree-like data structures (not necessarily hash tables, just not trees). The industry will get here eventually, some emerging tech sectors absolutely require it for their data models.

AtlasBarfed 7y ago

IMO the biggest problem is that SQL joins don't scale on commodity hardware networks due to the CAP theorem, assuming you want availability. Network IO is a lot better these days, and per Scylla it can get to a pretty small instruction window, but still it isn't what XPoint and SSD crunch. Database rearchitecture at the single node level can help optimize this, and maybe improve the megamachine SQL node to scale up to fatter sizes, but really big data is still a network IO and reliability challenge. And it may be true we can pump a single fat node up to really huge throughput once some db rearchitecture fully leverages SSDs and I/O, but that just exposes more data to downtime if there is a network partition.