4 ms·

Sure. But the point of (early) NoSQL was IO optimization, not logical consistency.

bogomipz 7y ago
I was curious about the parenthetical "early" in your comment. Is this optimization no longer the case then? Could you elaborate on recent developments regarding this? I've been out of the NoSQL loop for some time and am genuinely curious.
- tracker1 7y ago
  The problem is, it varies... you have what are effectively key/value stores to document databases, column stores and everything in between. You have systems built on other systems. RethinkDB and Cockroach have different approaches than Redis, Mongo, Cassandra or others. CockroachDB gives an SQL interface over the top of a distributed data store with better consistency and relations. Cassandra has no real relations over a BigTable/Column-Store solution. It really just depends. I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments.
  - dorlaor 7y ago
    We at Scylla tune the kernel, drives, stride, clock source automatically and benchmark the disk to figure out what's its maximum parallelism so it won't queue (we can saturate 8 NVMe) and we'll do the queuing within the DB's userspace so we can prioritize real time queries over background tasks: https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/ https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/
  - bogomipz 7y ago
    >"I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments." I think this is already happening. I think it was Aerospike that used the FTL of of NANd/NVMe drives for a direct key value store and I think another vendor maybe Fusion had an SDK for this as well. The Optane stuff looks really interesting, are there any server vendors shipping with those?
    - Rafuino 7y ago
      Not OP but, yes, server vendors are shipping with Optane DC Persistent Memory, like Lenovo [1] and Supermicro [2], among others. Google also has a testing program [3, 4] with access to instances with 7TB of total memory. [1] https://lenovopress.com/lp1066-intel-optane-dc-persistent-memory https://lenovopress.com/lp1066-intel-optane-dc-persistent-me... [2] https://www.storagereview.com/supermicro_superserver_with_intel_optane_dc_persistent_memory_first_look_review https://www.storagereview.com/supermicro_superserver_with_in... [3] https://cloud.google.com/blog/topics/partners/available-first-on-google-cloud-intel-optane-dc-persistent-memory https://cloud.google.com/blog/topics/partners/available-firs... [4] https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVioClFX5N-061jqO46vzpHzAPHkjwzVw/viewform https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVi...
- jchanimal 7y ago
  By early NoSQL, I mean the projects that are basically an API wrapped around a data structure, that push the understanding of the data structure onto the developer. Newer products (like my employer FaunaDB) prioritize correctness and offer abstract query interfaces like GraphQL.
bunderbunder 7y ago
This one gets hard to pick into, given how nebulous the term NoSQL is. One could probably argue, quite convincingly, that the point of truly early NoSQL was that SQL hadn't been invented yet.
acdha 7y ago
I never saw it portrayed that way: it was always higher-level programmer ease of use pitches along the lines of “isn’t this easier than doing a JOIN?” pitch shortly before the speaker had a valuable learning experience about the value of data integrity and consistency.
- virgilp 7y ago
  You really don't remember the "web scale" memes about Mongo? It was always primarily about "performance".
  - acdha 7y ago
    I don't remember seeing that predicated as a low-level I/O savings — it was always things about being easy to cluster or avoiding the performance overheads of ACID. The pitch was usually “it's easier and you don't need a skilled DBA”.
    - adrianhel 7y ago
      What I heard was "You can practically scale indefinitely". Which I guess is more true of NoSQL. As is I/O savings. But it's not something I've ever heard it advertised for.
  - rhizome 7y ago
    Performance was an imposed callsign of Mongo, the reason everybody started using it was the declarative schema stuff and so the dev didn't have to know how to start a terminal, which was a not-insignificant hurdle. It doesn't have significant performance differences over other simple KVs. "Faster" was primarily a measurement of learning curve.

I was curious about the parenthetical "early" in your comment. Is this optimization no longer the case then? Could you elaborate on recent developments regarding this? I've been out of the NoSQL loop for some time and am genuinely curious.

tracker1 7y ago
The problem is, it varies... you have what are effectively key/value stores to document databases, column stores and everything in between. You have systems built on other systems. RethinkDB and Cockroach have different approaches than Redis, Mongo, Cassandra or others. CockroachDB gives an SQL interface over the top of a distributed data store with better consistency and relations. Cassandra has no real relations over a BigTable/Column-Store solution. It really just depends. I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments.
- dorlaor 7y ago
  We at Scylla tune the kernel, drives, stride, clock source automatically and benchmark the disk to figure out what's its maximum parallelism so it won't queue (we can saturate 8 NVMe) and we'll do the queuing within the DB's userspace so we can prioritize real time queries over background tasks: https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/ https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/
- bogomipz 7y ago
  >"I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments." I think this is already happening. I think it was Aerospike that used the FTL of of NANd/NVMe drives for a direct key value store and I think another vendor maybe Fusion had an SDK for this as well. The Optane stuff looks really interesting, are there any server vendors shipping with those?
  - Rafuino 7y ago
    Not OP but, yes, server vendors are shipping with Optane DC Persistent Memory, like Lenovo [1] and Supermicro [2], among others. Google also has a testing program [3, 4] with access to instances with 7TB of total memory. [1] https://lenovopress.com/lp1066-intel-optane-dc-persistent-memory https://lenovopress.com/lp1066-intel-optane-dc-persistent-me... [2] https://www.storagereview.com/supermicro_superserver_with_intel_optane_dc_persistent_memory_first_look_review https://www.storagereview.com/supermicro_superserver_with_in... [3] https://cloud.google.com/blog/topics/partners/available-first-on-google-cloud-intel-optane-dc-persistent-memory https://cloud.google.com/blog/topics/partners/available-firs... [4] https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVioClFX5N-061jqO46vzpHzAPHkjwzVw/viewform https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVi...
jchanimal 7y ago
By early NoSQL, I mean the projects that are basically an API wrapped around a data structure, that push the understanding of the data structure onto the developer. Newer products (like my employer FaunaDB) prioritize correctness and offer abstract query interfaces like GraphQL.

bunderbunder 7y ago

This one gets hard to pick into, given how nebulous the term NoSQL is. One could probably argue, quite convincingly, that the point of truly early NoSQL was that SQL hadn't been invented yet.

acdha 7y ago

I never saw it portrayed that way: it was always higher-level programmer ease of use pitches along the lines of “isn’t this easier than doing a JOIN?” pitch shortly before the speaker had a valuable learning experience about the value of data integrity and consistency.

virgilp 7y ago
You really don't remember the "web scale" memes about Mongo? It was always primarily about "performance".
- acdha 7y ago
  I don't remember seeing that predicated as a low-level I/O savings — it was always things about being easy to cluster or avoiding the performance overheads of ACID. The pitch was usually “it's easier and you don't need a skilled DBA”.
  - adrianhel 7y ago
    What I heard was "You can practically scale indefinitely". Which I guess is more true of NoSQL. As is I/O savings. But it's not something I've ever heard it advertised for.
- rhizome 7y ago
  Performance was an imposed callsign of Mongo, the reason everybody started using it was the declarative schema stuff and so the dev didn't have to know how to start a terminal, which was a not-insignificant hurdle. It doesn't have significant performance differences over other simple KVs. "Faster" was primarily a measurement of learning curve.