8 ms·

The trouble with this is that the database is the hardest part to scale. In a busy site the db may have its work cut out just serving data. This particular pe

The trouble with this is that the database is the hardest part to scale. In a busy site the db may have its work cut out just serving data. This particular pendulum swings back and forth every decade or so. That said, plenty of sites will never have a scaling problem and andl looks interesting

ZenoArrow 10y ago
> "The trouble with this is that the database is the hardest part to scale." Probably depends on the plan followed for data replication, as some approaches seem to make it easier to add new DB nodes as you scale. I've not tried it myself, but I think Apache Samza has some promising ideas in this area: https://www.youtube.com/watch?v=fU9hR3kiOK0 https://www.youtube.com/watch?v=fU9hR3kiOK0
jacques_chester 10y ago
It's not databases that are hard to scale, it's consistent state. It's just that databases begin with the design assumption that you want state to be consistent.
- naasking 10y ago
  This is a great point. I think the way forward is to unite these properties in an append-only schema that can be efficiently queried for a consistent state without requiring expensive transactions. Something like [1] works well enough for RDMSs, but still isn't ideal. [1] http://higherlogics.blogspot.ca/2015/10/versioning-domain-entities.html http://higherlogics.blogspot.ca/2015/10/versioning-domain-en...
  - zaphar 10y ago
    This is also not as easy as it sounds though when you go distributed. Databases already handle the non distributed case pretty well. It's when you need the performance/latency benefits of distribution that you begin to run into problems that an append only schema doesn't help with.
  - takeda 10y ago
    IMO one of great things that came out of NoSQL are CRDTs[1]. This is something that would be interesting to see if it could work with a relational database. [1] https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type https://en.wikipedia.org/wiki/Conflict-free_replicated_data_...
    - naasking 10y ago
      Concurrent Revisions [1] are even better IMO. Much simpler to reason about. In fact, if you know how to work with git and other distributed SCM tools, then you already understand the model: branch, do your work locally, then deterministic merge. [1] http://research.microsoft.com/en-us/projects/revisions/ http://research.microsoft.com/en-us/projects/revisions/
      - Jweb_Guru 10y ago
        Yes, I just found out about these recently and am surprised no datastores have been built around them. They are basically a usable version of fork-join causal consistency. They also subsume CRDTs.
        naasking 10y ago
        Do they? The papers on CRDTs I've skimmed often mention concurrent revisions, but they've never claimed CR subsume CRDTs.
        Jweb_Guru 10y ago
        This paper does (in fact, it points out that any total function can be emulated by a query-update automaton, which can be used with concurrent revisions). At least, that's my recollection, it could be a generalization of CR.
        naasking 10y ago
        What is the "this paper" you're referring to? The papers at the link I provided don't mention CRDTs much.
Jweb_Guru 10y ago
The reality is that your site is nearly always going to be bottlenecked on the database anyway, though. It is generally the case that network latency completely overwhelms the cost of running your code on the database server, so executing everything requiring transactional consistency in a single round-trip communication to the database is usually the most scalable approach. It's also very much the case that centralization yields the best performance when it comes to both latency and throughput, assuming you can find reasonable partitions by datacenter, unless you can find a synchronization-free execution or are willing to tolerate inconsistency (usually, even then you're doing this for availability, not performance, but if you're Facebook or an ad server sometimes for latency too). Moreover, while this isn't usually taken advantage of by contemporary databases or schedulers, in research databases knowing details about the transactions that are going to execute allows dramatic improvements in throughput compared to ad-hoc queries (for a variety of reasons, from being able to find a synchronization-free execution that preserves certain invariants to being able to optimize for data locality to trading latency for throughput with group commit... the list goes on). My personal experience has been that for the vast majority of websites, there simply isn't enough computational work going on over the data to outweigh these benefits. All that being said: I still hate stored procedures with a fiery passion for other reasons (mostly that they often fail to integrate nicely into the rest of the application's deployment mechanisms, especially if you're using a hosted database solution which prevents custom extensions) and try not to let them anywhere near a database unless there are performance reasons to do so. (BTW, just to show I'm not talking out of my ass, here's the latest and greatest in serializable performance within a datacenter (aka centralized): http://research.microsoft.com/pubs/255848/SOSP15-final227.pdf http://research.microsoft.com/pubs/255848/SOSP15-final227.pd.... To quote: "FaRM achieves a peak throughput of 140 million TATP transactions per second on 90 machines with a 4.9 TB database, and it recovers from a failure in less than 50 ms." It also performs 4.5 million TPC-C New Order transactions per second with a 1.9 ms 99% latency [this includes 10% remote partition transactions]).
- kemiller 10y ago
  Food for thought. But what you are describing sounds more performance oriented, not scalability-oriented. Are the speed gains from extra optimization enough to make up form putting more eggs in one basket?

ZenoArrow 10y ago

> "The trouble with this is that the database is the hardest part to scale." Probably depends on the plan followed for data replication, as some approaches seem to make it easier to add new DB nodes as you scale. I've not tried it myself, but I think Apache Samza has some promising ideas in this area: https://www.youtube.com/watch?v=fU9hR3kiOK0 https://www.youtube.com/watch?v=fU9hR3kiOK0

jacques_chester 10y ago

It's not databases that are hard to scale, it's consistent state. It's just that databases begin with the design assumption that you want state to be consistent.

naasking 10y ago
This is a great point. I think the way forward is to unite these properties in an append-only schema that can be efficiently queried for a consistent state without requiring expensive transactions. Something like [1] works well enough for RDMSs, but still isn't ideal. [1] http://higherlogics.blogspot.ca/2015/10/versioning-domain-entities.html http://higherlogics.blogspot.ca/2015/10/versioning-domain-en...
- zaphar 10y ago
  This is also not as easy as it sounds though when you go distributed. Databases already handle the non distributed case pretty well. It's when you need the performance/latency benefits of distribution that you begin to run into problems that an append only schema doesn't help with.
- takeda 10y ago
  IMO one of great things that came out of NoSQL are CRDTs[1]. This is something that would be interesting to see if it could work with a relational database. [1] https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type https://en.wikipedia.org/wiki/Conflict-free_replicated_data_...
  - naasking 10y ago
    Concurrent Revisions [1] are even better IMO. Much simpler to reason about. In fact, if you know how to work with git and other distributed SCM tools, then you already understand the model: branch, do your work locally, then deterministic merge. [1] http://research.microsoft.com/en-us/projects/revisions/ http://research.microsoft.com/en-us/projects/revisions/
    - Jweb_Guru 10y ago
      Yes, I just found out about these recently and am surprised no datastores have been built around them. They are basically a usable version of fork-join causal consistency. They also subsume CRDTs.
      - naasking 10y ago
        Do they? The papers on CRDTs I've skimmed often mention concurrent revisions, but they've never claimed CR subsume CRDTs.
        Jweb_Guru 10y ago
        This paper does (in fact, it points out that any total function can be emulated by a query-update automaton, which can be used with concurrent revisions). At least, that's my recollection, it could be a generalization of CR.
        naasking 10y ago
        What is the "this paper" you're referring to? The papers at the link I provided don't mention CRDTs much.

Jweb_Guru 10y ago

The reality is that your site is nearly always going to be bottlenecked on the database anyway, though. It is generally the case that network latency completely overwhelms the cost of running your code on the database server, so executing everything requiring transactional consistency in a single round-trip communication to the database is usually the most scalable approach. It's also very much the case that centralization yields the best performance when it comes to both latency and throughput, assuming you can find reasonable partitions by datacenter, unless you can find a synchronization-free execution or are willing to tolerate inconsistency (usually, even then you're doing this for availability, not performance, but if you're Facebook or an ad server sometimes for latency too). Moreover, while this isn't usually taken advantage of by contemporary databases or schedulers, in research databases knowing details about the transactions that are going to execute allows dramatic improvements in throughput compared to ad-hoc queries (for a variety of reasons, from being able to find a synchronization-free execution that preserves certain invariants to being able to optimize for data locality to trading latency for throughput with group commit... the list goes on). My personal experience has been that for the vast majority of websites, there simply isn't enough computational work going on over the data to outweigh these benefits. All that being said: I still hate stored procedures with a fiery passion for other reasons (mostly that they often fail to integrate nicely into the rest of the application's deployment mechanisms, especially if you're using a hosted database solution which prevents custom extensions) and try not to let them anywhere near a database unless there are performance reasons to do so. (BTW, just to show I'm not talking out of my ass, here's the latest and greatest in serializable performance within a datacenter (aka centralized): http://research.microsoft.com/pubs/255848/SOSP15-final227.pdf http://research.microsoft.com/pubs/255848/SOSP15-final227.pd.... To quote: "FaRM achieves a peak throughput of 140 million TATP transactions per second on 90 machines with a 4.9 TB database, and it recovers from a failure in less than 50 ms." It also performs 4.5 million TPC-C New Order transactions per second with a 1.9 ms 99% latency [this includes 10% remote partition transactions]).

kemiller 10y ago
Food for thought. But what you are describing sounds more performance oriented, not scalability-oriented. Are the speed gains from extra optimization enough to make up form putting more eggs in one basket?