4 ms·

ComDB2: Bloomberg’s Highly Available Relational Database System [pdf]

93 points by irfansharif 10y ago

cwyers 10y ago
At the end of the paper they say they look forward to releasing it under an open source license. That's really interesting.
- elvinyung 10y ago
  In the first page, there's a footnote that says: > Comdb2 will be published under an open source license in 2016 which does not seem to be true.
  - irfansharif 10y ago
    I spoke to Mark Hannum (one of the authors of the paper) recently, it's slated for a release in either this quarter or the next.
    - elvinyung 10y ago
      Ah, cool!
elvinyung 10y ago
There's too many databases that abbreviate to CDB :P This is interesting, but the weird thing is that I'm not sure this is all that different from e.g. hand sharding at the application level. By that I mean that ComDB2 doesn't seem to provide a real way of seamlessly sharding your data. Near the end of the paper: > Under our design, each portion of the table has its own master (getting many of the benefits of a multi-master system) and is able to write to the portion it owns without coordinating with partitions uninvolved in the transaction. Maybe the paper is missing some details on how queries and transactions are coordinated between masters, but it seems to imply that the application level has to handle this kind of stuff.
- irfansharif 10y ago
  The section you're referring to is the work underway to increase write performance, which is currently bounded by the master node's ability to process bplogs (an internal structure used to detect write-write conflicts). re: 'the application level has to handle [sharding]', the table partitioning described in this section is merely a conceptual model 'assigning' masters to what are presumably non-overlapping row ranges in order to increase write throughput (writes/reads will be processed via that specific master). The behaviour of the system still resembles a single node database, this is alluded to intermittently throughout. (also CRDB is the preferred acronym for CockroachDB haha, source: interning there)
  - elvinyung 10y ago
    > The behaviour of the system still resembles a single node database, this is alluded to intermittently throughout. The impression that I get from reading the paper is that this is only true as long as the interaction involves only a single master. Or at the very least, situations where the working set has to be sharded among multiple masters are not described in any detail.
    - alexjscotti 10y ago
      Alex Scotti here. Sharding isn't something we dealt with. As the paper suggested, there was 'work in progress.'
      - elvinyung 10y ago
        I see -- thanks!

93 points by irfansharif 10y ago

At the end of the paper they say they look forward to releasing it under an open source license. That's really interesting.

elvinyung 10y ago
In the first page, there's a footnote that says: > Comdb2 will be published under an open source license in 2016 which does not seem to be true.
- irfansharif 10y ago
  I spoke to Mark Hannum (one of the authors of the paper) recently, it's slated for a release in either this quarter or the next.
  - elvinyung 10y ago
    Ah, cool!

elvinyung 10y ago

There's too many databases that abbreviate to CDB :P This is interesting, but the weird thing is that I'm not sure this is all that different from e.g. hand sharding at the application level. By that I mean that ComDB2 doesn't seem to provide a real way of seamlessly sharding your data. Near the end of the paper: > Under our design, each portion of the table has its own master (getting many of the benefits of a multi-master system) and is able to write to the portion it owns without coordinating with partitions uninvolved in the transaction. Maybe the paper is missing some details on how queries and transactions are coordinated between masters, but it seems to imply that the application level has to handle this kind of stuff.

irfansharif 10y ago
The section you're referring to is the work underway to increase write performance, which is currently bounded by the master node's ability to process bplogs (an internal structure used to detect write-write conflicts). re: 'the application level has to handle [sharding]', the table partitioning described in this section is merely a conceptual model 'assigning' masters to what are presumably non-overlapping row ranges in order to increase write throughput (writes/reads will be processed via that specific master). The behaviour of the system still resembles a single node database, this is alluded to intermittently throughout. (also CRDB is the preferred acronym for CockroachDB haha, source: interning there)
- elvinyung 10y ago
  > The behaviour of the system still resembles a single node database, this is alluded to intermittently throughout. The impression that I get from reading the paper is that this is only true as long as the interaction involves only a single master. Or at the very least, situations where the working set has to be sharded among multiple masters are not described in any detail.
  - alexjscotti 10y ago
    Alex Scotti here. Sharding isn't something we dealt with. As the paper suggested, there was 'work in progress.'
    - elvinyung 10y ago
      I see -- thanks!