5 ms·

Executing Parallel Statements to Improve Performance

irfansharif 9y ago
The accompanying RFC, Parallel SQL Statement Execution: https://github.com/cockroachdb/cockroach/blob/0da3da734ce6bfdf55f31b973130c3f58c8a1c7c/docs/RFCS/20170124_sql_parallelization.md https://github.com/cockroachdb/cockroach/blob/0da3da734ce6bf...
capkutay 9y ago
This seems to be a trend that I see across open source projects. They blog about features that Oracle has had for 25 years as if it's something new. Can someone enlighten me as to why there's interest around X DB's implementation of something that's been built dozens of times before?
- endymi0n 9y ago
  With all due respect, Oracle doesn't provide ACID semantics across a cluster spanning several continents...
  - lgas 9y ago
    Isn't that exactly what "Extended Distance Oracle RAC Clusters" provide?
    - danielmg 9y ago
      No. That still uses ASM. It's effectively low level IO mirroring and needs fast data-centre interconnects and low-latency. I work a lot with RAC and it introduces more problems than it solves tbh.
    - anonacct37 9y ago
      It's almost a meme. I was often told Companies RAC, experience RAC-caused outages, and then they de-RAC.
whileoneloop 9y ago
If you’re inventing new syntax... Wouldn’t an a async/await be more useful - to declaratively indicate the desired dependencies between simultaneously operating statements? That way you can work with the results of an inserted PK or the row count to inform later statements - when needed. Eg. ‘returning async’
- andreimatei1 9y ago
  I believe this suggestion came up at the time. I think we didn't do anything like it because it would have been more complicated both to implement and to describe, and because it's not particularly common for people to use/need the values that can be returned by the statements that we can parallelize. The future, though, is long and open.
carterehsmith 9y ago
"CockroachDB maintains strong consistency across a globally-distributed cluster of computers." No, CockroachDB does not maintain strong consistency across a globally-distributed cluster of computers. It is a "goal", but not the reality. People reviewed these claims, and they do not hold (see e.g. Aphyr's review)
- couchand 9y ago
  I think your information is incomplete or out of date. Do you have a source other than the Jepsen blog? That seems to make clear that the two issues that were found in the tested beta release were quickly resolved. [0] [0]: https://jepsen.io/analyses/cockroachdb-beta-20160829 https://jepsen.io/analyses/cockroachdb-beta-20160829
  - carterehsmith 9y ago
    So you guys claim ""CockroachDB maintains strong consistency across a globally-distributed cluster of computers." " Then some random dude walks in and finds that... the claims are false. Oh, you fixed the bugs... great. Here is the real question. If you guys really know what you are doing, there would be no chance of someone... Aphyr or not, breaking your consistency guarantees. In fact, you would be way ahead of Aphyr or whoever else can throw tests at you. Just my 5c
- andreimatei1 9y ago
  Why do you say that they don't hold, exactly? We take our consistency guarantees, and correctness more generally, deadly serious. I think it's fair to say that we did quite well on Aphyr's review; he's found some quite obscure issues that were quickly fixed. More details here: https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-jepsen-testing/ https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-j...
  - carterehsmith 9y ago
    "We take our consistency guarantees, and correctness more generally, deadly serious" That is empty talk, ie advertising. In a distributed database, as is the state of the art, you sacrifice one of: availability, consistency, performance. This is true for every db, be it Spanner, or MongoDb. Best to document what exactly was sacrificed. Just claiming that you "take it seriously" means nothing. BTW good luck :)

The accompanying RFC, Parallel SQL Statement Execution: https://github.com/cockroachdb/cockroach/blob/0da3da734ce6bfdf55f31b973130c3f58c8a1c7c/docs/RFCS/20170124_sql_parallelization.md https://github.com/cockroachdb/cockroach/blob/0da3da734ce6bf...

capkutay 9y ago

This seems to be a trend that I see across open source projects. They blog about features that Oracle has had for 25 years as if it's something new. Can someone enlighten me as to why there's interest around X DB's implementation of something that's been built dozens of times before?

endymi0n 9y ago
With all due respect, Oracle doesn't provide ACID semantics across a cluster spanning several continents...
- lgas 9y ago
  Isn't that exactly what "Extended Distance Oracle RAC Clusters" provide?
  - danielmg 9y ago
    No. That still uses ASM. It's effectively low level IO mirroring and needs fast data-centre interconnects and low-latency. I work a lot with RAC and it introduces more problems than it solves tbh.
  - anonacct37 9y ago
    It's almost a meme. I was often told Companies RAC, experience RAC-caused outages, and then they de-RAC.

whileoneloop 9y ago

If you’re inventing new syntax... Wouldn’t an a async/await be more useful - to declaratively indicate the desired dependencies between simultaneously operating statements? That way you can work with the results of an inserted PK or the row count to inform later statements - when needed. Eg. ‘returning async’

andreimatei1 9y ago
I believe this suggestion came up at the time. I think we didn't do anything like it because it would have been more complicated both to implement and to describe, and because it's not particularly common for people to use/need the values that can be returned by the statements that we can parallelize. The future, though, is long and open.

carterehsmith 9y ago

"CockroachDB maintains strong consistency across a globally-distributed cluster of computers." No, CockroachDB does not maintain strong consistency across a globally-distributed cluster of computers. It is a "goal", but not the reality. People reviewed these claims, and they do not hold (see e.g. Aphyr's review)

couchand 9y ago
I think your information is incomplete or out of date. Do you have a source other than the Jepsen blog? That seems to make clear that the two issues that were found in the tested beta release were quickly resolved. [0] [0]: https://jepsen.io/analyses/cockroachdb-beta-20160829 https://jepsen.io/analyses/cockroachdb-beta-20160829
- carterehsmith 9y ago
  So you guys claim ""CockroachDB maintains strong consistency across a globally-distributed cluster of computers." " Then some random dude walks in and finds that... the claims are false. Oh, you fixed the bugs... great. Here is the real question. If you guys really know what you are doing, there would be no chance of someone... Aphyr or not, breaking your consistency guarantees. In fact, you would be way ahead of Aphyr or whoever else can throw tests at you. Just my 5c
andreimatei1 9y ago
Why do you say that they don't hold, exactly? We take our consistency guarantees, and correctness more generally, deadly serious. I think it's fair to say that we did quite well on Aphyr's review; he's found some quite obscure issues that were quickly fixed. More details here: https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-jepsen-testing/ https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-j...
- carterehsmith 9y ago
  "We take our consistency guarantees, and correctness more generally, deadly serious" That is empty talk, ie advertising. In a distributed database, as is the state of the art, you sacrifice one of: availability, consistency, performance. This is true for every db, be it Spanner, or MongoDb. Best to document what exactly was sacrificed. Just claiming that you "take it seriously" means nothing. BTW good luck :)