4 ms·

Do I understand correctly that the author went from a distributed database to a single-master scenario? That's a valid tradeoff, but I'd clearly describe it as

by jwr 10y ago

Do I understand correctly that the author went from a distributed database to a single-master scenario? That's a valid tradeoff, but I'd clearly describe it as such. My experiences with RethinkDB have been rather positive, but my load is nowhere near that of what the article describes. I agree that ReQL could be improved, I found that there are too many limitations in chaining once you start using it for more complex things. But the two most important advantages remain for me: * changefeeds (they work for me), * a distributed database that I can run on multiple nodes. I do agree that PostgreSQL is fantastic and that SQL is a fine tool. In my case the above points were the only reasons why I did not use PostgreSQL. EDIT: after thinking about this for a while, I wonder if the RethinkDB changefeed scenario is doable with the tools in PostgreSQL: get initial user data, then get all subsequent changes to that data, with no race conditions. Many workloads seem to concentrate on twitter-like functionality, where the is no clear concept of a change stream beginning and races do not matter.

ruslan_talpa 10y ago
You say that you did not have such loads, so why did you need multiple nodes then? Why the complexity when databases like Postgres can do 1.5M queries per second on a single box? StackOverflow is running just fine on basically one big mssql box (the other is just a standby replica, if i remember correctly)
- jwr 10y ago
  Two reasons: * I expect a larger load in the future, * I want multiple nodes not just for speed, but mostly for data replication.
  - ruslan_talpa 10y ago
    * you are solving problems you don't have * you can have replicas with postgres also even if you do end up getting the load you are hoping for (though i doubt you will have 1.5 million qps), reads are easily scalable with replicas, and writes are also possible with sharding or tools from CitusData
    - ruslan_talpa 10y ago
      I don't have of the top of my head other examples but i know PG is used by yandex.ru which is like the google of russia, and they migrated from oracle and they are very happy, so if it works for them i think it will work for you :)
      - rixed 10y ago
        Don't they use clickhouse instead?
        ruslan_talpa 10y ago
        they probably use a lot of tools for a lot of reasons but they do use postgres https://www.pgcon.org/2016/schedule/attachments/426_2016.05.19%20Yandex.Mail%20success%20story.pdf https://www.pgcon.org/2016/schedule/attachments/426_2016.05.... https://www.youtube.com/watch?v=-SS4R1sFH3c https://www.youtube.com/watch?v=-SS4R1sFH3c
    - williamstein 10y ago
      This is precisely the very painful lesson I learned - don't solve problems you don't have... (Blog author)
    - jazoom 10y ago
      He/she was also not having problems with RethinkDB, so why "solve" that by using Postgres? Disclaimer: I'm also happy running RethinkDB in a cluster of 3. It has been rock solid stable for over 2 years.
  - true_religion 10y ago
    It's much easier to write in a new database in the future, than it is to deal with greater complexity initially. I find that in the first phases, I often don't know what kind of load is going to be where, so I simply use a relational database since they give excelent data security, and okayish performance for most things under low load.
    - davidbanham 10y ago
      I think the inverse is actually the case here. If you need changefeeds, use Rethink up front rather than write your own implementation. When you hit a problem with its performance, then think real hard about your domain and see if it's possible to replicate the safety guarantees it gives you on another DB.
dhd415 10y ago
Unless you have a workload that won't fit on a single machine, a distributed database isn't really an advantage over a single master with one or more appropriate replicas that are available for failover.
deleted 10y ago
[deleted]
davidbanham 10y ago
I strongly suspect that the author has ignored race conditions in his changefeeds implementation. Atomic changefeeds was the thing that made me start using RethinkDB. I'd been excited about it before then but only started using it for production workloads once that feature shipped. They'd be possible to implement in Postgres with some kind of monotonic counter that increases on every update of every row in the table. That would be pretty expensive though.
- phamilton 10y ago
  Atomic changefeeds were a big addition, but resumable changefeeds are still a big gap. Something like what Kafka has. You can emulate this somewhat with atomic changefeeds and updating documents to mark them as "processed".
williamstein 10y ago
> EDIT: after thinking about this for a while, I wonder if the RethinkDB changefeed scenario is doable with the tools in PostgreSQL: get initial user data, then get all subsequent changes to that data, with no race conditions. It's easier if you do things in the other order, and can make assumptions about the structure of your data. In my implementation, I have to first start listening for changes, then do the initial query (relevant code: https://github.com/sagemathinc/smc/blob/ce594ff0574ce781bf784ebd3e0cff2a7a695a73/src/smc-hub/postgres-user-queries.coffee#L1306 https://github.com/sagemathinc/smc/blob/ce594ff0574ce781bf78...). As you say, race conditions aren't an issue for some applications. For me, (slightly simplifying) the main table that involves changefeeds is a table of (timestamp, patch) pairs, and for it, race conditions aren't an issue -- you get the data in whatever order you want and merge it on the client. I'm definitely making no claims to have implemented changefeeds for general PostgreSQL queries or general data. I am concerned that there is an edge case with one of the tables where there is a race condition that causes trouble, in which case I'll have to explicitly change the schema in order to account for this. Somebody below writes "I strongly suspect that the author has ignored race conditions in his changefeeds implementation."; that's not exactly true, since I'm worried about them and try to structure my data to account for them.

by jwr 10y ago

ruslan_talpa 10y ago

You say that you did not have such loads, so why did you need multiple nodes then? Why the complexity when databases like Postgres can do 1.5M queries per second on a single box? StackOverflow is running just fine on basically one big mssql box (the other is just a standby replica, if i remember correctly)

jwr 10y ago
Two reasons: * I expect a larger load in the future, * I want multiple nodes not just for speed, but mostly for data replication.
- ruslan_talpa 10y ago
  * you are solving problems you don't have * you can have replicas with postgres also even if you do end up getting the load you are hoping for (though i doubt you will have 1.5 million qps), reads are easily scalable with replicas, and writes are also possible with sharding or tools from CitusData
  - ruslan_talpa 10y ago
    I don't have of the top of my head other examples but i know PG is used by yandex.ru which is like the google of russia, and they migrated from oracle and they are very happy, so if it works for them i think it will work for you :)
    - rixed 10y ago
      Don't they use clickhouse instead?
      - ruslan_talpa 10y ago
        they probably use a lot of tools for a lot of reasons but they do use postgres https://www.pgcon.org/2016/schedule/attachments/426_2016.05.19%20Yandex.Mail%20success%20story.pdf https://www.pgcon.org/2016/schedule/attachments/426_2016.05.... https://www.youtube.com/watch?v=-SS4R1sFH3c https://www.youtube.com/watch?v=-SS4R1sFH3c
  - williamstein 10y ago
    This is precisely the very painful lesson I learned - don't solve problems you don't have... (Blog author)
  - jazoom 10y ago
    He/she was also not having problems with RethinkDB, so why "solve" that by using Postgres? Disclaimer: I'm also happy running RethinkDB in a cluster of 3. It has been rock solid stable for over 2 years.
- true_religion 10y ago
  It's much easier to write in a new database in the future, than it is to deal with greater complexity initially. I find that in the first phases, I often don't know what kind of load is going to be where, so I simply use a relational database since they give excelent data security, and okayish performance for most things under low load.
  - davidbanham 10y ago
    I think the inverse is actually the case here. If you need changefeeds, use Rethink up front rather than write your own implementation. When you hit a problem with its performance, then think real hard about your domain and see if it's possible to replicate the safety guarantees it gives you on another DB.

dhd415 10y ago

Unless you have a workload that won't fit on a single machine, a distributed database isn't really an advantage over a single master with one or more appropriate replicas that are available for failover.

deleted 10y ago

[deleted]

davidbanham 10y ago

I strongly suspect that the author has ignored race conditions in his changefeeds implementation. Atomic changefeeds was the thing that made me start using RethinkDB. I'd been excited about it before then but only started using it for production workloads once that feature shipped. They'd be possible to implement in Postgres with some kind of monotonic counter that increases on every update of every row in the table. That would be pretty expensive though.

phamilton 10y ago
Atomic changefeeds were a big addition, but resumable changefeeds are still a big gap. Something like what Kafka has. You can emulate this somewhat with atomic changefeeds and updating documents to mark them as "processed".

williamstein 10y ago

> EDIT: after thinking about this for a while, I wonder if the RethinkDB changefeed scenario is doable with the tools in PostgreSQL: get initial user data, then get all subsequent changes to that data, with no race conditions. It's easier if you do things in the other order, and can make assumptions about the structure of your data. In my implementation, I have to first start listening for changes, then do the initial query (relevant code: https://github.com/sagemathinc/smc/blob/ce594ff0574ce781bf784ebd3e0cff2a7a695a73/src/smc-hub/postgres-user-queries.coffee#L1306 https://github.com/sagemathinc/smc/blob/ce594ff0574ce781bf78...). As you say, race conditions aren't an issue for some applications. For me, (slightly simplifying) the main table that involves changefeeds is a table of (timestamp, patch) pairs, and for it, race conditions aren't an issue -- you get the data in whatever order you want and merge it on the client. I'm definitely making no claims to have implemented changefeeds for general PostgreSQL queries or general data. I am concerned that there is an edge case with one of the tables where there is a race condition that causes trouble, in which case I'll have to explicitly change the schema in order to account for this. Somebody below writes "I strongly suspect that the author has ignored race conditions in his changefeeds implementation."; that's not exactly true, since I'm worried about them and try to structure my data to account for them.