4 ms·

Looks amazing. I always loved how Rx composes with IO on the client side, this looks like the missing half. I hope it survives. For people who are soon to be c

Looks amazing. I always loved how Rx composes with IO on the client side, this looks like the missing half. I hope it survives. For people who are soon to be choosing a stack for their projects, please be careful. In 2015, we adopted RethinkDB which had very similar ambitions as RxDB and was open source. Unfortunately, RethinkDB is now abandoned (kinda). Many promising subscribe-able databases are still experimental. If you are thinking long-term, you may want to consider more boring options.

lukevp 6y ago
I agree, I think the move to a more mature state management on the client in general sort of obviated the needs for these databases. Eg if you’re using redux, or mobx, or Apollo, you can cache offline data directly in your state tree, and can define validation functions locally on state transitions to keep your data valid with your business rules while it’s offline. I still feel there’s a space for this tech, but that it needs to integrate into state management as a first class citizen. Object definitions, graphs, and validation functions need to be definable in one place and replicated to clients. I’ve never seen this implemented, but if you don’t, you end up either having a schemaless DB or building your schema twice, and same with validation functions. I shouldn’t have to create functions to observe changes and copy them to and from my redux state, or build my ui around PouchDB’s lifecycle methods. There are middleware providers out there but it’s not first class.
- IggleSniggle 6y ago
  I haven't tried this because I don't need persistent client state, but couldn't you just take your redux or whatever state tree and shove it into a persistent storage, and restore on load?
remon 6y ago
Subscribe-on-update databases are, almost by definition, problematic to use at scale as a generic storage solution. The fundamental problem is that they do not solve many real world problems efficiently enough to warrant the significantly higher running cost. Of course there are exceptions but it'll be hard to launch a MongoDB type of product that uses a subscription only model (see Firebase RTDB and its problems and lack of adoption in this space) The reason developers gravitate towards subscription/rx based paradigms is because it results in very clean architecture and code. Unfortunately it comes at a cost which increases rather than decreases per client/user when volume increases. Some companies or projects can absorb that cost but not all, and typically less so when the project or its userbase grows. A subscription based model will do work whenever data changes for each subscriber whereas more traditional pull based architectures only do work when a client specifically needs the data. This can be mitigated to some extent by being micromanaging subscriptions but that kills most of the value of the model. There are also plenty of issues with this model if multiple clients are allowed to write to the same data which every single example project seems to try and do. There's a reason master-master updates, consensus algorithms and CRDTs all come at significant cost. It's usually hard. And when it's easy you probably don't need it subscribe-on-update in the first place.
- joshribakoff 6y ago
  I think you’re oversimplifying it. I was on teams that participated in architecture design for this stuff at Twitch. If we had 100,000s of clients polling at the same moment, that would absolutely knock over the server, the canonical “stupid easy” fix is to add jitter, which can be done on either client or server. In fact the pull based approach is doing work all the time to process all the polling. A push based approach only does work when needed (when data changes). I’m not saying it’s not new or hard to scale. I’m just saying objectively that push based is more efficient at least in terms of raw data sent down the wire

lukevp 6y ago

I agree, I think the move to a more mature state management on the client in general sort of obviated the needs for these databases. Eg if you’re using redux, or mobx, or Apollo, you can cache offline data directly in your state tree, and can define validation functions locally on state transitions to keep your data valid with your business rules while it’s offline. I still feel there’s a space for this tech, but that it needs to integrate into state management as a first class citizen. Object definitions, graphs, and validation functions need to be definable in one place and replicated to clients. I’ve never seen this implemented, but if you don’t, you end up either having a schemaless DB or building your schema twice, and same with validation functions. I shouldn’t have to create functions to observe changes and copy them to and from my redux state, or build my ui around PouchDB’s lifecycle methods. There are middleware providers out there but it’s not first class.

IggleSniggle 6y ago
I haven't tried this because I don't need persistent client state, but couldn't you just take your redux or whatever state tree and shove it into a persistent storage, and restore on load?

remon 6y ago

Subscribe-on-update databases are, almost by definition, problematic to use at scale as a generic storage solution. The fundamental problem is that they do not solve many real world problems efficiently enough to warrant the significantly higher running cost. Of course there are exceptions but it'll be hard to launch a MongoDB type of product that uses a subscription only model (see Firebase RTDB and its problems and lack of adoption in this space) The reason developers gravitate towards subscription/rx based paradigms is because it results in very clean architecture and code. Unfortunately it comes at a cost which increases rather than decreases per client/user when volume increases. Some companies or projects can absorb that cost but not all, and typically less so when the project or its userbase grows. A subscription based model will do work whenever data changes for each subscriber whereas more traditional pull based architectures only do work when a client specifically needs the data. This can be mitigated to some extent by being micromanaging subscriptions but that kills most of the value of the model. There are also plenty of issues with this model if multiple clients are allowed to write to the same data which every single example project seems to try and do. There's a reason master-master updates, consensus algorithms and CRDTs all come at significant cost. It's usually hard. And when it's easy you probably don't need it subscribe-on-update in the first place.

joshribakoff 6y ago
I think you’re oversimplifying it. I was on teams that participated in architecture design for this stuff at Twitch. If we had 100,000s of clients polling at the same moment, that would absolutely knock over the server, the canonical “stupid easy” fix is to add jitter, which can be done on either client or server. In fact the pull based approach is doing work all the time to process all the polling. A push based approach only does work when needed (when data changes). I’m not saying it’s not new or hard to scale. I’m just saying objectively that push based is more efficient at least in terms of raw data sent down the wire