5 ms·

I feel like I'm missing something in this article. Several of the points just don't make any sense to me. * Their point about how you can just rewrite relation

by er4hn 5y ago

I feel like I'm missing something in this article. Several of the points just don't make any sense to me. * Their point about how you can just rewrite relational queries manually and not lose any performance does not seem true the moment that you need to join tables on a condition. * Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries. I don't understand how their NoSql model of "Download the latest doc during a change" is any better. It basically becomes "last writer wins" in a distributed system. * The downgrading to NoSql argument makes little sense to me as well. If your frontend is sqlite then you can have Postgres, or MariaDb in your backend. You need to account for having different queries on the front and the back, but the backend is presumably also operating under very different constraints than the frontend. * I'd actually make an argument that NoSql databases are a type of relational database heavily optimized towards not needing to do joins. The corollary to this is that over time you may find that, oops, you do want to do joins, and now you have to pay a tax on not being set up to do so in the first place. * Also, is it just me or is the Hybrid Logical Clock they mention a type of Lamport Clock where the linked article doesn't ever state that?

quixoticaxolotl 5y ago
Hybrid logical clocks are distinct from Lamport clocks in that they include a notion of local timestamp while also preserving partial order. They are well-established and used widely outside RxDB - see e.g. https://sergeiturukin.com/2017/06/26/hybrid-logical-clocks.html https://sergeiturukin.com/2017/06/26/hybrid-logical-clocks.h...
EdwardDiego 5y ago
> Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries I do love Debezium for easily streaming changelogs to Kafka.
lmm 5y ago
> Their point about how you can just rewrite relational queries manually and not lose any performance does not seem true the moment that you need to join tables on a condition. You can do the same thing a database would do - filter the first results on that condition before firing off the second query, or do the join "backwards" if you think that's going to be better / there's an index available for that. > Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries. You'd need to define a representation of that log, which ends up being equivalent to writing a NoSQL datastore. > I don't understand how their NoSql model of "Download the latest doc during a change" is any better. It basically becomes "last writer wins" in a distributed system. You at least get consistency. And NoSQL gives you the option of building something better like e.g. Riak does. > The downgrading to NoSql argument makes little sense to me as well. If your frontend is sqlite then you can have Postgres, or MariaDb in your backend. You need to account for having different queries on the front and the back, but the backend is presumably also operating under very different constraints than the frontend. This is kind of the same as the log replication problem - you want the protocol for what's replicated from frontend to backend to be something simple that you could implement on top of approximately any backend datastore. "SQL queries" aren't that, whereas a simple K/V store protocol could be (e.g. MySQL actually implements the memcached query protocol, or did at one point - that would be very hard to do the other way around). > I'd actually make an argument that NoSql databases are a type of relational database heavily optimized towards not needing to do joins Um, WTF? Can you flesh this out at all?
- sourcesmith 5y ago
  > > Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries. > You'd need to define a representation of that log, which ends up being equivalent to writing a NoSQL datastore. I think this covered by relational DBs redo/rollback/write-ahead log; whatever you want to call it.
  - lmm 5y ago
    Right, but they generally keep it as a hidden implementation detail, and when you do get access to it it's not in any standardised way. For a lot of use cases you're better off unbundling it and newer datastores tend to be better at that.
- er4hn 5y ago
  > Um, WTF? Can you flesh this out at all? Ha, sure. Let's say that you have a very-basic NoSql schema for a chat app: chat_app/person/<person id>/msgs/<chat documents> so for each <person id> you stick chat messages in the <chat documents> separated by date or something. Conversations with different people end up in different <person id> paths. Chats have metadata like timestamps in addition to the message content You can also represent this as a SQL table with something like: Person_Id (uint) | Timestamp (timestamp) | Message (string) Now the classic argument for NoSql is that NoSql is the best representation for this since you don't really care about doing complicated queries over the chats. You just want to be able to index in and get a persons chats. Maybe sort by the most recent chats so the <chat documents> have the timestamp in the key. That's what I mean. Where I get frustrated with NoSql is that you inevitably end up needing to redo a lot of SQL logic anyways. A basic example is mapping the person_id -> visible name. The NoSql way is to have a meta file under <person id>, like chat_app/person/<person id>/<meta file> where you would have the user name under there. Then you can look up easily what to display when showing chats for <person id>. But then you add group chats. Now you need to also have a group chat section and you need to go back to the person_id path to resolve the visible names. Having a relational table with: Person Id | Visible Name seems like a more elegant solution to me. Once you need to start gathering stats on the data as well, relational tables become more easy to gather information on how often you are talking to various people or lengths of conversations with people. Most of the functionality for statistical gathering and other use cases can be replicated in NoSql but it involves extra effort that SQL just gives you.

by er4hn 5y ago

quixoticaxolotl 5y ago

Hybrid logical clocks are distinct from Lamport clocks in that they include a notion of local timestamp while also preserving partial order. They are well-established and used widely outside RxDB - see e.g. https://sergeiturukin.com/2017/06/26/hybrid-logical-clocks.html https://sergeiturukin.com/2017/06/26/hybrid-logical-clocks.h...

EdwardDiego 5y ago

> Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries I do love Debezium for easily streaming changelogs to Kafka.

lmm 5y ago

> Their point about how you can just rewrite relational queries manually and not lose any performance does not seem true the moment that you need to join tables on a condition. You can do the same thing a database would do - filter the first results on that condition before firing off the second query, or do the join "backwards" if you think that's going to be better / there's an index available for that. > Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries. You'd need to define a representation of that log, which ends up being equivalent to writing a NoSQL datastore. > I don't understand how their NoSql model of "Download the latest doc during a change" is any better. It basically becomes "last writer wins" in a distributed system. You at least get consistency. And NoSQL gives you the option of building something better like e.g. Riak does. > The downgrading to NoSql argument makes little sense to me as well. If your frontend is sqlite then you can have Postgres, or MariaDb in your backend. You need to account for having different queries on the front and the back, but the backend is presumably also operating under very different constraints than the frontend. This is kind of the same as the log replication problem - you want the protocol for what's replicated from frontend to backend to be something simple that you could implement on top of approximately any backend datastore. "SQL queries" aren't that, whereas a simple K/V store protocol could be (e.g. MySQL actually implements the memcached query protocol, or did at one point - that would be very hard to do the other way around). > I'd actually make an argument that NoSql databases are a type of relational database heavily optimized towards not needing to do joins Um, WTF? Can you flesh this out at all?

sourcesmith 5y ago
> > Their section on reliable replications can be resolved by keeping a log of the changes made to the database rather than a log of the queries. > You'd need to define a representation of that log, which ends up being equivalent to writing a NoSQL datastore. I think this covered by relational DBs redo/rollback/write-ahead log; whatever you want to call it.
- lmm 5y ago
  Right, but they generally keep it as a hidden implementation detail, and when you do get access to it it's not in any standardised way. For a lot of use cases you're better off unbundling it and newer datastores tend to be better at that.
er4hn 5y ago
> Um, WTF? Can you flesh this out at all? Ha, sure. Let's say that you have a very-basic NoSql schema for a chat app: chat_app/person/<person id>/msgs/<chat documents> so for each <person id> you stick chat messages in the <chat documents> separated by date or something. Conversations with different people end up in different <person id> paths. Chats have metadata like timestamps in addition to the message content You can also represent this as a SQL table with something like: Person_Id (uint) | Timestamp (timestamp) | Message (string) Now the classic argument for NoSql is that NoSql is the best representation for this since you don't really care about doing complicated queries over the chats. You just want to be able to index in and get a persons chats. Maybe sort by the most recent chats so the <chat documents> have the timestamp in the key. That's what I mean. Where I get frustrated with NoSql is that you inevitably end up needing to redo a lot of SQL logic anyways. A basic example is mapping the person_id -> visible name. The NoSql way is to have a meta file under <person id>, like chat_app/person/<person id>/<meta file> where you would have the user name under there. Then you can look up easily what to display when showing chats for <person id>. But then you add group chats. Now you need to also have a group chat section and you need to go back to the person_id path to resolve the visible names. Having a relational table with: Person Id | Visible Name seems like a more elegant solution to me. Once you need to start gathering stats on the data as well, relational tables become more easy to gather information on how often you are talking to various people or lengths of conversations with people. Most of the functionality for statistical gathering and other use cases can be replicated in NoSql but it involves extra effort that SQL just gives you.