6 ms·

This post never actually delivers on its claims. What about CAP or FLP does this resolve in a new way? The author, additionally, seems to not be well-versed i

This post never actually delivers on its claims. What about CAP or FLP does this resolve in a new way? The author, additionally, seems to not be well-versed in the existing distributed database literature. Essentially they have added a queue in-front of all database operations. That queue is totally ordered, so you can't have consistent issues. That queue apparently does all operations via a two-phase commit (without calling it that and being unclear on semantics, so I am not 100% sure). Ok. So you've moved the question of availability and consistency to your queue. Is that a single queue? If so, then it's a liability. Why not just use a single database at that point. Is it multiple queues? Then you still have a consensus problem to solve. Are you using two-phase commit? Well, now your availability is seriously impacted. There is nothing there. It's a shame because there are models that are much more interesting that provide a useful mental model for this. PACELC is my favorite. The essence is that when everything is going fine, your decision is around latency and consistency. When there is a partition, your question is between availability and consistency.

andras_gerlits 3y ago
PACELC is an extension of CAP, so it suffers from the same problem of trying to apply a universal clock to the whole system. If you do that, you will suffer these limitations. With client-centric consistency, you can work around these problems and global order "unfolds" in a "just in time" manner. The consistency-levels can go all the way to SNAPSHOT.
- sausagefeet 3y ago
  CAP says nothing about a "universal clock over the whole system". CAP is about the decision that has to be made in some unit of the system, it could be the whole system or it could be a bit, at the point of an operation. It's physics, there is no way around it. You can make different decisions on the semantics your system needs, but if you have two nodes that physically cannot communicate but need to be consistent for a client to move forward, the client cannot move forward. Full stop. Could you please show a failure mode that this system can handle that CAP says is not possible?
  - andras_gerlits 3y ago
    Sure. Here: https://medium.com/p/5e397cb12e63#04a5 https://medium.com/p/5e397cb12e63#04a5
    - sausagefeet 3y ago
      I'm not sure if you gave the wrong link or not but this link doesn't describe any failure modes and how OmniLedger allegedly resolves them.
      - andras_gerlits 3y ago
        This section discusses some failure modes (the first one is about the failure of a specific node): https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and-fulfil-the-promise-of-microservices-5e397cb12e63#373c https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and... This section discusses latency-spike mitigation (which is how Brewer defines CAP colloquially): https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and-fulfil-the-promise-of-microservices-5e397cb12e63#7df1 https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and... This section dissects the problem when trying to apply CAP to non-linearizable systems like SQL: https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and-fulfil-the-promise-of-microservices-5e397cb12e63#04a5 https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and... Again, if you're not happy with the lack of scientific rigour in this technical article (ie: not science-paper), you can connect the dots in this one: https://www.researchgate.net/publication/359578461_Continuous_Integration_of_Data_Histories_into_Consistent_Namespaces https://www.researchgate.net/publication/359578461_Continuou...
        sausagefeet 3y ago
        I've read all this and I saw no description of failure modes and operationally how they are resolved. "If a node disappears just replace it with a new one". Ok, how?
        mrkeen 3y ago
- tsimionescu 3y ago
  Ultimately the problem we care about is very simple, and there is no way to solve it, despite your claims. Say we have a database of customers, with two replicas - one in the USA, one in Europe. Say a customer in Europe wants to update their shipping address. We ship products every month to this customer from the USA to their current shipping address, on the 10th of that month. The customer is updating their address on the 9th at 10 AM PST. However, we are in the midst of a massive network partition that started on the 9th at 02 AM PST and is expected to last until the 11th. Do we perform the update in the European replica and tell the customer it succeeded (giving up consistency)? If we do, the US side of the business will ship the products to the old address, even though the update happened a full day earlier. Alternatively, do we tell the customer the update failed (giving up availability)? If we do, then the customer can't even let the European side of the business know of their new address. This is the CAP theorem in a nutshell, and it is obviously inescapable. It doesn't require appeals to immediacy that are anywhere close to the bounds of relativity. And while 3-day long network partitions are quite rare, partitions that last for many hours are not.
  - andras_gerlits 3y ago
    Any modification to existing data must be "haggled for" somewhere, you're right about that. When you say "partition event", what is being partitioned here? A specific communication-link between two nodes. It's entirely possible (no, extremely likely) that your EU node would have access to a different US node, but not the one that's having the partition-event right now. That's exactly the point of this section in the essay: https://medium.com/p/5e397cb12e63#7df1 https://medium.com/p/5e397cb12e63#7df1 Networks will have latency-spikes, but if you can stream time-information the same way you can stream others, you can use redundancies to mitigate the disruption of any single channel.
    - mrkeen 3y ago
      > It's entirely possible (no, extremely likely) that your EU node would have access to a different US node CAP theorem beaten by declaring P to be unlikely.
benreesman 3y ago
Near as I can tell this system/paper/post thing is a hybrid between a database proxy and an (allegedly) strictly serializable log tailer sitting squarely at “CP” in conventional nomenclature, e.g. https://mark-burgess-oslo-mb.medium.com/using-promise-theory-to-solve-the-distributed-consensus-problem-4cc2116f24e1 https://mark-burgess-oslo-mb.medium.com/using-promise-theory... I haven’t done heavyweight distributed systems theory in a few years, and so who knows, maybe @aphyr shows up and Jepsen-stamps this thing, but after reading the paper and going over the commercial website personally I’m a pass. I’m one of about a zillion people on HN who have dealt in petabyte and trillion event days on logs as mundane subsystems for years on end and it’s just almost always: “there’s the lock, didn’t see it right away because of the bespoke nomenclature”. History is replete with misunderstood geniuses who weren’t appreciated because they were so far ahead of everyone else (Galois is my favorite example personally), and we might all end up eating crow over it, but this has a wink and a grin vibe that I’m going to regard as great satire rather than hyperventilating. I suspect the object of the satire would be blockchain.

andras_gerlits 3y ago

PACELC is an extension of CAP, so it suffers from the same problem of trying to apply a universal clock to the whole system. If you do that, you will suffer these limitations. With client-centric consistency, you can work around these problems and global order "unfolds" in a "just in time" manner. The consistency-levels can go all the way to SNAPSHOT.

sausagefeet 3y ago
CAP says nothing about a "universal clock over the whole system". CAP is about the decision that has to be made in some unit of the system, it could be the whole system or it could be a bit, at the point of an operation. It's physics, there is no way around it. You can make different decisions on the semantics your system needs, but if you have two nodes that physically cannot communicate but need to be consistent for a client to move forward, the client cannot move forward. Full stop. Could you please show a failure mode that this system can handle that CAP says is not possible?
- andras_gerlits 3y ago
  Sure. Here: https://medium.com/p/5e397cb12e63#04a5 https://medium.com/p/5e397cb12e63#04a5
  - sausagefeet 3y ago
    I'm not sure if you gave the wrong link or not but this link doesn't describe any failure modes and how OmniLedger allegedly resolves them.
    - andras_gerlits 3y ago
      This section discusses some failure modes (the first one is about the failure of a specific node): https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and-fulfil-the-promise-of-microservices-5e397cb12e63#373c https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and... This section discusses latency-spike mitigation (which is how Brewer defines CAP colloquially): https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and-fulfil-the-promise-of-microservices-5e397cb12e63#7df1 https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and... This section dissects the problem when trying to apply CAP to non-linearizable systems like SQL: https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and-fulfil-the-promise-of-microservices-5e397cb12e63#04a5 https://itnext.io/how-simple-can-scale-your-sql-beat-cap-and... Again, if you're not happy with the lack of scientific rigour in this technical article (ie: not science-paper), you can connect the dots in this one: https://www.researchgate.net/publication/359578461_Continuous_Integration_of_Data_Histories_into_Consistent_Namespaces https://www.researchgate.net/publication/359578461_Continuou...
      - sausagefeet 3y ago
        I've read all this and I saw no description of failure modes and operationally how they are resolved. "If a node disappears just replace it with a new one". Ok, how?
        mrkeen 3y ago
tsimionescu 3y ago
Ultimately the problem we care about is very simple, and there is no way to solve it, despite your claims. Say we have a database of customers, with two replicas - one in the USA, one in Europe. Say a customer in Europe wants to update their shipping address. We ship products every month to this customer from the USA to their current shipping address, on the 10th of that month. The customer is updating their address on the 9th at 10 AM PST. However, we are in the midst of a massive network partition that started on the 9th at 02 AM PST and is expected to last until the 11th. Do we perform the update in the European replica and tell the customer it succeeded (giving up consistency)? If we do, the US side of the business will ship the products to the old address, even though the update happened a full day earlier. Alternatively, do we tell the customer the update failed (giving up availability)? If we do, then the customer can't even let the European side of the business know of their new address. This is the CAP theorem in a nutshell, and it is obviously inescapable. It doesn't require appeals to immediacy that are anywhere close to the bounds of relativity. And while 3-day long network partitions are quite rare, partitions that last for many hours are not.
- andras_gerlits 3y ago
  Any modification to existing data must be "haggled for" somewhere, you're right about that. When you say "partition event", what is being partitioned here? A specific communication-link between two nodes. It's entirely possible (no, extremely likely) that your EU node would have access to a different US node, but not the one that's having the partition-event right now. That's exactly the point of this section in the essay: https://medium.com/p/5e397cb12e63#7df1 https://medium.com/p/5e397cb12e63#7df1 Networks will have latency-spikes, but if you can stream time-information the same way you can stream others, you can use redundancies to mitigate the disruption of any single channel.
  - mrkeen 3y ago
    > It's entirely possible (no, extremely likely) that your EU node would have access to a different US node CAP theorem beaten by declaring P to be unlikely.

benreesman 3y ago

Near as I can tell this system/paper/post thing is a hybrid between a database proxy and an (allegedly) strictly serializable log tailer sitting squarely at “CP” in conventional nomenclature, e.g. https://mark-burgess-oslo-mb.medium.com/using-promise-theory-to-solve-the-distributed-consensus-problem-4cc2116f24e1 https://mark-burgess-oslo-mb.medium.com/using-promise-theory... I haven’t done heavyweight distributed systems theory in a few years, and so who knows, maybe @aphyr shows up and Jepsen-stamps this thing, but after reading the paper and going over the commercial website personally I’m a pass. I’m one of about a zillion people on HN who have dealt in petabyte and trillion event days on logs as mundane subsystems for years on end and it’s just almost always: “there’s the lock, didn’t see it right away because of the bespoke nomenclature”. History is replete with misunderstood geniuses who weren’t appreciated because they were so far ahead of everyone else (Galois is my favorite example personally), and we might all end up eating crow over it, but this has a wink and a grin vibe that I’m going to regard as great satire rather than hyperventilating. I suspect the object of the satire would be blockchain.