5 ms·

The write last, read first rule

102 points by vismit2000 11mo ago

esperent 11mo ago
[flagged]
- another_twist 11mo ago
  You would be missing out. Its a very well written blog post from a project that is solving a really important problem.
  - jorangreef 11mo ago
    Thank you for defending (handcrafted) art! :)
- MichaelGlass 11mo ago
  from https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html: "Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting."
  - evertedsphere 11mo ago
    As a general rule when it comes to this pattern of replying to "this is AI generated" with that link: the people that write these posts often read HN and attach a certain amount of importance to the opinions presented here, and it's important that people express their opinions about trends in how the majority of technical writing submitted to this website is either generated or presented, before they become well and truly entrenched as being problems "too common to be interesting". There's a difference between criticisms of the content or the reader's ability to view it and complaints about "tangential annoyances" surrounding it.
- s-lambert 11mo ago
  But it's not an AI generated image, it has the artist's signature in the bottom right corner. Tigerbeetle has a lot of custom artwork designed for their stuff that's high quality. I don't even see why you would think it was AI generated even without the artist signature, it's using the Tigerbeetle mascot and it looks drawn for the particular theme the blog is talking about.
  - pontsun 11mo ago
    Yeah, it's clearly by Joy Machs.
  - jorangreef 11mo ago
    Thanks, appreciate your kind words! Yes, everything we do at TigerBeetle is handcrafted (by humans) for quality, including the art. Joy Machs on our team is an artist, and as you spotted, he signs all his work—I always ask him to! :) Joy also did much of the Zig artwork, which is how we met. You can see more of his work here: https://sim.tigerbeetle.com https://sim.tigerbeetle.com (and see if you can find or "not find" an Easter egg in our docs).
  - deleted 11mo ago
    [deleted]
jorangreef 11mo ago
Joran from TigerBeetle! Happy to answer any questions. And thanks to Dominik Tornow of Resonate for writing this up as a guest post! It was a little rule we had coined, to help people remember how to preserve consistency across different DBMS's, and I think Dominik gave (beautiful) voice to it.
- xmcqdpt2 11mo ago
  IMO the example about checkpointing is not that demonstrative because creating entities is easy to make idempotent. What about transactions? Do they need idempotency and if so, how does it work?
- layer8 11mo ago
  I concur with https://news.ycombinator.com/item?id=45886850 https://news.ycombinator.com/item?id=45886850, the article doesn’t actually spell out what the “Write Last, Read First” rule is supposed to say. The name of the rule also isn’t great, because it leaves open what to write to/read from, and doesn’t even suggest that there might be several systems to write to/read from, which however is essential to the rule.
  - jorangreef 11mo ago
    The what here is the “system of record”. There might also be no other systems to write to/read from—and that's fine. The important thing is the order. Hopefully "Write Last, Read First" is unforgettable in that respect! Or, can you suggest something pithier?
    - layer8 11mo ago
      The article mixes two things: (1) That you should (depending on requirements) have one system of records while the other systems are merely systems of reference, and (2) that in that architecture, when (*) you write to both systems or read from both systems for the same logical record, then in the writing case you have to write to the system of records last, and in the reading case you have to read from the system of records first. The article however doesn’t make it clear that the titular rule is specifically (2), and that it is conditioned on (1) and on (*). I have no suggestion for a better name off the top of my head. The issue I see is that you already have to know well its context and when it applies, also in order to not misremember it as “Write First, Read Last”, and to not mistake it as LIFO, or to relate it to a read-modify-write scenario in which you naturally would read first and write last anyway, though in a different sense. You see how the name is confusing?
VeritySage07 11mo ago
[dead]
lintfordpickle 11mo ago
I found the article interesting, but I don't think I understand what is meant by 'Write Last, Read first' rule - even after reading it a few times. It seems to be too ambiguous a statement to be helpful. Under the section 'Order of Operations': > "Since the system of reference doesn’t determine existence, we can safely write to it first without committing anything. [...]" Then the next paragraph > "This principle—Write Last, Read First—ensures that we maintain application level consistency." What I think it means is, 'writing-last to the system-of-record' and 'a read-first from the system of record' yields authoritative results, but I don't get that just from the title. Is my understanding correct?
- withinboredom 11mo ago
  I think they just reinvented 2-phase commit? I'm not sure either tbh.
  - layer8 11mo ago
    No, in two-phase commit all target systems perform two phases (staging the commit, which may fail, and then actually committing it in a failsafe way), which isn’t the case here.
    - withinboredom 11mo ago
      You’re thinking too concretely. What do you think you’re doing before you write “to the source of truth”? You’re staging the commit. And then you commit. 2pc isn’t literally about transactions like you are probably thinking of in a database, its an abstract “atomic change” or “unit of work” that may or may not involve a database or a database transaction. You can do 2pc with just normal files, or APIs, or whatever.
      - layer8 11mo ago
        I disagree. The definition of a two-phase commit protocol is that you have a number of participants in a transaction, and the first phase consists of asking each participant if they can commit, and if or when all participants affirm positively, then the second phase consists of telling all participants to perform the commit. Let’s not dilute well-established terms. The procedure in the article is not a two-phase commit, because changes are committed to the system of reference regardless of whether the subsequent commit to the system of record succeeds or not. In addition, half of the rule in the article is about the ordering of reads, which two-phase commit isn’t concerned about at all.
tclancy 11mo ago
I was hoping this was going to be advice for being a good citizen of the ‘net, but this week seems to be Brought to You by Tiger Beetle, so here we are.
aaroninsf 11mo ago
One of the real pleasures of HN is that it provides for a clockwork-like enthusiastic rediscovery and re-articulation of basic computer science and systems concepts. It's almost enough to make me believe in the independent existence of Platonic truths. Almost.
- jorangreef 11mo ago
  Of course, there’s nothing new except shining a spotlight (and coining the rule!).
bigcojosh 11mo ago
This was a lovely piece. I'm only surprised by paucity of interesting comments here. If Joran or anybody sees this late comment: I'd appreciate any other references to the approach. Surely it wasn't invented by TigerBeetle?
- dtornow 11mo ago
  Hey thanks for the kind words, I am happy you liked the article. You are correct, we did not come up with the rule (but Joran did come up with the name) but drew from the rich history of transaction management systems. A wonderful paper that I highly recommend is A Transaction Model from Jim Gray, that illustrates like no other than transactions are all about coordinating actions and when to externalize their results to a user. The mental model presented in this paper frequently informs my thinking when designing systems https://jimgray.azurewebsites.net/papers/a%20transaction%20model%20rj%202895.pdf https://jimgray.azurewebsites.net/papers/a%20transaction%20m...

102 points by vismit2000 11mo ago

esperent 11mo ago

[flagged]

another_twist 11mo ago
You would be missing out. Its a very well written blog post from a project that is solving a really important problem.
- jorangreef 11mo ago
  Thank you for defending (handcrafted) art! :)
MichaelGlass 11mo ago
from https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html: "Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting."
- evertedsphere 11mo ago
  As a general rule when it comes to this pattern of replying to "this is AI generated" with that link: the people that write these posts often read HN and attach a certain amount of importance to the opinions presented here, and it's important that people express their opinions about trends in how the majority of technical writing submitted to this website is either generated or presented, before they become well and truly entrenched as being problems "too common to be interesting". There's a difference between criticisms of the content or the reader's ability to view it and complaints about "tangential annoyances" surrounding it.
s-lambert 11mo ago
But it's not an AI generated image, it has the artist's signature in the bottom right corner. Tigerbeetle has a lot of custom artwork designed for their stuff that's high quality. I don't even see why you would think it was AI generated even without the artist signature, it's using the Tigerbeetle mascot and it looks drawn for the particular theme the blog is talking about.
- pontsun 11mo ago
  Yeah, it's clearly by Joy Machs.
- jorangreef 11mo ago
  Thanks, appreciate your kind words! Yes, everything we do at TigerBeetle is handcrafted (by humans) for quality, including the art. Joy Machs on our team is an artist, and as you spotted, he signs all his work—I always ask him to! :) Joy also did much of the Zig artwork, which is how we met. You can see more of his work here: https://sim.tigerbeetle.com https://sim.tigerbeetle.com (and see if you can find or "not find" an Easter egg in our docs).
- deleted 11mo ago
  [deleted]

jorangreef 11mo ago

Joran from TigerBeetle! Happy to answer any questions. And thanks to Dominik Tornow of Resonate for writing this up as a guest post! It was a little rule we had coined, to help people remember how to preserve consistency across different DBMS's, and I think Dominik gave (beautiful) voice to it.

xmcqdpt2 11mo ago
IMO the example about checkpointing is not that demonstrative because creating entities is easy to make idempotent. What about transactions? Do they need idempotency and if so, how does it work?
layer8 11mo ago
I concur with https://news.ycombinator.com/item?id=45886850 https://news.ycombinator.com/item?id=45886850, the article doesn’t actually spell out what the “Write Last, Read First” rule is supposed to say. The name of the rule also isn’t great, because it leaves open what to write to/read from, and doesn’t even suggest that there might be several systems to write to/read from, which however is essential to the rule.
- jorangreef 11mo ago
  The what here is the “system of record”. There might also be no other systems to write to/read from—and that's fine. The important thing is the order. Hopefully "Write Last, Read First" is unforgettable in that respect! Or, can you suggest something pithier?
  - layer8 11mo ago
    The article mixes two things: (1) That you should (depending on requirements) have one system of records while the other systems are merely systems of reference, and (2) that in that architecture, when (*) you write to both systems or read from both systems for the same logical record, then in the writing case you have to write to the system of records last, and in the reading case you have to read from the system of records first. The article however doesn’t make it clear that the titular rule is specifically (2), and that it is conditioned on (1) and on (*). I have no suggestion for a better name off the top of my head. The issue I see is that you already have to know well its context and when it applies, also in order to not misremember it as “Write First, Read Last”, and to not mistake it as LIFO, or to relate it to a read-modify-write scenario in which you naturally would read first and write last anyway, though in a different sense. You see how the name is confusing?

VeritySage07 11mo ago

[dead]

lintfordpickle 11mo ago

I found the article interesting, but I don't think I understand what is meant by 'Write Last, Read first' rule - even after reading it a few times. It seems to be too ambiguous a statement to be helpful. Under the section 'Order of Operations': > "Since the system of reference doesn’t determine existence, we can safely write to it first without committing anything. [...]" Then the next paragraph > "This principle—Write Last, Read First—ensures that we maintain application level consistency." What I think it means is, 'writing-last to the system-of-record' and 'a read-first from the system of record' yields authoritative results, but I don't get that just from the title. Is my understanding correct?

withinboredom 11mo ago
I think they just reinvented 2-phase commit? I'm not sure either tbh.
- layer8 11mo ago
  No, in two-phase commit all target systems perform two phases (staging the commit, which may fail, and then actually committing it in a failsafe way), which isn’t the case here.
  - withinboredom 11mo ago
    You’re thinking too concretely. What do you think you’re doing before you write “to the source of truth”? You’re staging the commit. And then you commit. 2pc isn’t literally about transactions like you are probably thinking of in a database, its an abstract “atomic change” or “unit of work” that may or may not involve a database or a database transaction. You can do 2pc with just normal files, or APIs, or whatever.
    - layer8 11mo ago
      I disagree. The definition of a two-phase commit protocol is that you have a number of participants in a transaction, and the first phase consists of asking each participant if they can commit, and if or when all participants affirm positively, then the second phase consists of telling all participants to perform the commit. Let’s not dilute well-established terms. The procedure in the article is not a two-phase commit, because changes are committed to the system of reference regardless of whether the subsequent commit to the system of record succeeds or not. In addition, half of the rule in the article is about the ordering of reads, which two-phase commit isn’t concerned about at all.

tclancy 11mo ago

I was hoping this was going to be advice for being a good citizen of the ‘net, but this week seems to be Brought to You by Tiger Beetle, so here we are.

aaroninsf 11mo ago

One of the real pleasures of HN is that it provides for a clockwork-like enthusiastic rediscovery and re-articulation of basic computer science and systems concepts. It's almost enough to make me believe in the independent existence of Platonic truths. Almost.

jorangreef 11mo ago
Of course, there’s nothing new except shining a spotlight (and coining the rule!).

bigcojosh 11mo ago

This was a lovely piece. I'm only surprised by paucity of interesting comments here. If Joran or anybody sees this late comment: I'd appreciate any other references to the approach. Surely it wasn't invented by TigerBeetle?

dtornow 11mo ago
Hey thanks for the kind words, I am happy you liked the article. You are correct, we did not come up with the rule (but Joran did come up with the name) but drew from the rich history of transaction management systems. A wonderful paper that I highly recommend is A Transaction Model from Jim Gray, that illustrates like no other than transactions are all about coordinating actions and when to externalize their results to a user. The mental model presented in this paper frequently informs my thinking when designing systems https://jimgray.azurewebsites.net/papers/a%20transaction%20model%20rj%202895.pdf https://jimgray.azurewebsites.net/papers/a%20transaction%20m...