4 ms·

Years ago when I was a green programmer filled with p*ss and vinegar - I had to write a system to send a few million emails every day. That’s when I came acros

Years ago when I was a green programmer filled with p*ss and vinegar - I had to write a system to send a few million emails every day. That’s when I came across Qmail and studied it’s design. Qmail was designed so well that it literally absolutely would not send an email twice. But my scheduler did. To send a few hundred thousand emails more than once is not good is not good for your reputation lol. It taught me a valuable lesson about Idempotency from an impressionable stage of my career and now, with creating workers running under Sidekiq, Oban and other job schedulers that run very frequently, this has become quite useful. At one point I used to lean on Redis to “help” me be more idempotent. But with the high throughput of some systems I was working on, even redis with its atomic nature of put and get and handy dandy sets and lists, would have race conditions. Now I hardly touch redis because my skills in making sure things get processed just once have improved greatly. And I can just use PostgreSQL and my code. Being Idempotent from the beginning is now the center of my life.

Rule35 6y ago
I feel that all of the protections let us chase the problems into smaller areas. Rust's unsafe doesn't eliminate unsafe code, it just means you put it in a small auditable area. Similarly, some part of the system remains imperative. The network card at least, will always resend a packet. The goal is to pass around idempotent messages except for the very leaves. For instance, instead of an endpoint 'email(customer, data)' you might have 'email_or_report_on_send_status(customer, data)' and the later endpoint would check the cache for (customer, data) and merely report the previous results if it found them. I agree though, this stuff used to keep me up at night and eventually I've grown more natural about not mutating things unless I mean it. (This phrase sounds like a comic-book villain.)

Rule35 6y ago

I feel that all of the protections let us chase the problems into smaller areas. Rust's unsafe doesn't eliminate unsafe code, it just means you put it in a small auditable area. Similarly, some part of the system remains imperative. The network card at least, will always resend a packet. The goal is to pass around idempotent messages except for the very leaves. For instance, instead of an endpoint 'email(customer, data)' you might have 'email_or_report_on_send_status(customer, data)' and the later endpoint would check the cache for (customer, data) and merely report the previous results if it found them. I agree though, this stuff used to keep me up at night and eventually I've grown more natural about not mutating things unless I mean it. (This phrase sounds like a comic-book villain.)