Another potential problem is people being born or dying after you’re collecting your data and running your job. #3, that’s a good one. I started a long-running

Another potential problem is people being born or dying after you’re collecting your data and running your job. #3, that’s a good one. I started a long-running job like this and it failed in the middle after sending money to N people, and Airflow or L1 support just mashed restart. Now the first N people who got a dollar before the job failed each get 2 dollars (or 3 if mash retry again). Now you ran out of money, or you treated the last M people unfairly. (That’s how I learned “idempotent”.) One solution is to keep a log of the people you sent the money to and do a left outer join with the people_to_send_dollar_to table when the job starts. Then you filter out anybody in the people_we_already_sent_dollar_to table in your query so your job never iterates over them again. But now it’s worth thinking about if you succeeded sending somebody the dollar but they failed receiving it (it got stolen out of their mailbox or it fell out of the delivery truck). Do we need people to tell us when they received the dollar and resend it periodically until we get the confirmation message? What if we never get the confirmation message? Is this the 2 generals problem? What if some people never confirm so they can keep getting dollars in the mail? Is the answer cryptocurrency?