3 ms·

If you have external side-effects, and your transaction fails, undo whatever you did in the external system. Don't just roll-back your database, but forget to "

If you have external side-effects, and your transaction fails, undo whatever you did in the external system. Don't just roll-back your database, but forget to "request your money back" from that system. And remember, don't use transactions when you cannot compensate the failure. For example, if you were to transfer money to an external bank, you probably can't just "undo" the sending of money. Your bank is on the hook for that money, so if, for example, the account you withdrew money from would now be negative ... you probably want it to still be negative if a failure happens ... instead of staying zero after rolling back the transaction. Otherwise, that other bank is going to come asking for that money you said you sent them.

JoshGlazebrook 3y ago
This is a major pain point with dealing with Stripe. If you want to write a record in your local db (to record a cancellation), then call the stripe api to say cancel a subscription, if the api call fails, you can rollback your db changes by failing the transaction. If the api call succeeds, but your transaction fails to write to the db and rollsback, then what? You then have to rely on the Stripe webhooks to "sync" your local db state with what stripe has, oh but don't forget stripe webhooks don't have any guaranteed order in which they are delivered, so just ignore all of the event data, you have to fetch the current state of the stripe object your event is associated with and make sure the handling of these events are idempotent. It's not wise to rely solely on Stripe being the source of truth, but there is no way to really implement 2 phase commit with external services like this. Maybe you fire off an event into a pubsub or queue to handle the actual writing to your db, but what if the webhook gets to your service first? Shit is hard.
- tynorf 3y ago
  You might be interested in this: https://brandur.org/idempotency-keys https://brandur.org/idempotency-keys
- yencabulator 3y ago
  You record a want-to-cancel state, and then only after Stripe acknowledges the cancellation you transition from want-to-cancel to cancelled. State machines.
- withinboredom 3y ago
  Any sync needs two phases: - active state transitions (webhooks) - reconciliation (api) You can do reconciliation without active state transitions, but it usually takes awhile (slow and scheduled). However, it’s a requirement with webhooks because you don’t know if they are in-order or delayed or just broken. Thus, you need to reconcile your state every so often.
maxbond 3y ago
I'd caution against holding a transaction for the duration of an API call to a foreign system. API calls can take a very long time, especially if you are retrying on timeouts. Long lived transactions are dangerous, because it prevents you from releasing locks, which interferes with concurrent transactions and with vacuuming. I've seen it bring down production before. Consider for example the following scenario; you are experiencing higher than normal load, leading you to send a lot of requests to this other system. Their scaling is flawed, and your traffic actually brings them down. This leads to timeouts and long transaction times. Now you are holding lots of locks for a very long time, and you database is under a heavy load - you're now in danger of overloading your database and going down. I'd suggest commiting a record representing your intention to integrate with this other system, executing your API call, and then updating your state (assuming the call is successful). (For the example above, circuit breakers would also be an important mitigation.)

JoshGlazebrook 3y ago

This is a major pain point with dealing with Stripe. If you want to write a record in your local db (to record a cancellation), then call the stripe api to say cancel a subscription, if the api call fails, you can rollback your db changes by failing the transaction. If the api call succeeds, but your transaction fails to write to the db and rollsback, then what? You then have to rely on the Stripe webhooks to "sync" your local db state with what stripe has, oh but don't forget stripe webhooks don't have any guaranteed order in which they are delivered, so just ignore all of the event data, you have to fetch the current state of the stripe object your event is associated with and make sure the handling of these events are idempotent. It's not wise to rely solely on Stripe being the source of truth, but there is no way to really implement 2 phase commit with external services like this. Maybe you fire off an event into a pubsub or queue to handle the actual writing to your db, but what if the webhook gets to your service first? Shit is hard.

tynorf 3y ago
You might be interested in this: https://brandur.org/idempotency-keys https://brandur.org/idempotency-keys
yencabulator 3y ago
You record a want-to-cancel state, and then only after Stripe acknowledges the cancellation you transition from want-to-cancel to cancelled. State machines.
withinboredom 3y ago
Any sync needs two phases: - active state transitions (webhooks) - reconciliation (api) You can do reconciliation without active state transitions, but it usually takes awhile (slow and scheduled). However, it’s a requirement with webhooks because you don’t know if they are in-order or delayed or just broken. Thus, you need to reconcile your state every so often.

maxbond 3y ago

I'd caution against holding a transaction for the duration of an API call to a foreign system. API calls can take a very long time, especially if you are retrying on timeouts. Long lived transactions are dangerous, because it prevents you from releasing locks, which interferes with concurrent transactions and with vacuuming. I've seen it bring down production before. Consider for example the following scenario; you are experiencing higher than normal load, leading you to send a lot of requests to this other system. Their scaling is flawed, and your traffic actually brings them down. This leads to timeouts and long transaction times. Now you are holding lots of locks for a very long time, and you database is under a heavy load - you're now in danger of overloading your database and going down. I'd suggest commiting a record representing your intention to integrate with this other system, executing your API call, and then updating your state (assuming the call is successful). (For the example above, circuit breakers would also be an important mitigation.)