4 ms·

I know it's just an introduction so perhaps it shouldn't be taken too literally, but the sequence diagram of how to make an order idempotent feels like instruct

I know it's just an introduction so perhaps it shouldn't be taken too literally, but the sequence diagram of how to make an order idempotent feels like instructions on how to create race conditions. You can't check for existence to guard against duplicate creation without either locking or another approach to making the Exists-or-Create step atomic. If you were to put in the sequence as described, you'd have something which is idempotent most of the time, until it isn't, when API requests come in and get forwarded to the endpoint before the caching server can cache the first creation.

tehlike 3y ago
Presumably unique constraints could also solve for this, in a properly written transaction.
- xnorswap 3y ago
  That depends how your API is structured, and again comes back to the atomicity and idempotency. If your API is for example: "POST /orders/create {products:[{bananas:1}], shippingAddress:{},...}", then it's not clear how you would check uniqueness, because you'd want the same person to make the same order at different times by design. To prevent double-submit, and make it idempotent, you need to add extra information. One way to address this is generating transaction/sequence/correlation IDs at the very start of the process. So for example when you first request the order form, a transaction / correlation ID (called in the article an Idempotency key) is generated. Then you would include that transaction ID in the order, and could prevent duplicate use of an order ID at multiple levels of the system including the database. One design is to have a single-source-of-truth, and have at the (internal) API layer a state machine for a transaction. Once an order has been processed then it's impossible to make another order because there's no "make order" step from the state that an order is in having been processed. Again you need some kind of correlation ID to track which state machine needs loading. That however is less scalable than other solutions including things like event-sourced architecture, which would also prevent duplicate orders through eventual consistency although there's less (or no) guarantee in many such systems that the order submitted "first" would win the race and not be superseded by the second. Such behaviour is fine if documented.
projektfu 3y ago
As I understand it, it's not checking for the existence of the resource but for the existence of the request. Assuming the requests are unique to a user/client and the user has a sequential conversation with the API server, the server would process the received requests in order and would reject the duplicate. However, if the requests of the same key (duplicates) are sent to any of several API servers, you have a problem that must be resolved as you mentioned. In the first case, if the API server dies, the client should establish a new connection and get new idempotency keys.
- xnorswap 3y ago
  Even on the same server there's still a race condition if checking the key has been used and marking the key as used aren't atomic if more than one request can be handled at once.

tehlike 3y ago

Presumably unique constraints could also solve for this, in a properly written transaction.

xnorswap 3y ago
That depends how your API is structured, and again comes back to the atomicity and idempotency. If your API is for example: "POST /orders/create {products:[{bananas:1}], shippingAddress:{},...}", then it's not clear how you would check uniqueness, because you'd want the same person to make the same order at different times by design. To prevent double-submit, and make it idempotent, you need to add extra information. One way to address this is generating transaction/sequence/correlation IDs at the very start of the process. So for example when you first request the order form, a transaction / correlation ID (called in the article an Idempotency key) is generated. Then you would include that transaction ID in the order, and could prevent duplicate use of an order ID at multiple levels of the system including the database. One design is to have a single-source-of-truth, and have at the (internal) API layer a state machine for a transaction. Once an order has been processed then it's impossible to make another order because there's no "make order" step from the state that an order is in having been processed. Again you need some kind of correlation ID to track which state machine needs loading. That however is less scalable than other solutions including things like event-sourced architecture, which would also prevent duplicate orders through eventual consistency although there's less (or no) guarantee in many such systems that the order submitted "first" would win the race and not be superseded by the second. Such behaviour is fine if documented.

projektfu 3y ago

As I understand it, it's not checking for the existence of the resource but for the existence of the request. Assuming the requests are unique to a user/client and the user has a sequential conversation with the API server, the server would process the received requests in order and would reject the duplicate. However, if the requests of the same key (duplicates) are sent to any of several API servers, you have a problem that must be resolved as you mentioned. In the first case, if the API server dies, the client should establish a new connection and get new idempotency keys.

xnorswap 3y ago
Even on the same server there's still a race condition if checking the key has been used and marking the key as used aren't atomic if more than one request can be handled at once.