6 ms·

What if the original request is still being processed when the retry comes in? That doesn't fall into either of your categories: the request isn't successful, b

by pdonis 5mo ago

What if the original request is still being processed when the retry comes in? That doesn't fall into either of your categories: the request isn't successful, but it hasn't failed either.

stickfigure 5mo ago
That's usually solved with traditional database transactions. Even if you have a complex long-running multistep orchestration problem, you can break it down into simpler transactions. Eg you could start with a "lock the resources" txn. But 99% of these conversations around idempotence are simple POST operations like "create order" that regular old database concurrency management handles just fine.
- pdonis 5mo ago
  > That's usually solved with traditional database transactions. That doesn't answer my question. What response do you return to the client in the case I described?
  - matsemann 5mo ago
    The lock would normally make the second request wait, aka not return a response, until the first one is done. Then it sees it's a duplicate and returns that. Or it times out and returns an error. Then the client hopefully have some exponential back off strategy, so the third attempt doesn't suffer the same fate.
    - MattDaEskimo 5mo ago
      If a transaction is locked then subsequent requests would return a 409, ideally with an error message indicating that it's currently being processed.
  - stickfigure 5mo ago
    This is just normal concurrent programming? If two requests come in for the same idempotency key/customer reference id, only one will succeed. Use standard database transaction isolation. So one will complete with 200, one will complete with 409. It doesn't matter which. That said, there's something odd about the way you phrased this question. If the original request hasn't gotten a response yet, why is it sending a retry? What you're asking is more general: What happens when two conflicting requests come in? This is something we've been solving with RDBMSes since the 1970s.
    - pdonis 5mo ago
      > If the original request hasn't gotten a response yet, why is it sending a retry? Um, because connections over the Internet aren't 100% always on? Because packets can get lost? Because computers sometimes have to reboot? You're assuming that the client will always receive whatever response your server finally sends, and that the client will wait indefinitely to receive a response. Neither of those things are true. So the client can be in a state where it sends a retry because it got no response and doesn't know why. And that means a retry request could come in while the first one is still being resolved--because the client had a timeout or it rebooted or something else happened that made it lose the connection state it previously had. That's the case I'm asking about.
      - stickfigure 5mo ago
        I'm not assuming anything. Let me try to reframe this for you. The case of "client sends a retry with the same idempotency key" generalizes to "multiple requests come in for the same idempotency key". These can come in spread out over time (like a traditional loop), or they could come in at once. The solution is the same either way. The problem of "how do we deal with multiple conflicting requests coming in at once" is something we have been dealing with for decades. We have databases with transactions and isolation levels. If I said in an interview "make an endpoint that inserts a value in a database and returns an error if the value is a duplicate", any competent backend web developer should be able write it without Claude's help. Concurrency is part of our life. Whether you want to return 409 or replay the success is irrelevant to this question. You must serialize the idempotent operation on the server, because you can have multiple requests coming in simultaneously. If you put the operation in a database transaction with an appropriate isolation level, you are most of the way there.
- hn_throwaway_99 5mo ago
  To be honest, I liked your original response about returning a 409 - it's not something I'd done before and I like how it keeps things simpler. But your follow up responses here are making me rethink. Now you have to have all these special cases where the original request is still in process. I think or assertion of "99% are simple POST operations" is bullshit. For the times where idempotency is hard and really matters, often times you're calling a third party API, like a payment processing API. I would think a better approach would be to always return a 409 on a subsequent request, regardless of whether it passed or failed, and then have a separate standard API that lets you get the result of any request by its idempotency key.
  - stickfigure 5mo ago
    > special cases where the original request is still in process This isn't a special case, and it's the same problem if you want to replay the original response on conflict. If the original request isn't complete, what are you going to replay?
    - pdonis 5mo ago
      > If the original request isn't complete, what are you going to replay? Who says you have to replay? If you get a second request with the same idempotency key, and the original request is still in process, why not just send the client a response that says so?
      - deleted 5mo ago
        [deleted]
      - stickfigure 5mo ago
        It's not a terrible question. It would be complicated to implement and isn't more useful than using the database's consistency model, so nobody does it that way. Long running transactions create all sorts of problems, so transactions are generally expected to be short. The actual work behind "create payment" or "create order" is generally fairly trivial - more or less insert a row in a table. There's no good reason to make the API complicated... you either "win" at concurrency or you lose, and the difference is generally sub-millisecond. The only meaningful thing you need to communicate to the client is "you're done" (for both the win and lose cases) or "you need to try again" (for the "something unexpected went wrong" case). Complicated workflows can certainly have multiple steps, with "fetch the current status" calls in between. But somewhere near the beginning of every complicated workflow there will be a call to "create workflow" and it will need to have sort of mechanism which allows clients to call it idempotently. Otherwise you end up with multiple starts. I've literally received duplicate products in the mail because of this kind of problem. I've also sent multiple products in the mail because services I relied on didn't offer the necessary idempotency mechanisms.
saltcured 5mo ago
Being charitable, I'd say the poster above is saying that in the web architecture you can (should?) shift more of the burden for idempotence to the client. But, rather than 409, I'd say that you should be using opportunistic concurrency control if you adopt this perspective. There should be a resource context for the request, so the client can obtain an ETag and send If-None-Match headers, and get a 412 response if things are out of sync. That allows them to retry a failed/lost request and safely prevent a double action. Under a 412, they have to step back and retry a larger loop where they GET some new state and prepare a new action. Just like in DB transaction programming, where your failed commit means you roll back, clean the slate, and start a whole new interrogation of transaction-protected state leading up to your new mutation request.
- jeremyjh 5mo ago
  The client is already participating in a transaction in a distributed system. There is no way to change the reality of that. Suggestions about masking this only make the composite system unsound and will not improve net service reliability improvement. That doesn't mean that idempotency keys have to be used. You can certainly hash message content if that is documented behavior. That probably only makes sense when there is already some logical session or transaction identifier that makes dedupe semantics clear. The system you propose might be sound and might be necessary in some systems, but I can't think of what they might be that wouldn't be better served by the simpler solution that is already widely used for this purpose.
jmalicki 5mo ago
Your database should not allow both commits to happen - one should get rolled back. If it processed 99% of the request and the final bookkeeping failed because of a duplicate, that's still a failed request. Arguably this should be the primary way you check for idempotent requests - you shouldn't have a separate check for existence, you should have the insert/update fail atomically. This is the same thing you see on filesystems for TOCTOU security holes - the right way is to atomically access and modify once, and you only know the request was already processed because that fails.
- scott_w 5mo ago
  Not in the payments world. If you’re 99% done but only the bookkeeping failed, then it’s likely that money is changing hands and you need to deal with that fact. Payments are not an atomic infrastructure and you cannot magic that into reality.
  - stickfigure 5mo ago
    Payments are multistep but each of the steps needs to be atomic. The "create payment" operation must be transactional and the communication channel between you and the processor must be idempotent so you don't inadvertently create multiple payments. The fact that payments have a settlement process is not relevant to this discussion.
    - jmalicki 5mo ago
      And if there is part of the process that isn't idempotent, you want to make that surface area as small as possible, so that only failures at that one discrete step can cause issues.
    - scott_w 5mo ago
      > The "create payment" operation must be transactional and the communication channel between you and the processor must be idempotent so you don't inadvertently create multiple payments. Yes, I agree. You want to generate a token, persist it locally and use that to communicate with the payment gateway, so re-submissions use the same key and either error or return the transaction state. > The fact that payments have a settlement process is not relevant to this discussion. I wasn't talking about settlement, I was talking about the processing aspect. What I meant was: once you kickstart the process with the gateway, money is highly likely to change hands as a result. This means a process of: 1. POST /checkout 2. Create token 3. POST to payment gateway with token 4. Wait for gateway to return 5. Persist transaction/error 6. Return success/error What is needed is to persist and return the token to the caller before contacting the payment gateway, to make a check + retry mechanism possible. And yes, I've seen code that follows steps 1-6 exactly as I've described and, yes, all the problems you imagine would occur from those steps have occurred at one time or another.
scott_w 5mo ago
That’s because everyone here thinking about payments completely incorrectly. They’re not atomic and your server shouldn’t pretend they are. You need to store the payment state at each relevant step and process it asynchronously. If requests time out, you check the status of it using the key you store (with the processor) to see if it was even received. It’s not perfect, some processors will 500 while processing the payment (Braintree), so you still need reconciliation on the backend.
- fud101 5mo ago
  I'm not as smart as you folks but doesn't durable execution being a part of the solution help a lot in this respect?
turtlebits 5mo ago
You have a transaction row lock and you tell the subsequent requests to f off. I would go as far as returning a 429. Devs are too scared to be nice (ie not return errors) to clients when they misbehave.
kelnos 5mo ago
You still return 409, with error detail in the response body, because it is still true that the client sent multiple requests with the same idempotency key, regardless of the progress of the original request. Regardless, I think your assumption about how the request/response cycle should be working is wrong. For this kind of API and transaction, the server should be returning a response immediately: 202 Accepted. The only thing the API server should be doing before returning is creating a row in a DB (with a "state" field with an initial value of "pending"), and pushing some work on a queue. The server should not be sitting there with the HTTP request open, trying to complete the transaction, and only returning a response to the client when the transaction is finished or has encountered an error. The client will have to learn about the progress of the state of the transaction outside of this initial request. There are many options here: polling, webhooks, a message queue like kinesis or kafka, etc.
- GRiMe2D 5mo ago
  Ah you see, you just essentially moved unique constraint to a client. It is no different generating an ID on client side and making a request. It is not idempotency. Idempotency-Key should not replay the response (it depends, actually). But also it should not error 409. You need to be content aware before adding Idemmpotency Key header handling. What will happen when the request is received and handled but during writing response body TCP connection dropped unexpectedly. And after second or two a connection reestablished. How two sides agree that previous request accepted and everything good to go? That's what Idempotency-Key header does.
- pdonis 5mo ago
  By "request" I didn't necessarily mean the HTTP request sent by the client, and I don't think the post I was responding to did either. But I agree my use of terminology was ambiguous. Let me restate more clearly, and in a way that shows the issue under your request/response process: An HTTP request comes in with a certain idempotency key. The server returns 202, as you say, and begins to process the database transaction. While the server is still procesing the database transaction, a second HTTP request comes in with the same idempotency key. What response does this second HTTP request get? The original transaction that the first HTTP request triggered hasn't succeeded and hasn't failed, so it doesn't fall into either of the categories in the post I responded to. Your answer is that the second HTTP request gets a 409, which makes sense to me, although others are objecting to it.

by pdonis 5mo ago

What if the original request is still being processed when the retry comes in? That doesn't fall into either of your categories: the request isn't successful, but it hasn't failed either.

stickfigure 5mo ago

That's usually solved with traditional database transactions. Even if you have a complex long-running multistep orchestration problem, you can break it down into simpler transactions. Eg you could start with a "lock the resources" txn. But 99% of these conversations around idempotence are simple POST operations like "create order" that regular old database concurrency management handles just fine.

pdonis 5mo ago
> That's usually solved with traditional database transactions. That doesn't answer my question. What response do you return to the client in the case I described?
- matsemann 5mo ago
  The lock would normally make the second request wait, aka not return a response, until the first one is done. Then it sees it's a duplicate and returns that. Or it times out and returns an error. Then the client hopefully have some exponential back off strategy, so the third attempt doesn't suffer the same fate.
  - MattDaEskimo 5mo ago
    If a transaction is locked then subsequent requests would return a 409, ideally with an error message indicating that it's currently being processed.
- stickfigure 5mo ago
  This is just normal concurrent programming? If two requests come in for the same idempotency key/customer reference id, only one will succeed. Use standard database transaction isolation. So one will complete with 200, one will complete with 409. It doesn't matter which. That said, there's something odd about the way you phrased this question. If the original request hasn't gotten a response yet, why is it sending a retry? What you're asking is more general: What happens when two conflicting requests come in? This is something we've been solving with RDBMSes since the 1970s.
  - pdonis 5mo ago
    > If the original request hasn't gotten a response yet, why is it sending a retry? Um, because connections over the Internet aren't 100% always on? Because packets can get lost? Because computers sometimes have to reboot? You're assuming that the client will always receive whatever response your server finally sends, and that the client will wait indefinitely to receive a response. Neither of those things are true. So the client can be in a state where it sends a retry because it got no response and doesn't know why. And that means a retry request could come in while the first one is still being resolved--because the client had a timeout or it rebooted or something else happened that made it lose the connection state it previously had. That's the case I'm asking about.
    - stickfigure 5mo ago
      I'm not assuming anything. Let me try to reframe this for you. The case of "client sends a retry with the same idempotency key" generalizes to "multiple requests come in for the same idempotency key". These can come in spread out over time (like a traditional loop), or they could come in at once. The solution is the same either way. The problem of "how do we deal with multiple conflicting requests coming in at once" is something we have been dealing with for decades. We have databases with transactions and isolation levels. If I said in an interview "make an endpoint that inserts a value in a database and returns an error if the value is a duplicate", any competent backend web developer should be able write it without Claude's help. Concurrency is part of our life. Whether you want to return 409 or replay the success is irrelevant to this question. You must serialize the idempotent operation on the server, because you can have multiple requests coming in simultaneously. If you put the operation in a database transaction with an appropriate isolation level, you are most of the way there.
hn_throwaway_99 5mo ago
To be honest, I liked your original response about returning a 409 - it's not something I'd done before and I like how it keeps things simpler. But your follow up responses here are making me rethink. Now you have to have all these special cases where the original request is still in process. I think or assertion of "99% are simple POST operations" is bullshit. For the times where idempotency is hard and really matters, often times you're calling a third party API, like a payment processing API. I would think a better approach would be to always return a 409 on a subsequent request, regardless of whether it passed or failed, and then have a separate standard API that lets you get the result of any request by its idempotency key.
- stickfigure 5mo ago
  > special cases where the original request is still in process This isn't a special case, and it's the same problem if you want to replay the original response on conflict. If the original request isn't complete, what are you going to replay?
  - pdonis 5mo ago
    > If the original request isn't complete, what are you going to replay? Who says you have to replay? If you get a second request with the same idempotency key, and the original request is still in process, why not just send the client a response that says so?
    - deleted 5mo ago
      [deleted]
    - stickfigure 5mo ago
      It's not a terrible question. It would be complicated to implement and isn't more useful than using the database's consistency model, so nobody does it that way. Long running transactions create all sorts of problems, so transactions are generally expected to be short. The actual work behind "create payment" or "create order" is generally fairly trivial - more or less insert a row in a table. There's no good reason to make the API complicated... you either "win" at concurrency or you lose, and the difference is generally sub-millisecond. The only meaningful thing you need to communicate to the client is "you're done" (for both the win and lose cases) or "you need to try again" (for the "something unexpected went wrong" case). Complicated workflows can certainly have multiple steps, with "fetch the current status" calls in between. But somewhere near the beginning of every complicated workflow there will be a call to "create workflow" and it will need to have sort of mechanism which allows clients to call it idempotently. Otherwise you end up with multiple starts. I've literally received duplicate products in the mail because of this kind of problem. I've also sent multiple products in the mail because services I relied on didn't offer the necessary idempotency mechanisms.

saltcured 5mo ago

Being charitable, I'd say the poster above is saying that in the web architecture you can (should?) shift more of the burden for idempotence to the client. But, rather than 409, I'd say that you should be using opportunistic concurrency control if you adopt this perspective. There should be a resource context for the request, so the client can obtain an ETag and send If-None-Match headers, and get a 412 response if things are out of sync. That allows them to retry a failed/lost request and safely prevent a double action. Under a 412, they have to step back and retry a larger loop where they GET some new state and prepare a new action. Just like in DB transaction programming, where your failed commit means you roll back, clean the slate, and start a whole new interrogation of transaction-protected state leading up to your new mutation request.

jeremyjh 5mo ago
The client is already participating in a transaction in a distributed system. There is no way to change the reality of that. Suggestions about masking this only make the composite system unsound and will not improve net service reliability improvement. That doesn't mean that idempotency keys have to be used. You can certainly hash message content if that is documented behavior. That probably only makes sense when there is already some logical session or transaction identifier that makes dedupe semantics clear. The system you propose might be sound and might be necessary in some systems, but I can't think of what they might be that wouldn't be better served by the simpler solution that is already widely used for this purpose.

jmalicki 5mo ago

Your database should not allow both commits to happen - one should get rolled back. If it processed 99% of the request and the final bookkeeping failed because of a duplicate, that's still a failed request. Arguably this should be the primary way you check for idempotent requests - you shouldn't have a separate check for existence, you should have the insert/update fail atomically. This is the same thing you see on filesystems for TOCTOU security holes - the right way is to atomically access and modify once, and you only know the request was already processed because that fails.

scott_w 5mo ago
Not in the payments world. If you’re 99% done but only the bookkeeping failed, then it’s likely that money is changing hands and you need to deal with that fact. Payments are not an atomic infrastructure and you cannot magic that into reality.
- stickfigure 5mo ago
  Payments are multistep but each of the steps needs to be atomic. The "create payment" operation must be transactional and the communication channel between you and the processor must be idempotent so you don't inadvertently create multiple payments. The fact that payments have a settlement process is not relevant to this discussion.
  - jmalicki 5mo ago
    And if there is part of the process that isn't idempotent, you want to make that surface area as small as possible, so that only failures at that one discrete step can cause issues.
  - scott_w 5mo ago
    > The "create payment" operation must be transactional and the communication channel between you and the processor must be idempotent so you don't inadvertently create multiple payments. Yes, I agree. You want to generate a token, persist it locally and use that to communicate with the payment gateway, so re-submissions use the same key and either error or return the transaction state. > The fact that payments have a settlement process is not relevant to this discussion. I wasn't talking about settlement, I was talking about the processing aspect. What I meant was: once you kickstart the process with the gateway, money is highly likely to change hands as a result. This means a process of: 1. POST /checkout 2. Create token 3. POST to payment gateway with token 4. Wait for gateway to return 5. Persist transaction/error 6. Return success/error What is needed is to persist and return the token to the caller before contacting the payment gateway, to make a check + retry mechanism possible. And yes, I've seen code that follows steps 1-6 exactly as I've described and, yes, all the problems you imagine would occur from those steps have occurred at one time or another.

scott_w 5mo ago

That’s because everyone here thinking about payments completely incorrectly. They’re not atomic and your server shouldn’t pretend they are. You need to store the payment state at each relevant step and process it asynchronously. If requests time out, you check the status of it using the key you store (with the processor) to see if it was even received. It’s not perfect, some processors will 500 while processing the payment (Braintree), so you still need reconciliation on the backend.

fud101 5mo ago
I'm not as smart as you folks but doesn't durable execution being a part of the solution help a lot in this respect?

turtlebits 5mo ago

You have a transaction row lock and you tell the subsequent requests to f off. I would go as far as returning a 429. Devs are too scared to be nice (ie not return errors) to clients when they misbehave.

kelnos 5mo ago

You still return 409, with error detail in the response body, because it is still true that the client sent multiple requests with the same idempotency key, regardless of the progress of the original request. Regardless, I think your assumption about how the request/response cycle should be working is wrong. For this kind of API and transaction, the server should be returning a response immediately: 202 Accepted. The only thing the API server should be doing before returning is creating a row in a DB (with a "state" field with an initial value of "pending"), and pushing some work on a queue. The server should not be sitting there with the HTTP request open, trying to complete the transaction, and only returning a response to the client when the transaction is finished or has encountered an error. The client will have to learn about the progress of the state of the transaction outside of this initial request. There are many options here: polling, webhooks, a message queue like kinesis or kafka, etc.

GRiMe2D 5mo ago
Ah you see, you just essentially moved unique constraint to a client. It is no different generating an ID on client side and making a request. It is not idempotency. Idempotency-Key should not replay the response (it depends, actually). But also it should not error 409. You need to be content aware before adding Idemmpotency Key header handling. What will happen when the request is received and handled but during writing response body TCP connection dropped unexpectedly. And after second or two a connection reestablished. How two sides agree that previous request accepted and everything good to go? That's what Idempotency-Key header does.
pdonis 5mo ago
By "request" I didn't necessarily mean the HTTP request sent by the client, and I don't think the post I was responding to did either. But I agree my use of terminology was ambiguous. Let me restate more clearly, and in a way that shows the issue under your request/response process: An HTTP request comes in with a certain idempotency key. The server returns 202, as you say, and begins to process the database transaction. While the server is still procesing the database transaction, a second HTTP request comes in with the same idempotency key. What response does this second HTTP request get? The original transaction that the first HTTP request triggered hasn't succeeded and hasn't failed, so it doesn't fall into either of the categories in the post I responded to. Your answer is that the second HTTP request gets a 409, which makes sense to me, although others are objecting to it.