4 ms·
That doesn't make any sense. the timeout is how long to block for and retry, now how long to block for and continue.
by Racing0461 3y ago
That doesn't make any sense. the timeout is how long to block for and retry, now how long to block for and continue.
- 8organicbits 3y agoInstance A grabs the lock and makes an API call that takes 120 seconds. Instance B sees the lock but considers it expired after the lock times out at 100 seconds. Instance B falsely concludes A died, overwrites A's lock so the system doesn't dead lock waiting for A, and makes its own request. Unfortunately, A's request was still processing and B's accidentally concurrent request cause corruption.
- deleted 3y ago[deleted]
- Racing0461 3y ago> falsely concludes i disagree here. instance b did the right thing given the information it has. instance a should realize it no longer owns the lock and stop proceeding. but in reality it also signifies concurrency based limitations in the api itself (no ability to perform a do and then commit call). https://microservices.io/patterns/data/saga.html https://microservices.io/patterns/data/saga.html
- 8organicbits 3y agoI think we both agree that A did something wrong and that B followed the locking algorithm correctly. "Falsely" refers to a matter of fact: A is not dead. You're right that A could try to stop, but I think it's more complicated than that. A is calling a third party API, which may not have a way to cancel an in-flight request. If A can't cancel, then A should refresh its claim on the lock. A must have done neither in the example.
- ngc248 3y agoI have implemented a Distributed lock using DynamooDB and the timeout for the lock release needs to be slightly greater than the time taken to process the thing under lock. Else things like what you mention will happen.