4 ms·

optimistic locking is a useful technique that seems unfamiliar to many - it's worth looking into

nesarkvechnep 5y ago
How could it be familiar to people as most of them hopped on the GraphQL hype train and treat HTTP as a dumb pipe.
karmakaze 5y ago
I thought I didn't know what it was when I heard of them, then realized that it's not a lock at all, it's just a row version column, with the basic rule don't overwrite without having seen what you're about to overwrite. Same goes for `git push --force`, always use `git push --force-with-lease` instead.
- marcos100 5y ago
  Why do you think it is not a lock?
  - mirekrusin 5y ago
    It's not a lock in the sense that that is succeeds or fails immediatelly. It is a lock in the sense that it allows success only. Pesimistic lock usually means "maybe wait then success". In complex locking spaghetti it may mean deadlock. It may also mean wait then timeout. It may also mean wait then timeout then I don't know what actually happened, maybe success, maybe not.
  - karmakaze 5y ago
    It is a locking convention, achieving something locks are used for, but you can't point to one thing and say, there, that's the lock. Contrast with a database row lock where no other transaction can change that row or get a locked access to it until released. e.g. if the version column is an incrementing number, then it relies on no client unilaterally incrementing the value on failure and retrying--not much of a 'lock'.
    - speed_spread 5y ago
      You can actually think of the version number in optimistic locking as a shared key combination. First party to use the key gets to pick the next combination, invalidating any other concurrent copies. Performance and implementation characteristics differ but the end result is functionally the same as a pessimistic lock, which is why they're both called "locking" mechanisms.
      - cormacrelf 5y ago
        The end result is functionally very different. If you were given a shared updatable value, and were told that the only API was non-blocking and you would have to retry in a loop until it succeeded, would you call it a lock? It’s straight up not a lock. The functional difference is crucial here, the non-lock characteristics are what allows them to offer atomic updates via HTTP. They couldn’t have used pessimistic locking to do this, not only because it is is limited to a single db transaction, but because they can not trust API users or the network to ensure the rows ever get unlocked.
        speed_spread 5y ago
        A lock does not offer a guarantee of completion but one of integrity. Any serious pessimistic lock granted will have a timeout associated with it (in case the owner dies, etc.), which might cause your operation to fail if it takes too long. Which would require a retry in the same way that you would retry an optimistic lock. Then again, it just depends on timing.
        cormacrelf 5y ago
        Do you define locks as things that guarantee data integrity? Is not offering concurrency at all locking? Is your point that optimistic is locking or that it bears some surface similarities to locking? I accept that surface similarities exist. But that doesn’t really help anyone understand the difference. Nobody is struggling to see that these two solutions are applicable to some of the same problems. Locks don’t directly guarantee data integrity. They guarantee mutual exclusion around a critical section or mutable access to data they logically contain. Mutual exclusion is a coarse-grained way to get actually serialized execution of concurrent accesses. That gives you the data integrity. Mutual exclusion also gives you all the problems when you send an acquired lock over a network. It also explains why parallelism on the guarded section/data is 1. This can be blocky in a read heavy environment, because each read blocks all the others. Two transactions can get into deadlock by acquiring in a particular order. These are characteristics of all locks. Optimistic locking (also known as optimistic concurrency control, because it’s not locking) does not provide mutual exclusion, in any way, shape, or form. You cannot acquire any kind of lock with it. There aren’t any. A lock has a pair of operations, one to begin mutual exclusion and one to end it, but OCC doesn’t have either. Because it’s not a lock. It therefore does not suffer from network partitions between the lock and the client, does not shove all readers through a parallelism=1 bottleneck, you do not have to worry about deadlock prevention or avoidance. Those problems simply do not exist for OCC. It is literally lock-free. It almost couldn’t be ANY further from being a lock. You don’t have to accept that retries being mandatory in the API makes it not a lock. You just have to observe that it isn’t a lock. Some things are locks, some things aren’t. I think we should avoid calling things that aren’t locks locks, because it’s confusing enough as it is. Frankly, I think if people hadn’t been calling OCC “optimistic locking” then this blog post would have been easier to write. Similarly, as I illustrated above, calling it “lock-free” helps people understand OCC’s advantages and challenges in the same terms we use to talk about lock-free algorithms on a multi-core CPU. It gives you the intuition that every client side call should be a loop. It gives you the intuition that overflow ≈ the ABA problem. All of these bits of understanding flow from calling things what they are.
- omegalulw 5y ago
  Wait, how do you ensure that the lock version itself is safely written by two or more writers? Not to mention what happens if you read the version and in the time you decide to do a write someone else incremented the version.

How could it be familiar to people as most of them hopped on the GraphQL hype train and treat HTTP as a dumb pipe.

I thought I didn't know what it was when I heard of them, then realized that it's not a lock at all, it's just a row version column, with the basic rule don't overwrite without having seen what you're about to overwrite. Same goes for `git push --force`, always use `git push --force-with-lease` instead.

marcos100 5y ago
Why do you think it is not a lock?
- mirekrusin 5y ago
  It's not a lock in the sense that that is succeeds or fails immediatelly. It is a lock in the sense that it allows success only. Pesimistic lock usually means "maybe wait then success". In complex locking spaghetti it may mean deadlock. It may also mean wait then timeout. It may also mean wait then timeout then I don't know what actually happened, maybe success, maybe not.
- karmakaze 5y ago
  It is a locking convention, achieving something locks are used for, but you can't point to one thing and say, there, that's the lock. Contrast with a database row lock where no other transaction can change that row or get a locked access to it until released. e.g. if the version column is an incrementing number, then it relies on no client unilaterally incrementing the value on failure and retrying--not much of a 'lock'.
  - speed_spread 5y ago
    You can actually think of the version number in optimistic locking as a shared key combination. First party to use the key gets to pick the next combination, invalidating any other concurrent copies. Performance and implementation characteristics differ but the end result is functionally the same as a pessimistic lock, which is why they're both called "locking" mechanisms.
    - cormacrelf 5y ago
      The end result is functionally very different. If you were given a shared updatable value, and were told that the only API was non-blocking and you would have to retry in a loop until it succeeded, would you call it a lock? It’s straight up not a lock. The functional difference is crucial here, the non-lock characteristics are what allows them to offer atomic updates via HTTP. They couldn’t have used pessimistic locking to do this, not only because it is is limited to a single db transaction, but because they can not trust API users or the network to ensure the rows ever get unlocked.
      - speed_spread 5y ago
        A lock does not offer a guarantee of completion but one of integrity. Any serious pessimistic lock granted will have a timeout associated with it (in case the owner dies, etc.), which might cause your operation to fail if it takes too long. Which would require a retry in the same way that you would retry an optimistic lock. Then again, it just depends on timing.
        cormacrelf 5y ago
        Do you define locks as things that guarantee data integrity? Is not offering concurrency at all locking? Is your point that optimistic is locking or that it bears some surface similarities to locking? I accept that surface similarities exist. But that doesn’t really help anyone understand the difference. Nobody is struggling to see that these two solutions are applicable to some of the same problems. Locks don’t directly guarantee data integrity. They guarantee mutual exclusion around a critical section or mutable access to data they logically contain. Mutual exclusion is a coarse-grained way to get actually serialized execution of concurrent accesses. That gives you the data integrity. Mutual exclusion also gives you all the problems when you send an acquired lock over a network. It also explains why parallelism on the guarded section/data is 1. This can be blocky in a read heavy environment, because each read blocks all the others. Two transactions can get into deadlock by acquiring in a particular order. These are characteristics of all locks. Optimistic locking (also known as optimistic concurrency control, because it’s not locking) does not provide mutual exclusion, in any way, shape, or form. You cannot acquire any kind of lock with it. There aren’t any. A lock has a pair of operations, one to begin mutual exclusion and one to end it, but OCC doesn’t have either. Because it’s not a lock. It therefore does not suffer from network partitions between the lock and the client, does not shove all readers through a parallelism=1 bottleneck, you do not have to worry about deadlock prevention or avoidance. Those problems simply do not exist for OCC. It is literally lock-free. It almost couldn’t be ANY further from being a lock. You don’t have to accept that retries being mandatory in the API makes it not a lock. You just have to observe that it isn’t a lock. Some things are locks, some things aren’t. I think we should avoid calling things that aren’t locks locks, because it’s confusing enough as it is. Frankly, I think if people hadn’t been calling OCC “optimistic locking” then this blog post would have been easier to write. Similarly, as I illustrated above, calling it “lock-free” helps people understand OCC’s advantages and challenges in the same terms we use to talk about lock-free algorithms on a multi-core CPU. It gives you the intuition that every client side call should be a loop. It gives you the intuition that overflow ≈ the ABA problem. All of these bits of understanding flow from calling things what they are.
omegalulw 5y ago
Wait, how do you ensure that the lock version itself is safely written by two or more writers? Not to mention what happens if you read the version and in the time you decide to do a write someone else incremented the version.