3 ms·

A distributed locking library built on top of Cloud Spanner and TrueTime

31 points by ithkuil 5y ago

kyrra 5y ago
Googler opinions are my own. I'm not a spanner expert, but I use F1 at least a little. If you start having global spanner replicas, don't you start getting into read safe times? According to the GCP docs, a given node may not know if it has the latest data, as global replicas could diverge by up to 10 seconds ( assuming I'm reading the docs right). I am little sad that Google never made chubby a service on GCP. That is underlying locking system that many systems that Google use. Chubby: https://research.google/pubs/pub27897/ https://research.google/pubs/pub27897/ Spanner read write docs: https://cloud.google.com/spanner/docs/whitepapers/life-of-reads-and-writes#strong_read_multi-node https://cloud.google.com/spanner/docs/whitepapers/life-of-re...
- leajkinUnk 5y ago
  From what I remember, Chubby was one of those services that was a bit more sensitive to poor client behavior. You could cause problems for other users with poor Chubby behavior, and you could accidentally end up with code that causes a high Chubby load.
  - gravypod 5y ago
    (Opinions are my own) Chubby is a lot like etcd except without the hindsight. Distributed locking is pretty complicated and if I remember correctly there was a great podcast from Google's Kube podcast about distributed locking that goes into a lot of this. [0] If you are going to do any distributed locking I strongly recommend listening to it. Has a lot of pain points discussed. [0] - https://kubernetespodcast.com/episode/123-leader-election/ https://kubernetespodcast.com/episode/123-leader-election/
  - lokar 5y ago
    Chubby depends on clients using a complex library for full correctness.
- joshuamorton 5y ago
  > I am little sad that Google never made chubby a service on GCP. That is underlying locking system that many systems that Google use. I think one of the issues with this is that its far easier to use chubby incorrectly than to use it correctly. Like, you can use chubby to build really, really cool and powerful tools (https://www.usenix.org/system/files/conference/osdi16/osdi16-adya.pdf https://www.usenix.org/system/files/conference/osdi16/osdi16...), but I wouldn't be surprised if most users misused or overused it, and that would likely be magnified on a public version.
nn3 5y ago
On the one hand it looks interesting. On the other hand Spanner must have a perfectly fine distributed lock manager on its own. It seems odd to implement a lock manager on another lock manager (which just happens to be a database too) just because you can't directly access it. I'm sure that must cause unnecessary overhead. It's a bit like those odd nesting cases that happen in networking, like PPPoE over Ethernet. Maybe that's the eventual fate of computing as it evolves, more and more layers everywhere, that are more and more redundant but still exist for some odd reason, with low level facilities being reinvented.
- dekhn 5y ago
  Spanner links in a paxos library, IIUC for every distinct spanserver.
zie 5y ago
Arguably most of BTC's energy usage is spent trying to lock transactions in a safe, distributed manner. How long until we get SpindleCoin? :P

31 points by ithkuil 5y ago

Googler opinions are my own. I'm not a spanner expert, but I use F1 at least a little. If you start having global spanner replicas, don't you start getting into read safe times? According to the GCP docs, a given node may not know if it has the latest data, as global replicas could diverge by up to 10 seconds ( assuming I'm reading the docs right). I am little sad that Google never made chubby a service on GCP. That is underlying locking system that many systems that Google use. Chubby: https://research.google/pubs/pub27897/ https://research.google/pubs/pub27897/ Spanner read write docs: https://cloud.google.com/spanner/docs/whitepapers/life-of-reads-and-writes#strong_read_multi-node https://cloud.google.com/spanner/docs/whitepapers/life-of-re...

leajkinUnk 5y ago
From what I remember, Chubby was one of those services that was a bit more sensitive to poor client behavior. You could cause problems for other users with poor Chubby behavior, and you could accidentally end up with code that causes a high Chubby load.
- gravypod 5y ago
  (Opinions are my own) Chubby is a lot like etcd except without the hindsight. Distributed locking is pretty complicated and if I remember correctly there was a great podcast from Google's Kube podcast about distributed locking that goes into a lot of this. [0] If you are going to do any distributed locking I strongly recommend listening to it. Has a lot of pain points discussed. [0] - https://kubernetespodcast.com/episode/123-leader-election/ https://kubernetespodcast.com/episode/123-leader-election/
- lokar 5y ago
  Chubby depends on clients using a complex library for full correctness.
joshuamorton 5y ago
> I am little sad that Google never made chubby a service on GCP. That is underlying locking system that many systems that Google use. I think one of the issues with this is that its far easier to use chubby incorrectly than to use it correctly. Like, you can use chubby to build really, really cool and powerful tools (https://www.usenix.org/system/files/conference/osdi16/osdi16-adya.pdf https://www.usenix.org/system/files/conference/osdi16/osdi16...), but I wouldn't be surprised if most users misused or overused it, and that would likely be magnified on a public version.

nn3 5y ago

On the one hand it looks interesting. On the other hand Spanner must have a perfectly fine distributed lock manager on its own. It seems odd to implement a lock manager on another lock manager (which just happens to be a database too) just because you can't directly access it. I'm sure that must cause unnecessary overhead. It's a bit like those odd nesting cases that happen in networking, like PPPoE over Ethernet. Maybe that's the eventual fate of computing as it evolves, more and more layers everywhere, that are more and more redundant but still exist for some odd reason, with low level facilities being reinvented.

dekhn 5y ago
Spanner links in a paxos library, IIUC for every distinct spanserver.

zie 5y ago

Arguably most of BTC's energy usage is spent trying to lock transactions in a safe, distributed manner. How long until we get SpindleCoin? :P