3 ms·
Author of gun.js here, great comments/concerns. First, Aphyr (Kyle Kingsbury of Call Me Maybe fame, Jepsen database testing suite) did some tweets about GUN (
by marknadal 11y ago
Author of gun.js here, great comments/concerns.
First, Aphyr (Kyle Kingsbury of Call Me Maybe fame, Jepsen database testing suite) did some tweets about GUN ( https://twitter.com/aphyr/status/646302398575587332?ref_src=twsrc%5Etfw https://twitter.com/aphyr/status/646302398575587332?ref_src=... ), and I met with him at a conference in Berlin where we both were talking. He had some concerns (wanted the conflict resolution to be able to be plug-in-play with your-own-CRDT, and if journaling is disabled then some stale/old updates would be dropped), but overall seemed positive about GUN (check his own tweets). I hope to have him do a more serious review at some point, but I also hope that his initial thoughts provides some initial industry peer review.
Tailing the first point, part of the reason why we didn't receive a more critical response from him is because we make NO claims of serializability or linearizability, which is what most academic research focuses on. In fact, GUN takes a somewhat controversial view with regards to classical things like Strong Consistency. For more information on how GUN's algorithm works, check out this recent podcast we did with the LondonJS community - https://youtu.be/qJNDplwJ8aQ?t=32m45s https://youtu.be/qJNDplwJ8aQ?t=32m45s .
Second, GUN does use CRDTs. All CRDT means is "Convergent Replicated Data Type", which is a fancy way of saying that you'll get the same end result regardless of the ordering of the operations involved. Because GUN is offline-first it is incredibly important that it can retry an update multiple times to handle network partitions - this fundamentally results in operations not having a consistent ordering, especially when dealing with multiple peers updating (offline) simultaneously. If you watch the "Holy Grail Demo" (1min long, https://youtu.be/-i-11T5ZI9o https://youtu.be/-i-11T5ZI9o ) you see that this does not wind up being a problem.
Third, there are other CRDTs that can be implemented on top of GUN's core. A specific instance of where GUN's default will fail you is in commutative operations (something Kyle was also concerned about), by default you cannot do increment (adding numbers) operations with GUN - partly because GUN treats a single value as being atomic. If you have a HN upvote count and you naively were to use GUN's out of the box CRDT, and two people locally added 1 to their local value at the same time... they would both converge to the same end result (5 + 1 = 6) but the intended result is 7. This intention-preserving CRDT is called a Commutative Replicated Data Type, which a specific variant on the general category. However, this does not mean that it is impossible to do this with GUN - you can, because GUN's underlying CRDT is designed to be emergent, allowing for other CRDTs to be built on top. At some point we'll (or somebody in the community) will provide these as plug-in-play module/extensions to GUN, so that way you won't have to build them yourself.
Fourth, "home-grown" is somewhat true, but I think that is giving me too much credit. GUN's underlying conflict resolution algorithm is based on a hybrid lexical, vector, and timestamp clock - hardly magic at all, in fact they are somewhat naive intentionally. The nice thing is that they "just work" for the majority of web based apps right out of the box, which is probably why it seems magical. However, they won't work (alone) for more complicated business logic, and that is why we encourage the UNIX/NodeJS philosophy of adding those algorithms on top. It is important that your bread-and-butter machine is not based on a monolithic black box (like a lot of databases are), instead you should understand the base of the system and then plug your business-specific needs on top.
Fifth, we're looking for more academic and peer review! I've already had numerous discussions with a Distributed Systems PhD at Carnegie Mellon that got poached by Uber's self driving car division, he's trying to help me find some people to put together a paper. That said, if you know anybody or could help connect me into that world it would be greatly appreciated.
Thanks so much for including us in the list, please shout if you have any more questions! :)
- lewisl9029 11y agoThanks Mark, that definitely cleared up a lot of my concerns on Gun. To be honest, I only had a cursory glance at it when I first heard about it, but reading your comment definitely sparked enough interest for me to take a deeper look. One question still on my mind after quickly skimming your docs: Can Gun sync directly between browsers (using something like WebRTC) without involving any servers? If so, could you point me to where it might be documented? If not, please consider this a feature request. =)
- marknadal 11y agoThe answer is "Yes, it is in development" by one of our contributors: https://github.com/PsychoLlama/gun-rtc https://github.com/PsychoLlama/gun-rtc . But you'll realistically have to wait (or help) with it, before it is usable. As you know though, WebRTC unfortunately requires some initial servers (STUN/ICE, or a signaling server, etc.) so it is not fool-proof P2P sadly.