4 ms·
This is a really great comment, thank you - hopefully I can address some of the points you brought up: - LRU/GC hasn't been added (planned for v0.8.x), so you
by marknadal 9y ago
This is a really great comment, thank you - hopefully I can address some of the points you brought up:
- LRU/GC hasn't been added (planned for v0.8.x), so you are correct sending in a bunch of updates will crash the peer currently. The "out of bound" updates (for everybody else not sure what the parent is referencing, see this tech talk where I explain what is going on: http://gun.js.org/distributed/matters.html http://gun.js.org/distributed/matters.html ) are volatile (this is intentional) because they are considered potentially malicious, so upon crash they'll be lost as it is the origin peer's responsibility to retry updates (and gun automatically does) until ACKs have been received. Thankfully, updates that are "within bounds" will be kept safe.
- However, you are right (in your "If we don't...") "within bound" updates may also be malicious. GUN's base algorithm is designed to work in an entirely ad-hoc anonymous mesh network. To deal with this, we just recently announced our Security, Encryption, and Authorization framework to handle trusted peers, here: https://github.com/amark/gun/wiki/auth https://github.com/amark/gun/wiki/auth
- GUN is master-master, so no, even with a sliding window of 300ms from latency shouldn't effect the data. Your Sydney to London example is good, I've played a live action game built on top of GUN from Australia <-> USA, and even with P2P logic (no master server) it is responsive. The whole space ship game is only 190 LOC and you can play it here: http://gunjs.herokuapp.com/game/space.html http://gunjs.herokuapp.com/game/space.html (Warning: it is kinda a lame game, but proof conflicting updates in game states work just fine.)
- Also, fun fact, in a Master-Master system, the clock drift/skew on machines from different continents can become quite bad. Since we don't have atomic clocks like Spanner, we had to write a P2P version of NTP that runs along side the game. You can test how well it works across your devices here: http://gunjs.herokuapp.com/game/nts.html http://gunjs.herokuapp.com/game/nts.html
- Additionally, very rarely are you going to have that many writes within a 300ms span on the /same/ record. However you very easily might have that many for a Twitter like app. Back a year ago, we ran this load test on a prototype storage driver for work loads like this (append-only) and scaled to 100M+ messages for $10/day (all costs: CPU, disk, backup), check out the proof here: https://www.youtube.com/watch?v=x_WqBuEA7s8 https://www.youtube.com/watch?v=x_WqBuEA7s8
Thank you very much for writing your comment. Did I miss anything? More than happy to address any other concerns. The more we can challenge the assertions/claims of database vendors (like me), the better the industry will be off. Let me know if I can answer anything else. Thanks again!
- bastawhiz 9y agoIt would be good to see your claims backed up with data and benchmarks. I would love to see you run the Jepsen test suite against GUN also, so it's clear how it behaves in weird circumstances.
- marknadal 9y agoAgreed! I know Kyle, and we're planning on having him review the system, but we're still polishing some things up on our end and he's backlogged. You'll definitely hear about it when it happens. As for the other claims, most of these links are already scattered throughout the comments, but here they are again: - Numbers on the performance benchmark: https://github.com/amark/gun/wiki/100000-ops-sec-in-IE6-on-2GB-Atom-CPU https://github.com/amark/gun/wiki/100000-ops-sec-in-IE6-on-2... , also see https://youtu.be/BEqH-oZ4UXI https://youtu.be/BEqH-oZ4UXI , and run it yourself clone repo/test/ptsd/ptsd.html (we'll make this easier in the future) - Tests, load/scaling, split brain, and other: see https://youtu.be/x_WqBuEA7s8 https://youtu.be/x_WqBuEA7s8 , https://youtu.be/-i-11T5ZI9o https://youtu.be/-i-11T5ZI9o , https://youtu.be/-FN_J3etdvY https://youtu.be/-FN_J3etdvY , with https://github.com/gundb/panic-server https://github.com/gundb/panic-server you can run (let me know if you need any help getting it set up) https://github.com/amark/gun/blob/master/test/panic/load.js https://github.com/amark/gun/blob/master/test/panic/load.js , https://github.com/amark/gun/blob/master/test/panic/holy-grail.js https://github.com/amark/gun/blob/master/test/panic/holy-gra... . Anything else I can provide?