3 ms·
First let me go over the levels of redundancy: 1. In memory in the browser tab's process. 2. If available, in the browser's localstorage or fallback. 3. In t
by marknadal 13y ago
First let me go over the levels of redundancy:
1. In memory in the browser tab's process.
2. If available, in the browser's localstorage or fallback.
3. In the server process's memory.
4. If available, in Redis on the server.
5. If in a multi-machine setup, any other connected server that is subscribed to that data set, being in memory (3) or in Redis (4) if available.
6. If configured, in a machine log on S3.
7. Persisted to S3, which replicates and shards it for you internally.
8. If configured, in a revision file on S3.
9. If configured, in a multi-region S3 setup, redundantly in many places.
(2) is not cleared till an acknowledgment that (7) is confirmed. (1) is not cleared until an acknowledgement that (7) is confirmed or if the tab is exited. In the case of (7) it is no longer the delta/diff, but a snapshot of that current data set with that delta/diff's update. Retries from (1) ~ (5) will happen at various events, if the confirmations are not satisfied. If a conflict has already occurred by (3) the acknowledgement from (5) will include a notification that the value has already been updated, along with the standard delta/diff of that conflicting update being sent down. Meaning (5) does not guarantee that your delta/diff has "won", only that it has been saved or is already outdated.
Worst case condition is that (2, 4, 5, 6, 8, 9) are turned off, in which your user's data is as volatile as them preemptively leaving the page (although I suppose you could use an onbeforeunload to warn them) - however this behavior is the current norm for most http post based forms and apps. Actually, pardon me, worst case condition is that everything is offline simultaneously, however this is not really interesting because then users won't even be able to access your app in the first place.
Please correct me if I have my terms wrong:
Fail stop - since gun runs in your application process, any bugs or errors in gun should result in a standard error being thrown. In the case of javascript, your process will crash. Generally speaking you are responsible for the liveliness of your app uptime, however if you use some of my other existing open source libraries, they will respawn the process for you. When your app restarts, so will gun.
Receive omission - as mentioned before, the peer that originated the message will attempt to retry messages until a confirmation is given. So even if a process somehow does not receive a message, it eventually will, unless the origin gives up.
Send omission - this is a bit trickier, gun tries its best to keep user changes by all means possible until a confirmation is received. If for some reason gun is unable to do this, or the user's browser starts going haywire and doing weird things... you have no way to know and neither does gun. At this level, something is fundamentally wrong with the runtime or the OS, and gun has no way to check this.
Arbitrary - malicious attacks are best done from server peers, however this would require the malicious node to know your key. Currently the event of this happening may be decently high, as all it would require is for an attacker to take over an IP that another one of your gun node might connect to - the connecting node will issue the key as proof of being not malicious. I definitely could use some help on figuring out a more secure, yet still fully decentralized means of authentication and trustworthiness of server nodes. In the instance of byzantine mistakes, this will only effect the system if the message actually complies with the spec - in which case it does, the message is indistinguishable from a real request, therefore will be treated as one. And finally, what about an intentionally malicious user? The way gun's conflict resolution algorithms work makes it very difficult to be abused in any way that is actually meaningful to the malicious user, but I'm saving the details of this for a real post. That being said, such abuse will happen and I can already describe the results for you - "annoying" and "spammy" - luckily gun as filters built into the piping channels, so if you as the developer notice any particularly fishy behavior, you can always block that user out.
Recovery - when gun starts (or restarts) it goes about its usual business of checking its cache (if Redis is available) and retrying whatever is there that had failed to be cleared, listening to messages which may be retries of things that had failed to be cached (especially in the case Redis isn't enabled), and intelligently pulling things back in from S3. When any data set is loaded into gun it will also subscribe to that data set so it can be notified of updates from any other node, gun will also merge (using the same conflict resolution algorithms) the data set it got from S3 with any other node that also has that data set, and even replay some persisted logs if available. This way it boots itself back up into the most valid and live operating status that it can.
Partitions - there are 9 different layers, of which communication and networking can all fail or go out. That is about 9 factorial amount of combinations that can, could, and probably will go wrong (although this is an over estimation since not all the layers require networking). I simply can't cover them in this comment, but will go over some of the most concerning and common cases in a separate post - I also plan on building a partition simulator, so that way people can play with causing these issues, and so that I can do testing against them. That being said, gun as a strong commitment to handling these things, since it can afford to (since gun is eventually consistent, we have time to make up for networking problems). I hope this comment was helpful in what I did go over - I'm not sure if you'll even wind up seeing it (please notify me that you did! Even if that is just an upvote) I'll probably reuse a lot of this in one of my actual posts.
Thanks again! Anything else?