4 ms·
My (unseen) emphasis was on best bet. It would work, but I think it would be dangerous and/or inconvenient. The danger part comes from having a sync-to-disk o
by yourad_io 12y ago
My (unseen) emphasis was on best bet.
It would work, but I think it would be dangerous and/or inconvenient.
The danger part comes from having a sync-to-disk operation that lasts any considerable amount of time - the longer it lasts, the more the likelihood that an inopportune crash would leave you with an incomplete (read: corrupted) JSON file. A DB built for fast disk persistence would only update the relevant records, keeping the disk writes as small as possible. I don't think Loki has any option other than writing the entire thing each time (with the current JSON savefiles, that is). Since save() is an expensive op, you also can't be calling it at every update (it wouldn't even work! the first one would lock the file for writing and all subsequent save() ops, until the first one is done, would fail!) so it would be inherently unsafe unless you committed GBs to disk for each update!.
So, this might be somewhat dangerous, but we can mitigate that, right? We'll save frequently, but not too frequently, and somehow version-control our JSON file. Which is GB+ in size. So we'll also compress them? And when we need to restore, we'd.. hmm.. start reading from the latest until a valid one is found? (<--inconvenience)
> Note, I'm really just dreaming here, and I appreciate you dreaming along.
Likewise!
> Sounds like there may be plenty of persistence schemes that make this not entirely painful (load latest data first, or fallback to disk reads if the data isn't entirely loaded yet, which makes the service slower but still available right after a crash).
While those certainly exist, the one currently chosen by Loki can't do any of these things. Since you can't parse half a JSON file (especially in this format, which appends index info etc at the end of the file), you have to read the entire thing from disk, JSON.parse it, and then feed it into Loki. Only after all of that will you know that you are actually restoring from meaningful (and complete) data files and not junk.
The way I see this, it would be great for throwaway-prototypes on the serverside (or for few/unimportant data) but its real value would show on the client side. You basically get a mini-mongodb for 2K LOC that you can embed on anything that speaks ECMA3.
What's even cooler IMHO is the future - on their slideshare presentation they list replication and horizontal scaling on their roadmap. While I have no idea how they have envisaged implementing those, I would love to experiment with the meteor.js concepts and this - providing a full, fast cache of the user's data on the client side and then throw differential updates around.
edit: spelling