5 ms·
Holy return of tuple spaces! Are there any features for controlling consistency, or is it just YOLO last write wins?
by noelwelsh 8y ago
Holy return of tuple spaces! Are there any features for controlling consistency, or is it just YOLO last write wins?
- kloepper 8y agoLast write wins. There are several features for controlling consistency that we have been prototyping. Based on how storage is used we will enable those as the product matures.
- zzzcpan 8y agoSo why not strong eventual consistency and CRDTs? As they fit perfectly for such functions running on edge nodes and no silly limitations of 1 write per second necessary.
- kentonv 8y agoIndeed. This is just stage 1 of our storage plans. We have some really cool stuff in the works but we wanted to get basic KV functionality out there for people to start using ASAP.
- _pmf_ 8y agoOT: we need to read more about your website-in-kv-thingy setup.
- kentonv 8y agoIt was pretty trivial. I just stored each page into a value and served them out of there, with some stupid code to infer content-type from file extension. I meant it as a dumb demo because I wasn't feeling creative. I didn't know John would mention it in his blog. :) Currently this isn't a great way to serve content because the values are limited to 64k, you have to infer content-type (which makes your URLs ugly), etc. So I don't recommend it. We'll come up with something better in the future. :)
- kloepper 8y agoYes, that is one of the models we have been considering.
- falcolas 8y agoSo, with a global consistency window of 10s, and 1 write per second, would consistency ever be reached if a key is written to every second? What value would be visible? One great example of why this would matter to me - could workers ever institute a coordinated rate limiting function using this KV store?
- zackbloom 8y agoOne thing to remember is you have at least two other ways of storing values. Each worker has global memory which persists between invocations (local, atomic), and you have access to the Cache API (per-data center, ephemeral). By combining those with KV to do global coordination on a more course time scale, I think you could.
- zzzcpan 8y agoRate limiting doesn't have to be coordinated to work. But you do need a PN-Counter, which they are only considering at this point.
- falcolas 8y agoA PN-Counter is a method of coordination. And if the writes were coordinated with the consistency time, that would be sufficient to implement consistent counters (or even a naive counter in a store given atomic operatons).
- zzzcpan 8y agoPN-Counter is coordination-free counter, it can't be used as a method of coordination. But it's ok for global rate limiting, as it doesn't have to guarantee exact rate at any point in time, as long as it eventually averages on specified rate.
- steego 8y agoI had to upvote you just because I plan on getting some mileage out the phrase "YOLO last write wins." It has just the right amount of snark to remind junior developers to not trivialize or discount the value of consistency.
- biesnecker 8y agoAnd it's really "YOLO last write recorded wins," because who knows what order anything arrives in.
- kjeetgill 8y ago"yolo a latter write probably wins ... I guess?" Seriously, if you're new to that domain go read: https://aphyr.com/posts/333-serializability-linearizability-and-locality https://aphyr.com/posts/333-serializability-linearizability-... https://aphyr.com/posts/313-strong-consistency-models https://aphyr.com/posts/313-strong-consistency-models
- zackmorris 8y agoFor anyone curious about this, it probably all comes down to an atomic compare and swap request: https://en.wikipedia.org/wiki/Compare-and-swap https://en.wikipedia.org/wiki/Compare-and-swap It might be straightforward to do this with an ETag like Firebase: https://firebase.googleblog.com/2017/07/introducing-conditional-rest-requests.html https://firebase.googleblog.com/2017/07/introducing-conditio... Not sure what this would require on their backend though.
- kentonv 8y agoYes, you'll be able to use etags with if-match for optimistic concurrency. When reading and writing from a Worker, if you do a get() followed by a put() on the same key in the context of the same request, we'll take care of the if-match automatically under the hood, so the put() fails if there was a concurrent write. (This logic is not quite in production yet but should be within a week or two.)
- panic 8y agoCan you implement IPC with this system? It seems wrong to relate this to tuple spaces otherwise.
- zackbloom 8y agoPut the target channel name and a message id at the front of each key and you can use a tuple space for IPC.
- panic 8y agoWhat I mean is: can a process block until a particular key appears, then resume execution after receiving the associated value? That's the main differentiating feature of a tuple space, and it wasn't clear to me whether this system supports that or not (or even has a concept of a process).
- kentonv 8y agoNo, that's not supported at present, but that's an interesting idea. Workers don't have the concept of processes. The unit of compute is an individual event -- e.g. handling an individual HTTP request. So we'd have to augment it a bit, but we can probably come up with something useful.
- _pmf_ 8y agoA poor man's IPC, like HTTP and HTML is a poor man's application platform ...