6 ms·

I can't wrap my head around the way this solves the global mutable state problem. First, here's what I do understand about databases and global state: compared

• nathanmarz 3y ago
  Rama PStates are globally readable but not globally writable. They are only writable from the topology that declares them. All code writing to PStates is thereby always in the exact same program. Additionally, since PStates are not the source of truth – the depots (event logs) are – mistakes can be corrected via recompute from the source of truth. "Alice's current location" would be done in Rama like this: * Have a depot that receives new locations for people. The appended records would have three keys: userId, location, and timestamp. The depot is partitioned by userId. * Have an ETL with a PState called $$currentLocation that's a map from userId to location. * The ETL consumes the depot and update the PState as new data comes in.
  • fipar 3y ago
    Thanks for the answer. So with a depot being responsible for locations, and partitioned by user id, is is too off to think of this as the way cockroachdb does sharding, with ranges where there's a single raft leader per range? From your description of PStates, I reckon there's some inherent delay from when "an event happens" (bear with me, I know ...), the depot appends it to its log/stream, and a PState consumes it? Let me try to make a concrete example: suppose I'm a consumer that will make a decision based on Alice's location (it could be a media streaming service that must offer a different catalog view depending on the region the user is, even for the same user). What's the Rama way to know how good my location knowledge of Alice is (e.g., "I can be sure I know where Alice was no longer than T time ago")?
    • nathanmarz 3y ago
      When you use stream ETLs, the delay between a depot append and the corresponding PState updates becoming visible is in the single-digit millis range. With microbatching it's at least a few hundred millis. Coordination of understanding whether your writes have propagated to PStates in consuming ETLs is provided at the depot append level. If you do depot appends with full acking enabled (which is the default), the append call doesn't complete until all colocated stream topologies have finished processing that record. So in your client code (using Clojure here for the example), you could do: (foreign-append! locations-depot {:user-id alice-id :location "NYC" :timestamp 12345}) (foreign-select-one (keypath alice-id) current-locations-pstate) That PState query is guaranteed to include the depot write that was done prior.
• cryptonector 3y ago
  > I can't wrap my head around the way this solves the global mutable state problem. That's because you can't get rid of global mutable state. The only thing you can do is try to spread it around to add write concurrency, but that only works for some transactional/concurrency models and schemas. It looks from the other reply like the answer is that the data is sharded to effectively increase write concurrency, but it's still 1 writer per shared.
• bunderbunder 3y ago
  I don't think there's much to wrap your head around. It doesn't seem to. As far as I can tell, this commits its own version of the classic multithreading 101 error of assuming you can make any data structure "thread-safe" by slapping some lock statements around all its getters and setters, or that some relative newcomers to functional programming commit when they believe that all you need to eliminate concurrency problems is the `state` monad, and proceed to Greenspun Haskell into an imperative language by littering it all over their code. The truth is, you can mitigate concurrency problems, but there's no way to eliminate them, and believing that you have is a great way to make them worse.