4 ms·
Rama is a much broader platform than a database, so the consistency semantics you get depend on how you use it. When using Rama, you're not mutating indexes dir
by nathanmarz 3y ago
Rama is a much broader platform than a database, so the consistency semantics you get depend on how you use it. When using Rama, you're not mutating indexes directly like you do with a database, but adding source data that then gets materialized into any number of indexes.
You get read-after-write consistency for any PStates in a streaming ETL colocated with the depot you appended to. This is if you do the depot append with "full acking", which coordinates its response with the completion of colocated streaming ETLs. If you append at a lower level of acking, then you get eventual consistency on those PStates at the benefit of lower latency appends.
Microbatching is always eventually consistent as processing is asynchronous and uncoordinated to depot appends. Microbatching is higher thorughput than streaming and has simpler fault-tolerance semantics.
You'll be able to read a lot more about this when we release the docs next week.
- datavirtue 3y agoI would like your newsletter.
- chubot 3y agoOK, thanks for the response I think many people are going to have problems programming with this consistency model, as they will with any that's different than a single machine. But that's basically "physics", so it's inevitable :) But it seems like great work within the constraints -- look forward to learning more I have indeed wondered why none of the cloud platforms have built more forward-looking tech like this -- instead it's copies of AWS and so forth