3 ms·
If I understand that example correctly it takes careful up-front design of event-sources to achieve the desired constraints. E.g. if, for some reason, transact
by benlivengood 3y ago
If I understand that example correctly it takes careful up-front design of event-sources to achieve the desired constraints. E.g. if, for some reason, transactions between users needed to now be initiated from the receiving user (toUser asks for $amt from fromUser) the microbatching would need to be paused and switched to a new microbatch that feeds from the new event source. Does Rama natively support sequencing of these kinds of schema changes if, e.g. you had to replay all the events from scratch to restore the Pstates from scratch. Traditionally, an event-id or timestamp would be used as a demarcation of different aggregation semantics before and after a sentinel value, and I suppose each microbatch could check against sentinel value before proceeding.
In short, though, I struggle to see lower complexity for transactional workloads in Rama v.s. traditional RDBMS.
Do you plan to provide a higher-level abstraction like SQL that internally compiles to correct Rama implementations of Pstate generation?
I think combining ETL and OTP in a single system has a lot of merit, as several RDBMS systems have been adding materialized views to support ETL workflows for that reason. I'm not convinced that low-level index generation is the right solution. I could be convinced that SQL-style high-level schema declarations parsed into malleable low-level operations that can be modified as necessary with solid support for the semantics of point-in-time schema changes could win over traditional RDBMS. For most use-cases the high-level definitions would be enough but reaching under the hood in a reliable and ops-friendly way would avoid the need for an explosion of custom and mutually-vendor-incompatible RDBMS extensions to SQL.
- nathanmarz 3y agoThe transactionality comes from the fact that the depot append is of the entire transaction. So it atomically captures both the debit and credit that happen as a materialization of that record into PStates by the ETL. If you need a different kind of transaction, then that can be done with a different depot or a different event type within the same depot. And the ETL code would be updated to handle that. We won't be providing a SQL -> dataflow compiler for Rama ETLs anytime soon. SQL is way more limited than a general purpose programming language, even with extensions.