4 ms·

What's an ELI5 of Rama? I found the docs confusing as well: https://redplanetlabs.com/docs/~/index.html https://redplanetlabs.com/docs/~/index.html Please no b

What's an ELI5 of Rama? I found the docs confusing as well: https://redplanetlabs.com/docs/~/index.html https://redplanetlabs.com/docs/~/index.html Please no buzzwords like "paradigm shift" or "platform". If diagrams are necessary, I'd love to read a post that explains clearer.

pi-e-sigma 3y ago
Seems like another attempt at No-SQL. "But this time it's different!"
- fulafel 3y ago
  That's how things typically take off, not on the first attempt. Depends on what's different this time. (Though NoSQL has outlived its usefulness as a concept IMO, it's just too loose to be useful beyond its early use for "something like CouchDB/Mongo", which this is clearly not)
- nineteen999 3y ago
  Negative SQL?
  - deleted 3y ago
    [deleted]
- archarios 3y ago
  This is completely different from no-sql. Much more than just a different database..
  - pi-e-sigma 3y ago
    How exactly is it different from No-SQL? No schema? Check. No consistency? Check. (eventually consistent) Key-value store? Check. (because it's using ZooKeeper under the hood) Promising amazing results and freeing you from the chains of the SQL? CHECK!
    - nathanmarz 3y ago
      Everything you wrote here is false, with the exception of Rama not using SQL. Rama has strong schemas, is strongly consistent, and is not limited to key/value (PStates can be any data structure combination). Zookeeper is used only for cluster metadata and is not involved with user storage/processing in any way.
fwip 3y ago
By my reading, it's a variant of the "Kappa architecture" (aka "event sourcing"). You have a "Depot", which is an append-only log of events, and then build arbitrary views on top of it, which they call "P-States". The Rama software promises low-latency updates of these views. Applications built on this would query the views, and submit new events/commands to the Depot.
nathanmarz 3y ago
It's a backend development platform that can handle all the data ingestion, processing, indexing, and querying needs of an application, at any scale. Rather than construct your backend using a hodgepodge of databases, processing systems, queues, and schedulers, you can do everything within Rama within a single platform. Rama runs as a cluster, and any number of applications (called "modules") are deployed onto that cluster. Deep and detailed telemetry is also built-in. The programming model of Rama is event sourcing plus materialized views. When building a Rama application, you materialize as many indexes as you need as whatever shapes you need (different combinations of durable data structures). Indexes are materialized using a distributed dataflow API. Since Rama is so different than anything that's existed before, that's about as good of a high-level explanation as I can do. The best resource for learning the basics is rama-demo-gallery, which contains short, end-to-end, thoroughly commented examples of applying Rama towards very different use cases (all completely scalable and fault-tolerant): https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/rama-demo-gallery
- Faark 3y ago
  I like this description. Most one point one i've seen in the thread and you doc. So its not really a tool to use, but more of framework to follow. Wouldn't be the first framework to provide tools / setup processes and workflows in a better than ever tradeoff of features/complexity/skill floor/etc. But yeah, quite a lot of hype and red flags. My favorite from the website: "Rama is programmed entirely with a Java API – no custom languages or DSLs." And when you look at the example BankTransferModule.java: > .ifTrue("isSuccess", Block.localTransform("$$funds", Path.key("toUserId").nullToVal(0).term(Ops.PLUS, "*amt"))) Yeah, it's probably fair to call that a DSL, even if entirly java. Anyway, hope to get the chance to work with event based systems one day and who knows, maybe it will be Rama.
  - nathanmarz 3y ago
    I consider a DSL something that has its own lexer/parser, like SQL. Since Rama's dataflow API is just Java (there's also a Clojure API, btw), that means you never leave the realm of a general purpose programming language. So you can do things higher-order things like generate dataflow code dynamically, factor reusable code into normal Java functions, and so on. And all these are done without the complexity and risks of doing so by generating strings for a separate DSL, like you get when generating SQL (e.g. injection attacks).
morsecodist 3y ago
It seems like an event sourcing database. Basically, instead of writing you write a message, then you can make read-only tables that update based on those messages. People do this today in certain domains but it is definitely more complicated than traditional databases.
- archarios 3y ago
  More complicated in what ways specifically? I think the relevant thing is wether building an app with Rama is more or less complicated. Rama may be more complicated than mysql in implementation, but that doesn't affect me as a developer if it makes my job easier overall.
  - morsecodist 3y ago
    Discussing levels of complexity quickly gets pretty subjective. It is possible that Rama has found good abstractions that hide a lot of the complexity. It is also possible that taking on more complexity in this area saves you from other sorts of complexity you may encounter elsewhere in your application. However, there is just more going on in an event sourcing model. Instead of saving data to a location and retrieving it from that location you save data to one location, read it from another location, and you need to implement some sort of linker between the two (or more). This also comes down to my personal subjective experience. I actually really like event sourcing but I have worked on teams with these systems and I have found that the majority of people find them much harder to reason about than traditional databases.
    - nathanmarz 3y ago
      There can be a lot of integration pain when implementing event sourcing and materialized views by combining individual tools together. However, these are all integrated in Rama, so there's nothing you have to glue yourself as a developer. For example, using the Clojure API here's how you declare a depot (an event log): (declare-depot setup *my-events (hash-by :user-id)) That's it, and you can make as many of those as you want. And here's how a topology (a streaming computation that materializes indexes based on depots) subscribes to that depot: (source> my-events :> *data) If you want to subscribe to more depots in the topology, then it's just another source> call. That these are integrated and colocated also means the performance is excellent.

pi-e-sigma 3y ago

Seems like another attempt at No-SQL. "But this time it's different!"

fulafel 3y ago
That's how things typically take off, not on the first attempt. Depends on what's different this time. (Though NoSQL has outlived its usefulness as a concept IMO, it's just too loose to be useful beyond its early use for "something like CouchDB/Mongo", which this is clearly not)
nineteen999 3y ago
Negative SQL?
- deleted 3y ago
  [deleted]
archarios 3y ago
This is completely different from no-sql. Much more than just a different database..
- pi-e-sigma 3y ago
  How exactly is it different from No-SQL? No schema? Check. No consistency? Check. (eventually consistent) Key-value store? Check. (because it's using ZooKeeper under the hood) Promising amazing results and freeing you from the chains of the SQL? CHECK!
  - nathanmarz 3y ago
    Everything you wrote here is false, with the exception of Rama not using SQL. Rama has strong schemas, is strongly consistent, and is not limited to key/value (PStates can be any data structure combination). Zookeeper is used only for cluster metadata and is not involved with user storage/processing in any way.

fwip 3y ago

By my reading, it's a variant of the "Kappa architecture" (aka "event sourcing"). You have a "Depot", which is an append-only log of events, and then build arbitrary views on top of it, which they call "P-States". The Rama software promises low-latency updates of these views. Applications built on this would query the views, and submit new events/commands to the Depot.

nathanmarz 3y ago

It's a backend development platform that can handle all the data ingestion, processing, indexing, and querying needs of an application, at any scale. Rather than construct your backend using a hodgepodge of databases, processing systems, queues, and schedulers, you can do everything within Rama within a single platform. Rama runs as a cluster, and any number of applications (called "modules") are deployed onto that cluster. Deep and detailed telemetry is also built-in. The programming model of Rama is event sourcing plus materialized views. When building a Rama application, you materialize as many indexes as you need as whatever shapes you need (different combinations of durable data structures). Indexes are materialized using a distributed dataflow API. Since Rama is so different than anything that's existed before, that's about as good of a high-level explanation as I can do. The best resource for learning the basics is rama-demo-gallery, which contains short, end-to-end, thoroughly commented examples of applying Rama towards very different use cases (all completely scalable and fault-tolerant): https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/rama-demo-gallery

Faark 3y ago
I like this description. Most one point one i've seen in the thread and you doc. So its not really a tool to use, but more of framework to follow. Wouldn't be the first framework to provide tools / setup processes and workflows in a better than ever tradeoff of features/complexity/skill floor/etc. But yeah, quite a lot of hype and red flags. My favorite from the website: "Rama is programmed entirely with a Java API – no custom languages or DSLs." And when you look at the example BankTransferModule.java: > .ifTrue("isSuccess", Block.localTransform("$$funds", Path.key("toUserId").nullToVal(0).term(Ops.PLUS, "*amt"))) Yeah, it's probably fair to call that a DSL, even if entirly java. Anyway, hope to get the chance to work with event based systems one day and who knows, maybe it will be Rama.
- nathanmarz 3y ago
  I consider a DSL something that has its own lexer/parser, like SQL. Since Rama's dataflow API is just Java (there's also a Clojure API, btw), that means you never leave the realm of a general purpose programming language. So you can do things higher-order things like generate dataflow code dynamically, factor reusable code into normal Java functions, and so on. And all these are done without the complexity and risks of doing so by generating strings for a separate DSL, like you get when generating SQL (e.g. injection attacks).

morsecodist 3y ago

It seems like an event sourcing database. Basically, instead of writing you write a message, then you can make read-only tables that update based on those messages. People do this today in certain domains but it is definitely more complicated than traditional databases.

archarios 3y ago
More complicated in what ways specifically? I think the relevant thing is wether building an app with Rama is more or less complicated. Rama may be more complicated than mysql in implementation, but that doesn't affect me as a developer if it makes my job easier overall.
- morsecodist 3y ago
  Discussing levels of complexity quickly gets pretty subjective. It is possible that Rama has found good abstractions that hide a lot of the complexity. It is also possible that taking on more complexity in this area saves you from other sorts of complexity you may encounter elsewhere in your application. However, there is just more going on in an event sourcing model. Instead of saving data to a location and retrieving it from that location you save data to one location, read it from another location, and you need to implement some sort of linker between the two (or more). This also comes down to my personal subjective experience. I actually really like event sourcing but I have worked on teams with these systems and I have found that the majority of people find them much harder to reason about than traditional databases.
  - nathanmarz 3y ago
    There can be a lot of integration pain when implementing event sourcing and materialized views by combining individual tools together. However, these are all integrated in Rama, so there's nothing you have to glue yourself as a developer. For example, using the Clojure API here's how you declare a depot (an event log): (declare-depot setup *my-events (hash-by :user-id)) That's it, and you can make as many of those as you want. And here's how a topology (a streaming computation that materializes indexes based on depots) subscribes to that depot: (source> my-events :> *data) If you want to subscribe to more depots in the topology, then it's just another source> call. That these are integrated and colocated also means the performance is excellent.