5 ms·

You're missing the point. Rama is a generic platform that provides a new baseline for how expensive it is to build applications at scale. There's nothing about

You're missing the point. Rama is a generic platform that provides a new baseline for how expensive it is to build applications at scale. There's nothing about Rama specific to social networks. What we're showing is that Rama creates a new era in software engineering where the cost of building applications at scale is radically reduced. With Rama, anyone embarking on a new application today has a radically different economic outlook for the end-to-end cost of developing that application from prototype through large scale.

yid 3y ago
> What we're showing is that Rama creates a new era in software engineering where the cost of building applications at scale is radically reduced. Bold of you to come to HN with the breathless hyperbolic marketing fluff that may work on Twitter...
- nathanmarz 3y ago
  I think we provided a ton of substance backing up that claim, and we will provide even more next week when we release the build of Rama that anyone can use and its corresponding 100k words of documentation.
3cats-in-a-coat 3y ago
If I grasp the essence of Rama: - "Depots" are event streams (for event sourced data repositories) - ETL read one or more streams and project them to indexable read models... - Which read models are called "PStates" and represent nested combinations of indices like hashtables, b-trees, linked lists and so on. The point of those being they have the data in fast to query way. - And you have query engine which splits a query into 1+ index sub-queries and then aggregates. Am I missing something, this seems relatively standard event-sourced / CQRS-like architecture, but streamlined to avoid redundancy and reimplementation of common abstractions. It would've helped if the terms were less obscure than "depots" and "PStates".
- dustingetz 3y ago
  you're missing automatic/free linear scaling
  - 3cats-in-a-coat 3y ago
    Systems that promise "free linear scaling" without qualifiers either withhold or have not analyzed/realized their bottlenecks yet. Say if there is eventual consistency maybe the "eventuality" becomes so long that the service fails at its purpose. Or the communication link bandwidth is exhausted between key business logic (mutation event generating) services, and so on. The only systems that scale linearly are stateless systems. Mastodon is not stateless. And even stateless systems hit some bottlenecks eventually, as they exist and run in a scale-variant Universe. So this claim by itself doesn't immediately impress me, just turns my red lights on, awaiting further investigation. But we can of course discuss why this claim is made and how is it supported. The article is long so I've not had the chance to read it entirely yet. But we have X number of event streams mapped through Y number of ETLs to produce Z number of read model indices, in a shape that seems to form a highly interlinked DAG, which eventually loops back on itself in terms of message flow. Just the increased cross-chatter here as we introduce more features suggests non-linear scaling.
    - dustingetz 3y ago
      for example it can scale the way persistent data structures scale, which is to say "O(1) within target operational bounds" despite technically being log-n with high branch factor)
- nathanmarz 3y ago
  From the post: Individually, none of these concepts are new. I’m sure you’ve seen them all before. You may be tempted to dismiss Rama’s programming model as just a combination of event sourcing and materialized views. But what Rama does is integrate and generalize these concepts to such an extent that you can build entire backends end-to-end without any of the impedance mismatches or complexity that characterize and overwhelm existing systems. You have the general model correct, but here are a few clarifications: - PStates are partitioned, durable, replicated indexes that are represented as arbitrary combinations of data structures. A PState can be as simple an an integer per partition, or it can be complex like a map of lists of maps of sets. PStates allow you to shape your indexes to perfectly match your application's use cases. - I wouldn't call Rama queries an "engine", as it's considerably more straightforward in how it works than something like SQL. The base query API is called "paths", which are an imperative way to concisely reach into one partition of one PState to fetch or aggregate values. There's also "query topologies" which are predefined, on-demand distributed computations that can fetch and aggregate data from many partitions of many PStates.

yid 3y ago

> What we're showing is that Rama creates a new era in software engineering where the cost of building applications at scale is radically reduced. Bold of you to come to HN with the breathless hyperbolic marketing fluff that may work on Twitter...

nathanmarz 3y ago
I think we provided a ton of substance backing up that claim, and we will provide even more next week when we release the build of Rama that anyone can use and its corresponding 100k words of documentation.

3cats-in-a-coat 3y ago

If I grasp the essence of Rama: - "Depots" are event streams (for event sourced data repositories) - ETL read one or more streams and project them to indexable read models... - Which read models are called "PStates" and represent nested combinations of indices like hashtables, b-trees, linked lists and so on. The point of those being they have the data in fast to query way. - And you have query engine which splits a query into 1+ index sub-queries and then aggregates. Am I missing something, this seems relatively standard event-sourced / CQRS-like architecture, but streamlined to avoid redundancy and reimplementation of common abstractions. It would've helped if the terms were less obscure than "depots" and "PStates".

dustingetz 3y ago
you're missing automatic/free linear scaling
- 3cats-in-a-coat 3y ago
  Systems that promise "free linear scaling" without qualifiers either withhold or have not analyzed/realized their bottlenecks yet. Say if there is eventual consistency maybe the "eventuality" becomes so long that the service fails at its purpose. Or the communication link bandwidth is exhausted between key business logic (mutation event generating) services, and so on. The only systems that scale linearly are stateless systems. Mastodon is not stateless. And even stateless systems hit some bottlenecks eventually, as they exist and run in a scale-variant Universe. So this claim by itself doesn't immediately impress me, just turns my red lights on, awaiting further investigation. But we can of course discuss why this claim is made and how is it supported. The article is long so I've not had the chance to read it entirely yet. But we have X number of event streams mapped through Y number of ETLs to produce Z number of read model indices, in a shape that seems to form a highly interlinked DAG, which eventually loops back on itself in terms of message flow. Just the increased cross-chatter here as we introduce more features suggests non-linear scaling.
  - dustingetz 3y ago
    for example it can scale the way persistent data structures scale, which is to say "O(1) within target operational bounds" despite technically being log-n with high branch factor)
nathanmarz 3y ago
From the post: Individually, none of these concepts are new. I’m sure you’ve seen them all before. You may be tempted to dismiss Rama’s programming model as just a combination of event sourcing and materialized views. But what Rama does is integrate and generalize these concepts to such an extent that you can build entire backends end-to-end without any of the impedance mismatches or complexity that characterize and overwhelm existing systems. You have the general model correct, but here are a few clarifications: - PStates are partitioned, durable, replicated indexes that are represented as arbitrary combinations of data structures. A PState can be as simple an an integer per partition, or it can be complex like a map of lists of maps of sets. PStates allow you to shape your indexes to perfectly match your application's use cases. - I wouldn't call Rama queries an "engine", as it's considerably more straightforward in how it works than something like SQL. The base query API is called "paths", which are an imperative way to concisely reach into one partition of one PState to fetch or aggregate values. There's also "query topologies" which are predefined, on-demand distributed computations that can fetch and aggregate data from many partitions of many PStates.