nathanmarz

7 ms·

31.
by nathanmarz 2y ago
What we've released publicly on our public Maven repository is a different build of Rama which can only be used for testing/experimentation in a single process. So it can't be used to run full clusters. It's API-equivale
32.
by nathanmarz 2y ago
Well, this article is to help people understand just Rama's dataflow API, as opposed to an introduction to Rama for backend development. Rama does have a learning curve. If you think its API is "clunky", then you just haven&#
33.
by nathanmarz 2y ago
Actually, we've eliminated a massive amount of the complexity of backend development. This is most pronounced at large scale, but it's true at small scale as well. Our Twitter-scale Mastodon example is literally 100x less code tha
34.
by nathanmarz 2y ago
The examples in this repository are representative of what it's like to use Rama for different kinds of use cases: https://github.com/redplanetlabs/rama-demo-gallery
35.
Rama on Clojure's terms, and the magic of continuation-passing style(blog.redplanetlabs.com)
106 points by nathanmarz 2y ago|73 comments
36.
Rama on Clojure's terms, and the magic of continuation-passing style(blog.redplanetlabs.com)
4 points by nathanmarz 2y ago|0 comments
37.
Migrating terabytes of data instantly (can your ALTER TABLE do this?)(blog.redplanetlabs.com)
2 points by nathanmarz 2y ago|0 comments
38.
by nathanmarz 2y ago
Fragments have nothing to do with actors. Check out the doc on the dataflow language and you'll clearly see it's a generalization of a function. https://redplanetlabs.com/docs/~/clj-dataflow-lang.html
39.
by nathanmarz 2y ago
The best documentation on the mental model of using Rama is the last page of the tutorial, linked below. However, I would recommend going through the whole tutorial rather than starting there. https://redplanetlabs.com/docs&
40.
by nathanmarz 2y ago
It's named after the Arthur C. Clarke book.
41.
by nathanmarz 2y ago
It's an article about Clojure's influence on Rama, not an introductory tutorial on Rama. But we have plenty of code samples elsewhere: https://github.com/redplanetlabs/rama-demo-gallery https://git
42.
by nathanmarz 2y ago
All the examples in rama-demo-gallery have both Java and Clojure versions, including tests. There's also the introductory blog post for the Clojure API which builds a highly scalable auction application with timed listings, bids, and n
43.
by nathanmarz 2y ago
Indeed, we're in private beta and aren't publicizing much about what we're doing. We'll eventually be releasing many case studies on how our private beta users are using Rama.
44.
by nathanmarz 2y ago
A fragment itself is a generic programming construct that serves the same purpose as a function (just more general). When used in Rama topologies, they serve a similar role as observables in terms of reacting to new data as it flows through
45.
by nathanmarz 2y ago
Back in the days of the fail whale, Twitter would go down when Justin Bieber tweeted because their system couldn't handle the load.
46.
by nathanmarz 2y ago
The chronological timeline at Twitter fans out on write. This makes sense when you consider that the most important application metric is the latency to load the timeline. That latency is a lot lower when you only need one query on the mate
47.
by nathanmarz 2y ago
I suggest starting with the tutorial and the heavily commented examples in rama-demo-gallery, linked below. https://redplanetlabs.com/docs/~/tutorial1.html https://github.com/redplanetlabs/ram
48.
by nathanmarz 2y ago
We released the public build last August, which can be used to experiment with Rama. Details on that are at the link below. Otherwise, we're still in private beta and access to the full Rama release to run real clusters is just for pri
49.
by nathanmarz 2y ago
A few of our private beta users are using Rama at larger scale (>1M appends/second). We'll be publishing case studies eventually.
50.
by nathanmarz 2y ago
Check out twitter-scale-mastodon, which is an implementation of Mastodon's backend from scratch that scales to Twitter scale. It's more than 40% less code than Mastodon's backend and 100x less code than Twitter wrote to buil
51.
by nathanmarz 3y ago
This example uses microbatching for the processing, so the latency will be ~200 millis. You don't need to poll and could set this up with a reactive PState query to know when the transaction is done. 200 millis is an acceptable latency
52.
by nathanmarz 3y ago
You need all four in the context of databases and the architectures in which they live, where computation is separate from storage and concurrent updates are possible. In the context of Rama it's different, where computation is colocat
53.
by nathanmarz 3y ago
Mastodon is extremely similar to the Twitter consumer product. The differences are minor. We compare with Twitter because unlike Mastodon Twitter runs at large scale. Yes, we implemented the entirety of Mastodon from scratch.
54.
by nathanmarz 3y ago
A good idea with a flawed implementation is still a good idea
55.
by nathanmarz 3y ago
This stuff is all covered thoroughly in our docs. This is a post about the complexity aspect of backend development and how those are addressed with Rama. It's not a thorough explanation of every aspect of Rama, since that would be ext
56.
by nathanmarz 3y ago
Sounds like you've engineered a great way to manage complexity while using an RDBMS. A few things that Rama provides above this: * Rama's indexing is much more flexible. For example, if you need to have a nested set with 100M elem
57.
by nathanmarz 3y ago
The transactionality comes from the fact that the depot append is of the entire transaction. So it atomically captures both the debit and credit that happen as a materialization of that record into PStates by the ETL. If you need a differen
58.
by nathanmarz 3y ago
Those concepts are relevant to databases like RDBMS's where you have multiple writes happening concurrently to the same index. Rama doesn't work like that – you get parallelism by having multiple partitions, writes on a single par
59.
by nathanmarz 3y ago
I consider a DSL something that has its own lexer/parser, like SQL. Since Rama's dataflow API is just Java (there's also a Clojure API, btw), that means you never leave the realm of a general purpose programming language. So
60.
by nathanmarz 3y ago
When you use stream ETLs, the delay between a depot append and the corresponding PState updates becoming visible is in the single-digit millis range. With microbatching it's at least a few hundred millis. Coordination of understanding

More ›