4 ms·

> Rama can build end-to-end backends at any scale on its own in a tiny fraction of the code. At its core is a new programming language implementing a new progra

> Rama can build end-to-end backends at any scale on its own in a tiny fraction of the code. At its core is a new programming language implementing a new programming paradigm.. Grand claims, but where are the super powerful demo apps? The getting started docs are a trickle of info.. https://redplanetlabs.com/docs/~/operating-rama.html#_accessing_depots_pstates_and_query_topologies_from_remote_java_client https://redplanetlabs.com/docs/~/operating-rama.html#_access... The "demo gallery" has a few cherry picked examples, but I don't see how this fulfills the claim of a "new paradigm". I wanted to be blown.. but this ain't it. https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/rama-demo-gallery Oof.

nathanmarz 3y ago
Check out twitter-scale-mastodon, which is an implementation of Mastodon's backend from scratch that scales to Twitter scale. It's more than 40% less code than Mastodon's backend and 100x less code than Twitter wrote to build the equivalent. https://github.com/redplanetlabs/twitter-scale-mastodon https://github.com/redplanetlabs/twitter-scale-mastodon https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/ https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the...
- metadat 3y ago
  > Running on a real cluster > Until Rama is publicly available, the only way to get access to the full Rama release is by applying to our private beta. I thought it was coming out last August? What happened? (As per your link, https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/ https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the...) Edit: @nathanmarz that's cool you have enough interest to keep things moving with it being private. I'm only interested in open solutions for this domain, though. IMO, accepting lock-in is a lot to ask. Best wishes ~ MD
  - nathanmarz 3y ago
    We released the public build last August, which can be used to experiment with Rama. Details on that are at the link below. Otherwise, we're still in private beta and access to the full Rama release to run real clusters is just for private beta users. https://redplanetlabs.com/docs/~/downloads-maven-local-dev.html#gsc.tab=0 https://redplanetlabs.com/docs/~/downloads-maven-local-dev.h...
- finnh 3y ago
  > These numbers are a bit better than Twitter’s numbers. Because of how unbalanced the social graph is, getting performance this good and this reliable is not easy. For example, when someone with 20M followers posts a status, that creates a huge burst of load which could delay other statuses from fanning out. How we handled this is described more below. Huh. I haven't written myself a twitter, but I've always assumed that this problem is solved by a mix of fan-on-write and fan-on-read. Most people are fan-on-write, but those with a follower count > $THRESHOLD are fan-on-read. The fan-on-read set is small enough to be easily cacheable, so the extra reads required to service a follower's feed are more than compensated for by the fact that the content is in RAM in a cache. Sounds like RedPanda went exclusively with fan-on-write. I'd be surprised if Twitter does the same - but it sounds like OP has more knowledge about this than me, so I'm happy to be educated!
  - nathanmarz 3y ago
    The chronological timeline at Twitter fans out on write. This makes sense when you consider that the most important application metric is the latency to load the timeline. That latency is a lot lower when you only need one query on the materialized timeline rather than a ton of queries for everyone you follow.
    - finnh 3y ago
      Good to learn, thanks. "Everyone you follow" isn't quite what I was saying, but I also don't use twitter so I'm doubly in the dark here. I'm surprised that someone with, say, a Twitter problem and tens of millions of followers - the two seem to go hand-in-hand - drives tens of millions of writes every time they post. But there you go, learned something today.
      - nathanmarz 3y ago
        Back in the days of the fail whale, Twitter would go down when Justin Bieber tweeted because their system couldn't handle the load.
      - fizx 3y ago
ntonozzi 3y ago
Here's an example of the backend code: https://github.com/redplanetlabs/twitter-scale-mastodon/blob/master/backend/src/main/java/com/rpl/mastodon/modules/Search.java#L359 https://github.com/redplanetlabs/twitter-scale-mastodon/blob... It certainly looks like it does a lot with their DSL, but as a newcomer it's really hard to parse.
- nathanmarz 3y ago
  I suggest starting with the tutorial and the heavily commented examples in rama-demo-gallery, linked below. https://redplanetlabs.com/docs/~/tutorial1.html https://redplanetlabs.com/docs/~/tutorial1.html https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/rama-demo-gallery
  - mintplant 3y ago
    Any examples of the Clojure version?
    - nathanmarz 3y ago
      All the examples in rama-demo-gallery have both Java and Clojure versions, including tests. There's also the introductory blog post for the Clojure API which builds a highly scalable auction application with timed listings, bids, and notifications in 100 LOC. https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-clojure-api/ https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-...
geodel 3y ago
Well there is whole bank transfer implementation there and thoroughly commented too. Some entrepreneur can take this and run with it to create next generation banking platform.
- sokoloff 3y ago
  Maybe they could call their new Clojure-based bank Nubank.
  - erichocean 3y ago
    Obv. Unicorn potential.
cactusfrog 3y ago
The Clojure community is extremely cult like. I worked with a dev who tried to push Datomic, and despite the huge flaws with his system, and the fact that it was closed source, he would continuously tout how much better it was than a relational database. I don’t believe the hype.
- lispisok 3y ago
  You just generalized an entire community based on one person. I can add a counter anecdote to your N=1. I'm a Clojure dev. I've used and enjoyed Datomic. Postgres is still my goto database.
- undershirt 3y ago
  I know our engineering discipline is in shambles, but there are probably better ways to argue against hype without ourselves using hyperbole.
  - hu3 3y ago
    Closed source is enough reason in my book.
  - cactusfrog 3y ago
    I don’t believe I’m exaggerating. You can replace “Clojure” with “Jesus” in the linked article and it still makes sense.
    - gorjusborg 3y ago
      Clojure is not special in that it has some zealots. It's easier to call others out when they aren't in the mainstream, but I'd say that there are way more mainstream cult members than these niche communities. For instance, HN itself can feel a bit cult-like. The greater tech culture feels similarly.
- jabradoodle 3y ago
  N=1, I work with many pragmatic clojure devs who just want to ship working software.
smackeyacky 3y ago
Not even a code sample in the article. You'd think if they were so convinced it was a new paradigm we'd see why. Makes me skeptical.
- nathanmarz 3y ago
  It's an article about Clojure's influence on Rama, not an introductory tutorial on Rama. But we have plenty of code samples elsewhere: https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/twitter-scale-mastodon https://github.com/redplanetlabs/twitter-scale-mastodon https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-clojure-api/ https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-... https://redplanetlabs.com/docs/~/tutorial1.html https://redplanetlabs.com/docs/~/tutorial1.html
kitd 3y ago
Tbf, the purpose of the blog is to talk about the power of Clojure/Lisp for defining the new paradigm, rather than the paradigm itself. Though the author then goes on to talk about "fragments" as the basis for the new paradigm. The first sentence has a link to the project page which has most of the info you need.

nathanmarz 3y ago

Check out twitter-scale-mastodon, which is an implementation of Mastodon's backend from scratch that scales to Twitter scale. It's more than 40% less code than Mastodon's backend and 100x less code than Twitter wrote to build the equivalent. https://github.com/redplanetlabs/twitter-scale-mastodon https://github.com/redplanetlabs/twitter-scale-mastodon https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/ https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the...

metadat 3y ago
> Running on a real cluster > Until Rama is publicly available, the only way to get access to the full Rama release is by applying to our private beta. I thought it was coming out last August? What happened? (As per your link, https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the-cost-of-building-twitter-at-twitter-scale-by-100x/ https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the...) Edit: @nathanmarz that's cool you have enough interest to keep things moving with it being private. I'm only interested in open solutions for this domain, though. IMO, accepting lock-in is a lot to ask. Best wishes ~ MD
- nathanmarz 3y ago
  We released the public build last August, which can be used to experiment with Rama. Details on that are at the link below. Otherwise, we're still in private beta and access to the full Rama release to run real clusters is just for private beta users. https://redplanetlabs.com/docs/~/downloads-maven-local-dev.html#gsc.tab=0 https://redplanetlabs.com/docs/~/downloads-maven-local-dev.h...
finnh 3y ago
> These numbers are a bit better than Twitter’s numbers. Because of how unbalanced the social graph is, getting performance this good and this reliable is not easy. For example, when someone with 20M followers posts a status, that creates a huge burst of load which could delay other statuses from fanning out. How we handled this is described more below. Huh. I haven't written myself a twitter, but I've always assumed that this problem is solved by a mix of fan-on-write and fan-on-read. Most people are fan-on-write, but those with a follower count > $THRESHOLD are fan-on-read. The fan-on-read set is small enough to be easily cacheable, so the extra reads required to service a follower's feed are more than compensated for by the fact that the content is in RAM in a cache. Sounds like RedPanda went exclusively with fan-on-write. I'd be surprised if Twitter does the same - but it sounds like OP has more knowledge about this than me, so I'm happy to be educated!
- nathanmarz 3y ago
  The chronological timeline at Twitter fans out on write. This makes sense when you consider that the most important application metric is the latency to load the timeline. That latency is a lot lower when you only need one query on the materialized timeline rather than a ton of queries for everyone you follow.
  - finnh 3y ago
    Good to learn, thanks. "Everyone you follow" isn't quite what I was saying, but I also don't use twitter so I'm doubly in the dark here. I'm surprised that someone with, say, a Twitter problem and tens of millions of followers - the two seem to go hand-in-hand - drives tens of millions of writes every time they post. But there you go, learned something today.
    - nathanmarz 3y ago
      Back in the days of the fail whale, Twitter would go down when Justin Bieber tweeted because their system couldn't handle the load.
    - fizx 3y ago

ntonozzi 3y ago

Here's an example of the backend code: https://github.com/redplanetlabs/twitter-scale-mastodon/blob/master/backend/src/main/java/com/rpl/mastodon/modules/Search.java#L359 https://github.com/redplanetlabs/twitter-scale-mastodon/blob... It certainly looks like it does a lot with their DSL, but as a newcomer it's really hard to parse.

nathanmarz 3y ago
I suggest starting with the tutorial and the heavily commented examples in rama-demo-gallery, linked below. https://redplanetlabs.com/docs/~/tutorial1.html https://redplanetlabs.com/docs/~/tutorial1.html https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/rama-demo-gallery
- mintplant 3y ago
  Any examples of the Clojure version?
  - nathanmarz 3y ago
    All the examples in rama-demo-gallery have both Java and Clojure versions, including tests. There's also the introductory blog post for the Clojure API which builds a highly scalable auction application with timed listings, bids, and notifications in 100 LOC. https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-clojure-api/ https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-...

geodel 3y ago

Well there is whole bank transfer implementation there and thoroughly commented too. Some entrepreneur can take this and run with it to create next generation banking platform.

sokoloff 3y ago
Maybe they could call their new Clojure-based bank Nubank.
- erichocean 3y ago
  Obv. Unicorn potential.

cactusfrog 3y ago

The Clojure community is extremely cult like. I worked with a dev who tried to push Datomic, and despite the huge flaws with his system, and the fact that it was closed source, he would continuously tout how much better it was than a relational database. I don’t believe the hype.

lispisok 3y ago
You just generalized an entire community based on one person. I can add a counter anecdote to your N=1. I'm a Clojure dev. I've used and enjoyed Datomic. Postgres is still my goto database.
undershirt 3y ago
I know our engineering discipline is in shambles, but there are probably better ways to argue against hype without ourselves using hyperbole.
- hu3 3y ago
  Closed source is enough reason in my book.
- cactusfrog 3y ago
  I don’t believe I’m exaggerating. You can replace “Clojure” with “Jesus” in the linked article and it still makes sense.
  - gorjusborg 3y ago
    Clojure is not special in that it has some zealots. It's easier to call others out when they aren't in the mainstream, but I'd say that there are way more mainstream cult members than these niche communities. For instance, HN itself can feel a bit cult-like. The greater tech culture feels similarly.
jabradoodle 3y ago
N=1, I work with many pragmatic clojure devs who just want to ship working software.

smackeyacky 3y ago

Not even a code sample in the article. You'd think if they were so convinced it was a new paradigm we'd see why. Makes me skeptical.

nathanmarz 3y ago
It's an article about Clojure's influence on Rama, not an introductory tutorial on Rama. But we have plenty of code samples elsewhere: https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/rama-demo-gallery https://github.com/redplanetlabs/twitter-scale-mastodon https://github.com/redplanetlabs/twitter-scale-mastodon https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-clojure-api/ https://blog.redplanetlabs.com/2023/10/11/introducing-ramas-... https://redplanetlabs.com/docs/~/tutorial1.html https://redplanetlabs.com/docs/~/tutorial1.html

kitd 3y ago

Tbf, the purpose of the blog is to talk about the power of Clojure/Lisp for defining the new paradigm, rather than the paradigm itself. Though the author then goes on to talk about "fragments" as the basis for the new paradigm. The first sentence has a link to the project page which has most of the info you need.