Nathan Marz has been promoting this as the Lambda Architecture, has a couple presentations/blog posts about the idea and is writing a book. While other people p

Nathan Marz has been promoting this as the Lambda Architecture, has a couple presentations/blog posts about the idea and is writing a book. While other people probably have similar ideas, I'm unaware of anybody else attempting to teach them in a similarly cohesive manner. I don't know what I'm doing but I'm working my way through understanding this area so I'll work through the queries as an exercise. For your counter, you'd record the individual votes as tuples of (timestamp, voter, vote, subject). These get dumped into a distributed data store. From there, they get batch processed into the equivalent of a database view. The cascalog query for your 'posts' view would be something like: (defn post-view [post-source vote-source output] (?<- output [?timestamp ?postid ?upvotes ?downvotes ?title ?body] (post-source ?timestamp ?postid ?title ?body) (vote-source _ _ ?vote ?postid) (> ?vote 0 :> !up) (c/!count !up :> ?upvotes) (< ?vote 0 :> !down) (c/!count !down :> ?downvotes))) Produces a set of (timestamp, id, timestamp, upvotes, downvotes, title, body) tuples. You can then do another pass on it to run your sort function: (defn post-score [now-ms time-ms upvotes downvotes] (let [mins-since (div (- now-ms time-ms) (* 60 1000)) (div (- upvotes downvotes) (+ mins-since 1))])) (defn post-rank [post-view-source output] (?<- output [?postid ?votes ?title] (post-view-source ?timestamp ?postid ?upvotes ?downvotes ?title _) (- ?upvotes ?downvotes :> ?votes) (post-score (System/currentTimeMillis) ?timestamp ?upvotes ?downvotes :> ?score) (:sort ?score) (:reverse true)) This would give (id, vote total, title) tuples that you can send to your templating engine (assuming url is based entirely on id) to make the front page. The neat thing about the approach is that you can change everything later. If, for example, you wanted to do weighted upvotes/downvotes, you could produce a scoring function for the up/down votes like the one used here for scoring posts and aggregate the votes using sum instead of count.