11 ms·

Reddit 1.0 source code

251 points by dchest 9y ago

forkLding 9y ago
Is Reddit 2.0 the one in Python? Was slightly confused and expecting Python when opening up the github repo.
- SquareWheel 9y ago
  Reddit was lisp, then python, then python, then React (and some other stuff).
  - grzm 9y ago
    Can you expand the last one? The first three are server-side, while the last is front end. Are they using node on the back end?
    - Figs 9y ago
      Recent post about it: https://www.reddit.com/r/announcements/comments/8830oa/and_now_a_word_from_reddits_engineers/ https://www.reddit.com/r/announcements/comments/8830oa/and_n...
      - grzm 9y ago
        Cheers. Looks like this most recent work was primarily front-end: > "When we set out to rewrite our code to solve these problems, we wanted to make sure we weren't just fixing small, isolated issues but creating a new, more modern frontend stack..."
  - colejohnson66 9y ago
    Why’d you list Python twice?
    - grzm 9y ago
      The second rewrite was also in Python.
  - pknopf 9y ago
    There UI doesn't look like it is using React. Maybe they are still working on it?
    - SquareWheel 9y ago
      It's still in development. Available at alpha.reddit.com, but it may not be available to all yet.
notthemessiah 9y ago
It's unfortunate that Reddit silently went closed-source before being forced to admit it. Certainly a betrayal of trust with their long-standing userbase.
- Mononokay 9y ago
  The majority of their code is still open-source. https://github.com/reddit https://github.com/reddit
  - andrewguenther 9y ago
    It is definitely not the majority and doesn't match what they're running on the back-end anymore at all.
  - Sylos 9y ago
    That's like someone telling you the majority of ingredients that they put into something you're supposed to eat, but not that one ingredient. It's either their secret super sauce or it's poison. And I can tell you for free that whatever Reddit has built technologically, does not contain any secret super sauce whatsoever.
    - chme 9y ago
      I sort of get reddits reason to hold the post ranking algorithm secret. So that it is hard to game it. Until there is a way to easy 're-key' such an algorithm if someone found a way to abuse it, holding it secret is the best solution for them. I currently can't think of a better solution to avoid gaming of ranking algorithms than holding it secret and changing/adapting it often. Maybe some machine learning algorithm?
- confounded 9y ago
  Yup. I understand the scale and micro services thing, but they could at least keep the mobile clients open (even if with a lagged release for feature launches, or just periodic tarballs).
CodeArtisan 9y ago
Why Reddit has been rewritten in python: https://redditblog.com/2005/12/05/on-lisp/ https://redditblog.com/2005/12/05/on-lisp/ http://www.aaronsw.com/weblog/rewritingreddit http://www.aaronsw.com/weblog/rewritingreddit
- pknopf 9y ago
  What is it like maintaining a large project in such a dynamic language? I guess if you have proper unit tests and good coverage, you could quickly find any runtime bugs that you may unintentionally introduce with refactors/changes. When in JavaScript, I'd always lean on TypeScript. Is there something similar for Python?
  - hguant 9y ago
    Python 3.6 has typing/type annotations. It's opt in but seems to work fairly sanely. https://medium.com/@ageitgey/learn-how-to-use-static-type-checking-in-python-3-6-in-10-minutes-12c86d72677b https://medium.com/@ageitgey/learn-how-to-use-static-type-ch...
nodesocket 9y ago
Never played around with Lisp, so excuse the ignorance. Is this typical to construct HTML in Lisp? This feels incredibly verbose and error prone, not to mention confusing and hard to grok. Good luck having a designer mockup/write HTML. (defun login-panel () (pbox "login/register" (:form :id "logform" :class "nomargin" (:table :style "border-collapse: collapse" (:tr (:td :colspan "2" "username:")) (:tr (:td :colspan "2" (:input :id "loguser" :class "txt" :name "user" :type "text" :size 15))) (:tr (:td :colspan "2" "password:")) (:tr (:td :colspan "2" (:input :id "logpass" :class "txt" :name "pass" :type "password" :size 15))) (:tr (:td :colspan "2" (:input :id "logmem" :type "checkbox" :name "mem" "remember me"))) (:tr (:td :colspan "2" (:span :id "logerror" :class "error" ""))) (:tr (:td (:input :id "logbtn" :class "btn" :type "submit" :value "Login" :onclick "login(); return false")) (:td (:input :class "btn" :type "submit" :value "Register" :onclick "register(); return false"))) (:tr (:td :nowrap t :colspan "2" :align "center" :class "little" (:a :href "/password" "what's my password?")))))))
- grzm 9y ago
  It's not uncommon. An example in Racket is html-template.[0] Hiccup is one in Clojure.[1] There's a spectrum of methods for doing things like this: both Clojure and Racket have Mustache[2] implementations. "Confusing and hard to grok"† is in the eye of the beholder. Syntax highlighting and indentation go a long way (as well as exposure), though even in black and white, the parens fade for me much more than angle brackets do. Personally, I'd wrap some of the longer lines, but that's a style thing. [0]: https://docs.racket-lang.org/html-template/index.html https://docs.racket-lang.org/html-template/index.html. [1]: https://github.com/weavejester/hiccup https://github.com/weavejester/hiccup [2]: https://mustache.github.io/mustache.5.html https://mustache.github.io/mustache.5.html † Sorry. Couldn't let the typo stand. Heinlein made me do it.
  - spapas82 9y ago
    Clojure also has Selmer (https://github.com/yogthos/Selmer https://github.com/yogthos/Selmer) which is a Django inspired templating language!
1wd 9y ago
What kind of Lisp system was used to run this? Would it work with any Common Lisp? What other things are required, like libraries or a database etc.? I'm guessing reddit.asd lists some: :tbnl (a toolkit for building dynamic websites) :cl-ppcre (a regex library) :trivial-http (a HTTP client) :cl-who (a library for using S-expressions as HTML markup templates) :clsql (a SQL database interface and ORM) :clsql-postgresql (a PostgreSQL connector) :cl-smtp (a SMTP client for sending email) :ironclad (a cryptography library)
- chuchana 9y ago
  CMUCL on FreeBSD [https://redditblog.com/2005/12/05/on-lisp/ https://redditblog.com/2005/12/05/on-lisp/]
- lispm 9y ago
  SBCL
- adwf 9y ago
  The interesting thing is you can still use pretty much that entire stack today. The only one that stands out as outdated is tbnl which changed name to hunchentoot. The stack I used for a website just the other day: hunchentoot cl-who postmodern (prefer it over clsql-postgresql) cl-smtp ironclad drakma (fuller featured than trivial-http)
deleted 9y ago
[deleted]
lifthrasiir 9y ago
I chuckled over this bit of lines [1]. Fortunately (for Reddit) this name is being consistently used throughout the code. Can't guess why. (email :type string :accessor user-emai) [1] https://github.com/reddit-archive/reddit1.0/blob/bb4fbdb5871a32709df30540685cf44c637bf203/view-defs.lisp#L32-L34 https://github.com/reddit-archive/reddit1.0/blob/bb4fbdb5871...
- vzcx 9y ago
  Heh. If you see [1], they define a function called "user-email" (spelled correctly). The lisp convention here would be to define the accessor as something like "%user-email." While the code isn't too bad, these little things betray a lack of knowledge of lisp lore. Perhaps another reason for the rewrite was that the team actually didn't have much experience writing lisp code. [1] https://github.com/reddit-archive/reddit1.0/blob/bb4fbdb5871a32709df30540685cf44c637bf203/data.lisp#L336 https://github.com/reddit-archive/reddit1.0/blob/bb4fbdb5871...
  - kazinator 9y ago
    https://github.com/Clozure/ccl/blob/master/lib/case-error.lisp#L51 https://github.com/Clozure/ccl/blob/master/lib/case-error.li... %assertion-failure doesn't look like an accessor. http://clhs.lisp.se/Body/f_car_c.htm http://clhs.lisp.se/Body/f_car_c.htm car described as "accessor", not named %car. No public function in CL is named this way. In fact, the character % doesn't make an appearance at all: http://www.lispworks.com/documentation/lw50/CLHS/Front/X_Perm_9.htm http://www.lispworks.com/documentation/lw50/CLHS/Front/X_Per... I've never named a function or macro %something in 18 years of Lisp.
    - vzcx 9y ago
      Sure, sure. But while it's definitely not in the standard, I do see it all over, and it's something I picked up from reading a lot of other people's lisp. SBCL internals, for example. You might not like the convention and chose not to follow along with it, but I would be surprised if, after 18 years of lisp, you had never seen it before, and would choose to misspell an accessor to prevent a clash instead of naming it differently. Edit: ah, I think I understand. I don't mean to imply that all accessors should be named in this way. That would be gross.
melling 9y ago
Why Reddit switched to Python: https://news.ycombinator.com/item?id=3815491 https://news.ycombinator.com/item?id=3815491 Of course HN is written in Lisp. pg is a big fan: http://www.paulgraham.com/lisp.html http://www.paulgraham.com/lisp.html
- bringtheaction 9y ago
  Thanks, was about to ask what made them switch.
nathell 9y ago
It's fun to look at the ASDF system definition: https://github.com/reddit-archive/reddit1.0/blob/master/reddit.asd https://github.com/reddit-archive/reddit1.0/blob/master/redd... Some Ediware in the dependencies (CL-PPCRE, CL-WHO and Hunchentoot from back when it was called TBNL). I guess they used CL-SQL rather than Postmodern because the latter didn't exist back then. Should be reasonably easy to get it running on a modern CL.
rurban 9y ago
With mail username and password
Mister_Snuggles 9y ago
Is this really all the source code? It seems to be missing the database schema, which is pretty critical to using it. Beyond that, this is actually fascinating to look at. It's small enough that you can actually understand it, but it's more complete than a simple 'toy' application. I'm also fascinated to see how SQL constructs are expressed in LISP. I don't know LISP at all, but it's pretty obvious how the queries in the language map directly to SQL.
kjdsiheioahef 9y ago
Unrelated to the original post: Does anyone know if Reddit keeps or shares records of content that ends up on the (unauthenticated, uncustomized) homepage (top 30 results at any given time) for US-based users? If so, how far back does such data go? I don't believe the mainstream archive sites would be a definitive source. Perhaps there is another?

251 points by dchest 9y ago

forkLding 9y ago

Is Reddit 2.0 the one in Python? Was slightly confused and expecting Python when opening up the github repo.

SquareWheel 9y ago
Reddit was lisp, then python, then python, then React (and some other stuff).
- grzm 9y ago
  Can you expand the last one? The first three are server-side, while the last is front end. Are they using node on the back end?
  - Figs 9y ago
    Recent post about it: https://www.reddit.com/r/announcements/comments/8830oa/and_now_a_word_from_reddits_engineers/ https://www.reddit.com/r/announcements/comments/8830oa/and_n...
    - grzm 9y ago
      Cheers. Looks like this most recent work was primarily front-end: > "When we set out to rewrite our code to solve these problems, we wanted to make sure we weren't just fixing small, isolated issues but creating a new, more modern frontend stack..."
- colejohnson66 9y ago
  Why’d you list Python twice?
  - grzm 9y ago
    The second rewrite was also in Python.
- pknopf 9y ago
  There UI doesn't look like it is using React. Maybe they are still working on it?
  - SquareWheel 9y ago
    It's still in development. Available at alpha.reddit.com, but it may not be available to all yet.

notthemessiah 9y ago

It's unfortunate that Reddit silently went closed-source before being forced to admit it. Certainly a betrayal of trust with their long-standing userbase.

Mononokay 9y ago
The majority of their code is still open-source. https://github.com/reddit https://github.com/reddit
- andrewguenther 9y ago
  It is definitely not the majority and doesn't match what they're running on the back-end anymore at all.
- Sylos 9y ago
  That's like someone telling you the majority of ingredients that they put into something you're supposed to eat, but not that one ingredient. It's either their secret super sauce or it's poison. And I can tell you for free that whatever Reddit has built technologically, does not contain any secret super sauce whatsoever.
  - chme 9y ago
    I sort of get reddits reason to hold the post ranking algorithm secret. So that it is hard to game it. Until there is a way to easy 're-key' such an algorithm if someone found a way to abuse it, holding it secret is the best solution for them. I currently can't think of a better solution to avoid gaming of ranking algorithms than holding it secret and changing/adapting it often. Maybe some machine learning algorithm?
confounded 9y ago
Yup. I understand the scale and micro services thing, but they could at least keep the mobile clients open (even if with a lagged release for feature launches, or just periodic tarballs).

CodeArtisan 9y ago

Why Reddit has been rewritten in python: https://redditblog.com/2005/12/05/on-lisp/ https://redditblog.com/2005/12/05/on-lisp/ http://www.aaronsw.com/weblog/rewritingreddit http://www.aaronsw.com/weblog/rewritingreddit

pknopf 9y ago
What is it like maintaining a large project in such a dynamic language? I guess if you have proper unit tests and good coverage, you could quickly find any runtime bugs that you may unintentionally introduce with refactors/changes. When in JavaScript, I'd always lean on TypeScript. Is there something similar for Python?
- hguant 9y ago
  Python 3.6 has typing/type annotations. It's opt in but seems to work fairly sanely. https://medium.com/@ageitgey/learn-how-to-use-static-type-checking-in-python-3-6-in-10-minutes-12c86d72677b https://medium.com/@ageitgey/learn-how-to-use-static-type-ch...

nodesocket 9y ago

Never played around with Lisp, so excuse the ignorance. Is this typical to construct HTML in Lisp? This feels incredibly verbose and error prone, not to mention confusing and hard to grok. Good luck having a designer mockup/write HTML. (defun login-panel () (pbox "login/register" (:form :id "logform" :class "nomargin" (:table :style "border-collapse: collapse" (:tr (:td :colspan "2" "username:")) (:tr (:td :colspan "2" (:input :id "loguser" :class "txt" :name "user" :type "text" :size 15))) (:tr (:td :colspan "2" "password:")) (:tr (:td :colspan "2" (:input :id "logpass" :class "txt" :name "pass" :type "password" :size 15))) (:tr (:td :colspan "2" (:input :id "logmem" :type "checkbox" :name "mem" "remember me"))) (:tr (:td :colspan "2" (:span :id "logerror" :class "error" ""))) (:tr (:td (:input :id "logbtn" :class "btn" :type "submit" :value "Login" :onclick "login(); return false")) (:td (:input :class "btn" :type "submit" :value "Register" :onclick "register(); return false"))) (:tr (:td :nowrap t :colspan "2" :align "center" :class "little" (:a :href "/password" "what's my password?")))))))

grzm 9y ago
It's not uncommon. An example in Racket is html-template.[0] Hiccup is one in Clojure.[1] There's a spectrum of methods for doing things like this: both Clojure and Racket have Mustache[2] implementations. "Confusing and hard to grok"† is in the eye of the beholder. Syntax highlighting and indentation go a long way (as well as exposure), though even in black and white, the parens fade for me much more than angle brackets do. Personally, I'd wrap some of the longer lines, but that's a style thing. [0]: https://docs.racket-lang.org/html-template/index.html https://docs.racket-lang.org/html-template/index.html. [1]: https://github.com/weavejester/hiccup https://github.com/weavejester/hiccup [2]: https://mustache.github.io/mustache.5.html https://mustache.github.io/mustache.5.html † Sorry. Couldn't let the typo stand. Heinlein made me do it.
- spapas82 9y ago
  Clojure also has Selmer (https://github.com/yogthos/Selmer https://github.com/yogthos/Selmer) which is a Django inspired templating language!

1wd 9y ago

What kind of Lisp system was used to run this? Would it work with any Common Lisp? What other things are required, like libraries or a database etc.? I'm guessing reddit.asd lists some: :tbnl (a toolkit for building dynamic websites) :cl-ppcre (a regex library) :trivial-http (a HTTP client) :cl-who (a library for using S-expressions as HTML markup templates) :clsql (a SQL database interface and ORM) :clsql-postgresql (a PostgreSQL connector) :cl-smtp (a SMTP client for sending email) :ironclad (a cryptography library)

chuchana 9y ago
CMUCL on FreeBSD [https://redditblog.com/2005/12/05/on-lisp/ https://redditblog.com/2005/12/05/on-lisp/]
lispm 9y ago
SBCL
adwf 9y ago
The interesting thing is you can still use pretty much that entire stack today. The only one that stands out as outdated is tbnl which changed name to hunchentoot. The stack I used for a website just the other day: hunchentoot cl-who postmodern (prefer it over clsql-postgresql) cl-smtp ironclad drakma (fuller featured than trivial-http)

deleted 9y ago

[deleted]

lifthrasiir 9y ago

I chuckled over this bit of lines [1]. Fortunately (for Reddit) this name is being consistently used throughout the code. Can't guess why. (email :type string :accessor user-emai) [1] https://github.com/reddit-archive/reddit1.0/blob/bb4fbdb5871a32709df30540685cf44c637bf203/view-defs.lisp#L32-L34 https://github.com/reddit-archive/reddit1.0/blob/bb4fbdb5871...

vzcx 9y ago
Heh. If you see [1], they define a function called "user-email" (spelled correctly). The lisp convention here would be to define the accessor as something like "%user-email." While the code isn't too bad, these little things betray a lack of knowledge of lisp lore. Perhaps another reason for the rewrite was that the team actually didn't have much experience writing lisp code. [1] https://github.com/reddit-archive/reddit1.0/blob/bb4fbdb5871a32709df30540685cf44c637bf203/data.lisp#L336 https://github.com/reddit-archive/reddit1.0/blob/bb4fbdb5871...
- kazinator 9y ago
  https://github.com/Clozure/ccl/blob/master/lib/case-error.lisp#L51 https://github.com/Clozure/ccl/blob/master/lib/case-error.li... %assertion-failure doesn't look like an accessor. http://clhs.lisp.se/Body/f_car_c.htm http://clhs.lisp.se/Body/f_car_c.htm car described as "accessor", not named %car. No public function in CL is named this way. In fact, the character % doesn't make an appearance at all: http://www.lispworks.com/documentation/lw50/CLHS/Front/X_Perm_9.htm http://www.lispworks.com/documentation/lw50/CLHS/Front/X_Per... I've never named a function or macro %something in 18 years of Lisp.
  - vzcx 9y ago
    Sure, sure. But while it's definitely not in the standard, I do see it all over, and it's something I picked up from reading a lot of other people's lisp. SBCL internals, for example. You might not like the convention and chose not to follow along with it, but I would be surprised if, after 18 years of lisp, you had never seen it before, and would choose to misspell an accessor to prevent a clash instead of naming it differently. Edit: ah, I think I understand. I don't mean to imply that all accessors should be named in this way. That would be gross.

melling 9y ago

Why Reddit switched to Python: https://news.ycombinator.com/item?id=3815491 https://news.ycombinator.com/item?id=3815491 Of course HN is written in Lisp. pg is a big fan: http://www.paulgraham.com/lisp.html http://www.paulgraham.com/lisp.html

bringtheaction 9y ago
Thanks, was about to ask what made them switch.

nathell 9y ago

It's fun to look at the ASDF system definition: https://github.com/reddit-archive/reddit1.0/blob/master/reddit.asd https://github.com/reddit-archive/reddit1.0/blob/master/redd... Some Ediware in the dependencies (CL-PPCRE, CL-WHO and Hunchentoot from back when it was called TBNL). I guess they used CL-SQL rather than Postmodern because the latter didn't exist back then. Should be reasonably easy to get it running on a modern CL.

rurban 9y ago

With mail username and password

Mister_Snuggles 9y ago

Is this really all the source code? It seems to be missing the database schema, which is pretty critical to using it. Beyond that, this is actually fascinating to look at. It's small enough that you can actually understand it, but it's more complete than a simple 'toy' application. I'm also fascinated to see how SQL constructs are expressed in LISP. I don't know LISP at all, but it's pretty obvious how the queries in the language map directly to SQL.

kjdsiheioahef 9y ago

Unrelated to the original post: Does anyone know if Reddit keeps or shares records of content that ends up on the (unauthenticated, uncustomized) homepage (top 30 results at any given time) for US-based users? If so, how far back does such data go? I don't believe the mainstream archive sites would be a definitive source. Perhaps there is another?