9 ms·

I think Reddit's decision to move from Common Lisp to Python is interesting and their reasoning (ecosystem) is still valid 15 years later. The historical timel

by jasode 6y ago

I think Reddit's decision to move from Common Lisp to Python is interesting and their reasoning (ecosystem) is still valid 15 years later. The historical timeline is especially interesting because Reddit's cofounders Steve Huffman & Aaron Swartz were alumni of Paul Graham's first YC batch and PG is the author of the well-known Lisp essay "Beating the Averages": - 2001-04 Paul Graham : Lisp essay "Beating the Averages" [1] - 2005-07-26 Paul Graham : https://groups.google.com/forum/#!topic/comp.lang.lisp/vJmLVd9lJyk https://groups.google.com/forum/#!topic/comp.lang.lisp/vJmLV... - 2005-12-05 Steve Huffman : https://redditblog.com/2005/12/05/on-lisp/ https://redditblog.com/2005/12/05/on-lisp/ - 2005-12-06 Aaron Schwartz : http://www.aaronsw.com/weblog/rewritingreddit http://www.aaronsw.com/weblog/rewritingreddit The takeaway from the Reddit case study is this: Yes Lisp macros and language malleability gives it superpowers that other languages don't have (the "Blub Paradox") -- but other languages also have other superpowers (ecosystem/libraries) that can cancel out the power of Lisp macros. You have to analyze your personal project to predict if the ecosystem matters more than Lisp's elegant syntax powers. [1] http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html

nojito 6y ago
Or the simpler reason is that they just knew LISP better than other languages and chose to use it. Once they needed to grow by adding developers, they decided to rewrite into a language that other people already knew. The only "Super Power" is the complete understanding of what your endusers want so that you can grow.
- cornishpixels 6y ago
  Knowledgeable developers are part of every language's ecosystem.
  - nojito 6y ago
    Yes but ease of attracting developers is harder in some languages versus others. No need to include the "knowledgeable" modifier.
    - p_l 6y ago
      Interestingly I have encountered voices that using "elitist" language like Common Lisp, OCaml or Haskell combined with widening recruitment pool (by going globally distributed) actually increased some companies abilities to hire experienced developers - because it filtered popular languages that suffered from higher number of people who had higher opinion of their skills than warranted - or who just wanted to "fake it till you make it", especially after boot camps became popular.
- gowld 6y ago
  When you are writing a tiny protoype 15 years ago, any language you know is fine. But over time -- both the life of the product, and the world overall, software grows far more complex and the ecosystem matters much more.
andi999 6y ago
Do you know of any of this 'superpower'code released somewhere? Like is it possible to study the original viaweb code?
- Miraries 6y ago
  https://github.com/reddit-archive/reddit1.0 https://github.com/reddit-archive/reddit1.0
  - andi999 6y ago
    Thanks. Looks neat. Will see a bit more of it. Of course the next question is, why they abandoned it, and reading this https://news.ycombinator.com/item?id=3815491 https://news.ycombinator.com/item?id=3815491 My translation of this is they had bug which came from the libs which they couldnt fix.
  - andi999 6y ago
    Maybe thid is not a showcase of 'superpower'. From what I read it took a whole summer to write in lisp and the rewrite in python took a weekend. (But will try to double check this)
    - flavio81 6y ago
      >From what I read it took a whole summer to write in lisp and the rewrite in python took a weekend. And one guy in comp.lang.lisp wrote a working Reddit 1.0 clone in Lisp in 4 hours: https://groups.google.com/d/msg/comp.lang.lisp/9WD9-yEaqMs/HJzbZtoESl0J https://groups.google.com/d/msg/comp.lang.lisp/9WD9-yEaqMs/H...
      - andi999 6y ago
        It didnt seem to be online for a significant amount of time? How would one know it worked (and was mantainable)?
alehander42 6y ago
these days many newer languages have macros: lisp is not alone
- dreamcompiler 6y ago
  C has macros. They are nothing like lisp macros. Please cite a non-lisp language with macros a powerful as lisp's.
  - krebs_liebhaber 6y ago
    Elixir. https://elixir-lang.org/getting-started/meta/macros.html https://elixir-lang.org/getting-started/meta/macros.html
  - plinkplonk 6y ago
    Julia
    - dreamcompiler 6y ago
      Julia is lisp in drag so it doesn't really count, but I'll give you the point.
      - dTal 6y ago
        In what sense is Julia a lisp in drag? I'm just starting in on it it doesn't seem any more lispy than any other dynamic language. In particular I've yet to see a car/cdr.
  - JeremyBanks 6y ago
    Rust?
  - deleted 6y ago
    [deleted]
  - mumblemumble 6y ago
    C macros and lisp macros (In particular, I'm thinking of hygienic macros à la Scheme) are two wildly different extremes. In between them lies a huge continuum of interesting things. There are Scala's macros. There are Haskell's language extensions. There are code generators like the ones used to implement much of gRPC, and there are compiler plugins like the ones used to implement aspect-oriented programming in C# and Java. Most of them arguably qualify as being somewhere between an 80% solution and a 90% solution for the practical problems that lisp might solve with macros. They're frequently not anywhere near as convenient to use or understand. But that's arguably a good thing in the long run, because it encourages communities to pool resources and work on a coordinated solution that can be shared by the community. My take on the curse of lisp, for what it's worth, is economies of scale. Lisp makes it too easy (and fun) to just keep your head down and work on your own thing, which hinders pooling of resources.
mark_l_watson 6y ago
When Alexis Ohanian gave a talk at Google in 2013 I got to talk with him about he and Steve dropping Lisp for Python. He said the Lisp version just kept falling down. In production using SBCL on a customer project we had good results hiring a SBCL maintainer/developer to work past a few issues. From personal experience, I can assure you that LispWorks and Franz both provide incredibly good support for their customers. That said, if I had to develop Reddit right now, I would not use Common Lisp either.
- uxcolumbo 6y ago
  What would you use instead if you had to choose a Lisp based language today? Clojure? Or a non-lisp language instead?
- fiddlerwoaroof 6y ago
  I’ve looked a bit at the original reddit code (it is/was published on GitHub) it does a lot of things in unusual ways for a lisp project and I’m not too surprised that they abandoned it: I’m pretty sure that if they had rewritten it in lisp, taking advantage of what they learned, the second version would have worked as well as the Python rewrite. Also, quicklisp didn’t exist/wasn’t as widely used then: I’ve done all sorts of Lisp projects now and never had serious issues finding good libraries for standard web development tasks.
  - p_l 6y ago
    Didn't they also had to deal with wanting to use threaded architecture while simultaneously deciding to run on less supported platform (FreeBSD)?
justinmeiners 6y ago
I would like to consider Python a common lisp alternative. But it's one or two orders of magnitude slower and is not really standardized. I think there is a lot more that's attractive than just macros.
- elcomet 6y ago
  > and is not really standardized What do you mean by this ?
  - justinmeiners 6y ago
    My understanding is the Python standard is essentially whatever C Python chooses to do. They do write the changes to a specification separately, but essentially its one moving implementation, with alternatives that have compatibility limitations.
    - schmudde 6y ago
      Indeed. I'm curious what Python 4 will do to Python 3.
  - p_l 6y ago
    "standard" is "whatever CPython managed to miss a segfault for today". And PyEvalFrameEx is the stuff of nightmares that can't be really fixed because it hardcoded spaghetti that, if rewritten, could subtly break python in myriad ways that would make 2->3 migration a cakewalk. Oh, and there's no real test suite to verify if you didn't break something subtle in implementation. While parsing Ruby code is well known to be problematic, there is a very good reason why Ruby can have interchangeable implementation while Python pretty much can't.
- deleted 6y ago
  [deleted]
- oalae5niMiel7qu 6y ago
  ...and Python doesn't have it. Image-based programming is the other thing besides macros that gives Lisp an edge. Python might as well be batch-compiled.
snazz 6y ago
My understanding was that Steve mostly wrote it in Lisp to impress PG, and then Aaron convinced Steve to rewrite it (although Aaron was kind of a Python fanboy and it's telling that the rewrite only took a weekend).
pjc50 6y ago
> Blub Paradox") -- but other languages also have other superpowers (ecosystem/libraries) Indeed. The "blub paradox" is profoundly arrogant - it presumes not only that people are too stupid to see the advantages of Lisp, but that Lisp advocates are smart enough to know that other languages have nothing unique to offer that Lisp doesn't.
- snazz 6y ago
  I think that Beating the Averages demonstrated that Lisp was PG's Blub, and maybe he wasn't using a language with strong static typing or logic programming or something like that for the same reasons that Java programmers wouldn't choose Lisp. I don't think that the fundamental flaw with that essay is the arrogance so much as the assumption that "language power" lies on a continuum in the first place.
  - nojito 6y ago
    There was nothing special about the language chosen for viaweb. It was more the ability to understand what their endusers wanted better than their competitors.
    - snazz 6y ago
      If their competitors were writing their applications in C++ or Perl, it probably would have made a noticeable difference to their iteration speed. Writing a web application in C++ (especially in 1995) would not have been too efficient. I'm sure that they had a good understanding of their customers, but the language had to have made a difference as well. These days, we're writing web apps in languages like JavaScript and Ruby, not C++, so the difference would be a lot less significant.
      - nojito 6y ago
        Knowledge of the customer and programming ability is the most important driver of success. The idea that LISP was a secret sauce for viaweb does not hold up under scrutiny. If you have access to lexisnexis to lookup news articles from the era, it’s easy to see that.
flavio81 6y ago
>I think Reddit's decision to move from Common Lisp That original Reddit system was very simple, yet the Lisp source code was of poor quality (it is currently available on the web). When it was released, the lispers at comp.lang.lisp said they could write a clone in a few days. One of them did it in about 2 or 3 hours (true story). So it wasn't good code. Moreover they, for some strange reason, chose to develop using a different implementation than the one in the server, and things like that. Then Reddit got a famous name recommended by PG, who was a Python expert. So he rewrote the thing in Python and obviously, being a Python expert, did a good job.

by jasode 6y ago

nojito 6y ago

Or the simpler reason is that they just knew LISP better than other languages and chose to use it. Once they needed to grow by adding developers, they decided to rewrite into a language that other people already knew. The only "Super Power" is the complete understanding of what your endusers want so that you can grow.

cornishpixels 6y ago
Knowledgeable developers are part of every language's ecosystem.
- nojito 6y ago
  Yes but ease of attracting developers is harder in some languages versus others. No need to include the "knowledgeable" modifier.
  - p_l 6y ago
    Interestingly I have encountered voices that using "elitist" language like Common Lisp, OCaml or Haskell combined with widening recruitment pool (by going globally distributed) actually increased some companies abilities to hire experienced developers - because it filtered popular languages that suffered from higher number of people who had higher opinion of their skills than warranted - or who just wanted to "fake it till you make it", especially after boot camps became popular.
gowld 6y ago
When you are writing a tiny protoype 15 years ago, any language you know is fine. But over time -- both the life of the product, and the world overall, software grows far more complex and the ecosystem matters much more.

andi999 6y ago

Do you know of any of this 'superpower'code released somewhere? Like is it possible to study the original viaweb code?

Miraries 6y ago
https://github.com/reddit-archive/reddit1.0 https://github.com/reddit-archive/reddit1.0
- andi999 6y ago
  Thanks. Looks neat. Will see a bit more of it. Of course the next question is, why they abandoned it, and reading this https://news.ycombinator.com/item?id=3815491 https://news.ycombinator.com/item?id=3815491 My translation of this is they had bug which came from the libs which they couldnt fix.
- andi999 6y ago
  Maybe thid is not a showcase of 'superpower'. From what I read it took a whole summer to write in lisp and the rewrite in python took a weekend. (But will try to double check this)
  - flavio81 6y ago
    >From what I read it took a whole summer to write in lisp and the rewrite in python took a weekend. And one guy in comp.lang.lisp wrote a working Reddit 1.0 clone in Lisp in 4 hours: https://groups.google.com/d/msg/comp.lang.lisp/9WD9-yEaqMs/HJzbZtoESl0J https://groups.google.com/d/msg/comp.lang.lisp/9WD9-yEaqMs/H...
    - andi999 6y ago
      It didnt seem to be online for a significant amount of time? How would one know it worked (and was mantainable)?

alehander42 6y ago

these days many newer languages have macros: lisp is not alone

dreamcompiler 6y ago
C has macros. They are nothing like lisp macros. Please cite a non-lisp language with macros a powerful as lisp's.
- krebs_liebhaber 6y ago
  Elixir. https://elixir-lang.org/getting-started/meta/macros.html https://elixir-lang.org/getting-started/meta/macros.html
- plinkplonk 6y ago
  Julia
  - dreamcompiler 6y ago
    Julia is lisp in drag so it doesn't really count, but I'll give you the point.
    - dTal 6y ago
      In what sense is Julia a lisp in drag? I'm just starting in on it it doesn't seem any more lispy than any other dynamic language. In particular I've yet to see a car/cdr.
- JeremyBanks 6y ago
  Rust?
- deleted 6y ago
  [deleted]
- mumblemumble 6y ago
  C macros and lisp macros (In particular, I'm thinking of hygienic macros à la Scheme) are two wildly different extremes. In between them lies a huge continuum of interesting things. There are Scala's macros. There are Haskell's language extensions. There are code generators like the ones used to implement much of gRPC, and there are compiler plugins like the ones used to implement aspect-oriented programming in C# and Java. Most of them arguably qualify as being somewhere between an 80% solution and a 90% solution for the practical problems that lisp might solve with macros. They're frequently not anywhere near as convenient to use or understand. But that's arguably a good thing in the long run, because it encourages communities to pool resources and work on a coordinated solution that can be shared by the community. My take on the curse of lisp, for what it's worth, is economies of scale. Lisp makes it too easy (and fun) to just keep your head down and work on your own thing, which hinders pooling of resources.

mark_l_watson 6y ago

When Alexis Ohanian gave a talk at Google in 2013 I got to talk with him about he and Steve dropping Lisp for Python. He said the Lisp version just kept falling down. In production using SBCL on a customer project we had good results hiring a SBCL maintainer/developer to work past a few issues. From personal experience, I can assure you that LispWorks and Franz both provide incredibly good support for their customers. That said, if I had to develop Reddit right now, I would not use Common Lisp either.

uxcolumbo 6y ago
What would you use instead if you had to choose a Lisp based language today? Clojure? Or a non-lisp language instead?
fiddlerwoaroof 6y ago
I’ve looked a bit at the original reddit code (it is/was published on GitHub) it does a lot of things in unusual ways for a lisp project and I’m not too surprised that they abandoned it: I’m pretty sure that if they had rewritten it in lisp, taking advantage of what they learned, the second version would have worked as well as the Python rewrite. Also, quicklisp didn’t exist/wasn’t as widely used then: I’ve done all sorts of Lisp projects now and never had serious issues finding good libraries for standard web development tasks.
- p_l 6y ago
  Didn't they also had to deal with wanting to use threaded architecture while simultaneously deciding to run on less supported platform (FreeBSD)?

justinmeiners 6y ago

I would like to consider Python a common lisp alternative. But it's one or two orders of magnitude slower and is not really standardized. I think there is a lot more that's attractive than just macros.

elcomet 6y ago
> and is not really standardized What do you mean by this ?
- justinmeiners 6y ago
  My understanding is the Python standard is essentially whatever C Python chooses to do. They do write the changes to a specification separately, but essentially its one moving implementation, with alternatives that have compatibility limitations.
  - schmudde 6y ago
    Indeed. I'm curious what Python 4 will do to Python 3.
- p_l 6y ago
  "standard" is "whatever CPython managed to miss a segfault for today". And PyEvalFrameEx is the stuff of nightmares that can't be really fixed because it hardcoded spaghetti that, if rewritten, could subtly break python in myriad ways that would make 2->3 migration a cakewalk. Oh, and there's no real test suite to verify if you didn't break something subtle in implementation. While parsing Ruby code is well known to be problematic, there is a very good reason why Ruby can have interchangeable implementation while Python pretty much can't.
deleted 6y ago
[deleted]
oalae5niMiel7qu 6y ago
...and Python doesn't have it. Image-based programming is the other thing besides macros that gives Lisp an edge. Python might as well be batch-compiled.

snazz 6y ago

My understanding was that Steve mostly wrote it in Lisp to impress PG, and then Aaron convinced Steve to rewrite it (although Aaron was kind of a Python fanboy and it's telling that the rewrite only took a weekend).

pjc50 6y ago

> Blub Paradox") -- but other languages also have other superpowers (ecosystem/libraries) Indeed. The "blub paradox" is profoundly arrogant - it presumes not only that people are too stupid to see the advantages of Lisp, but that Lisp advocates are smart enough to know that other languages have nothing unique to offer that Lisp doesn't.

snazz 6y ago
I think that Beating the Averages demonstrated that Lisp was PG's Blub, and maybe he wasn't using a language with strong static typing or logic programming or something like that for the same reasons that Java programmers wouldn't choose Lisp. I don't think that the fundamental flaw with that essay is the arrogance so much as the assumption that "language power" lies on a continuum in the first place.
- nojito 6y ago
  There was nothing special about the language chosen for viaweb. It was more the ability to understand what their endusers wanted better than their competitors.
  - snazz 6y ago
    If their competitors were writing their applications in C++ or Perl, it probably would have made a noticeable difference to their iteration speed. Writing a web application in C++ (especially in 1995) would not have been too efficient. I'm sure that they had a good understanding of their customers, but the language had to have made a difference as well. These days, we're writing web apps in languages like JavaScript and Ruby, not C++, so the difference would be a lot less significant.
    - nojito 6y ago
      Knowledge of the customer and programming ability is the most important driver of success. The idea that LISP was a secret sauce for viaweb does not hold up under scrutiny. If you have access to lexisnexis to lookup news articles from the era, it’s easy to see that.

flavio81 6y ago

>I think Reddit's decision to move from Common Lisp That original Reddit system was very simple, yet the Lisp source code was of poor quality (it is currently available on the web). When it was released, the lispers at comp.lang.lisp said they could write a clone in a few days. One of them did it in about 2 or 3 hours (true story). So it wasn't good code. Moreover they, for some strange reason, chose to develop using a different implementation than the one in the server, and things like that. Then Reddit got a famous name recommended by PG, who was a Python expert. So he rewrote the thing in Python and obviously, being a Python expert, did a good job.