11 ms·

Lisp is amazing. Interesting why Python won the race to become the language of choice for the neural networks wave of AI.

kragen 3y ago
well, if we categorize python and js as non-lisp languages, i'd say it's mostly because python has been the language of choice at google since last millennium, at least for situations where flexibility outweighs performance last millennium when google started up, no lisp was a good option. clojure didn't exist, schemes were mostly for teaching, cmucl was in disrepair, and though there were proprietary common lisps, they were proprietary and wanted per-cpu license fees, which would have meant disclosing how many cpus google had—a closely guarded secret moreover, common lisp's approach of putting a uniform but somewhat repulsive veneer over all operating systems was a pure drawback in google's all-linux environment, and s-expression syntax is harder to read than python's (if easier to edit) also the python community was friendly and welcoming, while the scheme community was tiny and fragmented, and the common lisp community had degenerated into an ego trip for a world-class asshole named erik naggum this is guessing from the outside; i've never worked at google, which is why i'm allowed to write this
- throwaway123422 3y ago
  [flagged]
FullGarden_S 3y ago
Simple. Python had more hype than Lisp did when it mattered (partly because it was modern). So everyone boarded the hype train and here we are.
IKantRead 3y ago
As a huge Lisp/Scheme fan, I don't find it surprising that Python became the language of choice for neural networks over Lisp. NN in general just boil down to doing lots of linear algebra, which is a lot about mutating very large matrices. For this you really just want a wrapper around BLAS/LAPACK, so you can leverage existing optimized libraries. Working with matrices in Lisp always feels a bit clunky, and doubly so when you want to do a lot of stateful operations on them. The real compitetor in this space to Python is something like Matlab, which has probably the best interface for doing linear algebra (but is worse at everything else). The one area where Lisp and Python both shine is the ability to perform automatic differentiation. Lisps are great for this task since it's all symbolic manipulation. However, this is only important once you have a solid interface for working with matrices. If you want to get a feel for the difference I would suggest reading through the surprisingly excellent The Little Learner. I think you'll find that while it really demonstrates the power of Scheme (Racket in this case) in areas where it excels, you wouldn't want to use the framework in that book for anything other than toy examples.
- jimbokun 3y ago
  But is there is something inherent to Lisp that makes it "clunky" to work with matrices? Lisp can be at least as high level and expressive as any other language.
  - chlorion 3y ago
    I don't think so, I think it just happens to be that common lisp is more oriented around lists and not vectors. Since we're in lisp, you can imagine some macros that make working with matrices easier!
  - IKantRead 3y ago
    > But is there is something inherent to Lisp that makes it "clunky" to work with matrices? s-exps are basically coding at the level of the AST, which for many programming tasks is a powerful level of both abstraction and control. But this interface tends to work much better on tree-like structures (the most simple of which is the list, which in it's simplest case is a cons cell, Lisp's most fundamental particle). Most programming tasks can boil down to tree manipulation, but large matrices feel a bit out of place with this interface because you are basically to working with arrays (which in it's simplest cases is a pointer to a memory address, which is C's fundamental particle). C like languages (I know, technical ALGO-like), tend to feel more natural for these tasks. But of course if you can manipulate trees easily, you can manipulate code easily which gives you... > Lisp can be at least as high level and expressive as any other language. In practice, for Lisp this is also a weakness. Again, I love Lisps in general, and of course any Lisp and especially Common Lisp (with it's exceptionally excellent macro system) can do anything you want. However, my (and many others) experience has been that this leads to is it becomes very easy to create code that is amazing for the original developer, but very difficult for new developers to get their head around. Hence the adage "Lisp is optimal for team sizes of one." You could undoubtable build the worlds most elegant interface to working with matrices in Lisp, but now you essentially have a new language.
    - lispm 3y ago
      Lisps have n-dimensional arrays since decades. Often their implementations are a bit less efficient than the ones found written in C and used by Python. > You could undoubtable build the worlds most elegant interface to working with matrices in Lisp, but now you essentially have a new language. What?
pjmlp 3y ago
Not really, Python is where many Lisp folks went when it started to win attention around 20 years ago in research. CERN and Fermilab were adopting Python for Grid Computing (aka Cloud nowadays), already in 2003. MIT changed SICP from Scheme to Python back in 2006, thus news generations started playing with Python. Peter Norvig called Python an acceptable Lisp around 2010, in HN. Having written Python for Lisp programmers in 2000. https://news.ycombinator.com/item?id=1803351 https://news.ycombinator.com/item?id=1803351 https://norvig.com/python-lisp.html https://norvig.com/python-lisp.html What bums me out, is the lack of JIT in the box (yes I know there are some alternatives).
lenkite 3y ago
Maybe when Common LISP gets its next standard and stops being a geezer on the deathbed ? The world has changed since 1994. So has the computing landscape. If a programming language is static for more than a human generation, then it cannot expect any sort of adoption.
- vindarel 3y ago
  Implementations evolve and improve. https://lisp-journey.gitlab.io/blog/these-years-in-common-lisp-2022-in-review/ https://lisp-journey.gitlab.io/blog/these-years-in-common-li... I just learned today that another new GC for SBCL is in the works, with GC pauses under 100 microseconds.
- kagevf 3y ago
  It's unlikely the ANSI standard will ever get updated - it's a very expensive process. Python doesn't have a standard, but it has a canonical implementation, so maybe a "defacto standard". Not having an official standard didn't matter, though. Common Lisp has defacto "standard libraries" for things like threading. Other things, such as async, are less clear (to me), but libraries do exist. Just because something isn't in the standard, though, doesn't mean it can't exist. Also, implementation such as SBCL pick up a lot of the slack, filling in some of the missing gaps.
  - lenkite 3y ago
    Python evolves through PEP's https://peps.python.org https://peps.python.org. Can you kindly correct my deplorable ignorance and point me to the Common LISP enhancement proposals ? Maybe I am missing something elementary.
    - mepian 3y ago
      They are here: https://cdr.common-lisp.dev/ https://cdr.common-lisp.dev/
      - lenkite 3y ago
taeric 3y ago
I'm still 90% of the mind that python got the traction it did because it was installed by default on so many machines.
linguae 3y ago
My understanding is that enough time had passed between the AI winter of the late 1980s and the rise of big data and deep neural networks in the 2010s to where a relatively tiny minority of modern machine learning practitioners came from the old-school AI community, which tended to prefer Lisp or Prolog. When machine learning became hot in the past 15 years or so, many new entrants to ML who had no previous AI knowledge used the tools they’re familiar with, which are generally C, C++, Java, JavaScript, and Python. In addition, many in the natural sciences use Fortran and Matlab. Perhaps had the AI winter of the late 1980s not happened (which took out the Lisp machine market) and had Common Lisp not been relegated to certain niches, perhaps there would’ve been more usage of the language, which may have led to Python being less dominant today. I love Lisp and Smalltalk, but I work as a machine learning researcher as my day job, and I code in Python at work. Admittedly I miss Common Lisp whenever I code in Python, and I think Jupyter notebooks are a poor man’s Smalltalk environment, but when comparing Python to C and C++ (and I used to work primarily in systems programming, so I know my way around C), Python is a major productivity boost.
CyberDildonics 3y ago
I like that you're framing this as if it was between python and lisp and not python being the language of choice by a huge margin followed by a dozen other languages, then lisp.

kragen 3y ago

well, if we categorize python and js as non-lisp languages, i'd say it's mostly because python has been the language of choice at google since last millennium, at least for situations where flexibility outweighs performance last millennium when google started up, no lisp was a good option. clojure didn't exist, schemes were mostly for teaching, cmucl was in disrepair, and though there were proprietary common lisps, they were proprietary and wanted per-cpu license fees, which would have meant disclosing how many cpus google had—a closely guarded secret moreover, common lisp's approach of putting a uniform but somewhat repulsive veneer over all operating systems was a pure drawback in google's all-linux environment, and s-expression syntax is harder to read than python's (if easier to edit) also the python community was friendly and welcoming, while the scheme community was tiny and fragmented, and the common lisp community had degenerated into an ego trip for a world-class asshole named erik naggum this is guessing from the outside; i've never worked at google, which is why i'm allowed to write this

throwaway123422 3y ago
[flagged]

FullGarden_S 3y ago

Simple. Python had more hype than Lisp did when it mattered (partly because it was modern). So everyone boarded the hype train and here we are.

IKantRead 3y ago

As a huge Lisp/Scheme fan, I don't find it surprising that Python became the language of choice for neural networks over Lisp. NN in general just boil down to doing lots of linear algebra, which is a lot about mutating very large matrices. For this you really just want a wrapper around BLAS/LAPACK, so you can leverage existing optimized libraries. Working with matrices in Lisp always feels a bit clunky, and doubly so when you want to do a lot of stateful operations on them. The real compitetor in this space to Python is something like Matlab, which has probably the best interface for doing linear algebra (but is worse at everything else). The one area where Lisp and Python both shine is the ability to perform automatic differentiation. Lisps are great for this task since it's all symbolic manipulation. However, this is only important once you have a solid interface for working with matrices. If you want to get a feel for the difference I would suggest reading through the surprisingly excellent The Little Learner. I think you'll find that while it really demonstrates the power of Scheme (Racket in this case) in areas where it excels, you wouldn't want to use the framework in that book for anything other than toy examples.

jimbokun 3y ago
But is there is something inherent to Lisp that makes it "clunky" to work with matrices? Lisp can be at least as high level and expressive as any other language.
- chlorion 3y ago
  I don't think so, I think it just happens to be that common lisp is more oriented around lists and not vectors. Since we're in lisp, you can imagine some macros that make working with matrices easier!
- IKantRead 3y ago
  > But is there is something inherent to Lisp that makes it "clunky" to work with matrices? s-exps are basically coding at the level of the AST, which for many programming tasks is a powerful level of both abstraction and control. But this interface tends to work much better on tree-like structures (the most simple of which is the list, which in it's simplest case is a cons cell, Lisp's most fundamental particle). Most programming tasks can boil down to tree manipulation, but large matrices feel a bit out of place with this interface because you are basically to working with arrays (which in it's simplest cases is a pointer to a memory address, which is C's fundamental particle). C like languages (I know, technical ALGO-like), tend to feel more natural for these tasks. But of course if you can manipulate trees easily, you can manipulate code easily which gives you... > Lisp can be at least as high level and expressive as any other language. In practice, for Lisp this is also a weakness. Again, I love Lisps in general, and of course any Lisp and especially Common Lisp (with it's exceptionally excellent macro system) can do anything you want. However, my (and many others) experience has been that this leads to is it becomes very easy to create code that is amazing for the original developer, but very difficult for new developers to get their head around. Hence the adage "Lisp is optimal for team sizes of one." You could undoubtable build the worlds most elegant interface to working with matrices in Lisp, but now you essentially have a new language.
  - lispm 3y ago
    Lisps have n-dimensional arrays since decades. Often their implementations are a bit less efficient than the ones found written in C and used by Python. > You could undoubtable build the worlds most elegant interface to working with matrices in Lisp, but now you essentially have a new language. What?

pjmlp 3y ago

Not really, Python is where many Lisp folks went when it started to win attention around 20 years ago in research. CERN and Fermilab were adopting Python for Grid Computing (aka Cloud nowadays), already in 2003. MIT changed SICP from Scheme to Python back in 2006, thus news generations started playing with Python. Peter Norvig called Python an acceptable Lisp around 2010, in HN. Having written Python for Lisp programmers in 2000. https://news.ycombinator.com/item?id=1803351 https://news.ycombinator.com/item?id=1803351 https://norvig.com/python-lisp.html https://norvig.com/python-lisp.html What bums me out, is the lack of JIT in the box (yes I know there are some alternatives).

lenkite 3y ago

Maybe when Common LISP gets its next standard and stops being a geezer on the deathbed ? The world has changed since 1994. So has the computing landscape. If a programming language is static for more than a human generation, then it cannot expect any sort of adoption.

vindarel 3y ago
Implementations evolve and improve. https://lisp-journey.gitlab.io/blog/these-years-in-common-lisp-2022-in-review/ https://lisp-journey.gitlab.io/blog/these-years-in-common-li... I just learned today that another new GC for SBCL is in the works, with GC pauses under 100 microseconds.
kagevf 3y ago
It's unlikely the ANSI standard will ever get updated - it's a very expensive process. Python doesn't have a standard, but it has a canonical implementation, so maybe a "defacto standard". Not having an official standard didn't matter, though. Common Lisp has defacto "standard libraries" for things like threading. Other things, such as async, are less clear (to me), but libraries do exist. Just because something isn't in the standard, though, doesn't mean it can't exist. Also, implementation such as SBCL pick up a lot of the slack, filling in some of the missing gaps.
- lenkite 3y ago
  Python evolves through PEP's https://peps.python.org https://peps.python.org. Can you kindly correct my deplorable ignorance and point me to the Common LISP enhancement proposals ? Maybe I am missing something elementary.
  - mepian 3y ago
    They are here: https://cdr.common-lisp.dev/ https://cdr.common-lisp.dev/
    - lenkite 3y ago

taeric 3y ago

I'm still 90% of the mind that python got the traction it did because it was installed by default on so many machines.

linguae 3y ago

My understanding is that enough time had passed between the AI winter of the late 1980s and the rise of big data and deep neural networks in the 2010s to where a relatively tiny minority of modern machine learning practitioners came from the old-school AI community, which tended to prefer Lisp or Prolog. When machine learning became hot in the past 15 years or so, many new entrants to ML who had no previous AI knowledge used the tools they’re familiar with, which are generally C, C++, Java, JavaScript, and Python. In addition, many in the natural sciences use Fortran and Matlab. Perhaps had the AI winter of the late 1980s not happened (which took out the Lisp machine market) and had Common Lisp not been relegated to certain niches, perhaps there would’ve been more usage of the language, which may have led to Python being less dominant today. I love Lisp and Smalltalk, but I work as a machine learning researcher as my day job, and I code in Python at work. Admittedly I miss Common Lisp whenever I code in Python, and I think Jupyter notebooks are a poor man’s Smalltalk environment, but when comparing Python to C and C++ (and I used to work primarily in systems programming, so I know my way around C), Python is a major productivity boost.

CyberDildonics 3y ago

I like that you're framing this as if it was between python and lisp and not python being the language of choice by a huge margin followed by a dozen other languages, then lisp.