8 ms·

I encourage people to try out Common Lisp because, unlike with Hy, you will get: speed, ability to build binaries, truly interactive image-based development (ye

I encourage people to try out Common Lisp because, unlike with Hy, you will get: speed, ability to build binaries, truly interactive image-based development (yes, more interactive than ipython), more static type checks, more language features (no closures in Hy last time I checked), language stability… To reach to Python libs, you have https://github.com/bendudson/py4cl https://github.com/bendudson/py4cl My comparison of Python and CL: https://lisp-journey.gitlab.io/pythonvslisp/ https://lisp-journey.gitlab.io/pythonvslisp/

peatmoss 4y ago
Hy is cool because it allows people to explore some lisp concepts in context of libraries they may already be familiar with. And Python has an enviable set of batteries. But compared to CL or Racket, Python’s performance is a bitter pill to swallow.
- Kodiologist 4y ago
  Even with PyPy, or Python libraries that are themselves written in C?
  - mrtranscendence 4y ago
    PyPy won't be a candidate in some scenarios (nor is it always that much faster), and even libraries written in C leave some performance on the table when Python is the glue language. Don't get me wrong, I'm hardly ever in a position where Python limits me from a performance perspective ... there's always Cython. But it could be better.
    - masklinn 4y ago
      There's also a new kind on the block with mypyc, though it's pretty much pre-alpha and I assume it comes with a very long list of limitations.
  - peatmoss 4y ago
    I ended up replying in a different thread on Python Performance: https://news.ycombinator.com/item?id=31272681#31275210 https://news.ycombinator.com/item?id=31272681#31275210
ducktective 4y ago
What about other LISPs? Racket? Clojure? What is your view on Red language?
- vindarel 4y ago
  They feel all limited to me :D I only did serious CL development but neither of those two especially attract me: they are less interactive (and that's my ideal and a daily joy), I don't need the Java ecosystem, I want to be lighter on resources… I take news about Red from time to time and it's still in its early phases it seems.
  - didibus 4y ago
    I watched the video from your article: https://youtu.be/CNFr7zIfyeM https://youtu.be/CNFr7zIfyeM And as someone who programs in Clojure, I feel the interactivity is the same. Maybe I'm missing a detail of what he's doing though? The part you don't have in Clojure (without using a library), is the restart options, but he didn't seem to ever use them, seems he just looked at the error message, and went back to fix the code and re-evaluate his test again, and that would be exactly the interactive flow I would follow in Clojure. Am I missing something?
    - vindarel 4y ago
      ah, I see. There's another shorter one below: https://www.youtube.com/watch?v=KsHxgP3SRTs https://www.youtube.com/watch?v=KsHxgP3SRTs In that one I get the debugger, I go to the erroneous function, I re-compile it, come back to the debugger and resume execution (seeing everything pass). Can you do that in Clojure? There's also the typical class change: in CL you can alter a class definition and the existing objects are (lazily) updated. Also (and it's like the only time I tried Clojure) you can't install a Clojure library from the Clojure REPL, right? And then, what's explained here: https://mikelevins.github.io/posts/2020-12-18-repl-driven/ https://mikelevins.github.io/posts/2020-12-18-repl-driven/
      - didibus 4y ago
Kodiologist 4y ago
Hy has closures. They work just like Python's. (Common Lisp is super cool, though. Perhaps one day Hy will be as cool.)
- vindarel 4y ago
  oopsy, thanks, I was thinking about let (discussed below). (yep, I tried Hy but not for long, since I needed something more than Python)
mrtranscendence 4y ago
Common Lisp is great, I'm sure, but it's a bit of a nonstarter for me because most of my work is Spark ETL and machine learning. I can theoretically sneak some hy in for low-visibility code but I can't do the same for Common Lisp. > no closures in Hy last time I checked I don't know when the last time you checked was, but as far as I can tell hy definitely has closures. It largely follows Python semantics, after all.
- vindarel 4y ago
  If you're (very) motivated you could try MGL or CLML! https://github.com/melisgl/mgl https://github.com/melisgl/mgl - https://github.com/mmaul/clml https://github.com/mmaul/clml - https://github.com/CodyReichert/awesome-cl#machine-learning https://github.com/CodyReichert/awesome-cl#machine-learning MGL's auhor [won](https://github.com/melisgl/higgsml https://github.com/melisgl/higgsml) the Higgs Boson ML Challenge.
amelius 4y ago
What I look for in a language first is how good the support is for concurrency, and especially if it has a good, modern concurrent garbage collector. A quick search brought up this: https://lisp-journey.gitlab.io/blog/pro-mailing-list-on-common-lisp-and-parallel-gc/ https://lisp-journey.gitlab.io/blog/pro-mailing-list-on-comm... > This was primarily for the lack of good parallel, concurrent garbage collectors in Common Lisp implementations. The CL version of elPrep was actually still a tad faster than any of the C++, Go, or Java versions, but we had to work hard to avoid long GC pauses. elPrep allocates a lot of memory, and the pause time hurts a lot. We solved this by, basically, disabling the garbage collector, and reusing memory manually as much as possible, which turned the program into almost a manually memory-managed affair.
- vindarel 4y ago
  They didn't try Allegro CL's new parallel GC (see comments). Otherwise, yeah. It depends on one's use case, I guess.
- throwaway894345 4y ago
  With respect to concurrency, after a lot of experience operating async Python services, the thing that I look for is a single, standard, ubiquitous concurrency model. One of the biggest pain points for operating Python systems is some transitive dependency making a synchronous I/O call and blocking the whole event loop, causing timeouts in unrelated endpoints and often bringing the whole system down (healthchecks start to fail because the endpoint is blocked and then the supervisor kills the whole process/pod/etc). This doesn't happen in Go or (presumably) languages that run on Erlang's BEAM VM because there is no sync I/O (from the event loop's perspective, anyway).
  - gilch 4y ago
    Python has pre-emptive threading too! The GIL makes it all but useless for CPU-bound tasks, but it works fine for IO. Keep your health-check tasks in a non-IO thread and communicate with queues.
- throw10920 4y ago
catchclose8919 4y ago
Which CL "interactive development" feature can't you get with either a jupyter notebook or ipython + autoreload enables? With a smart editor like VSCode or Spyder you also get visual variable inspections, history of executed code in one way or another etc. Only that can't easily save an "image" auto-magically, but data science and ML people wouldn't want to do this other than explicitly (eg. using fast loading formats like feather to reload semi-processed vars etc.). I don't think you are aware how much the Python workflows of AI / ML / datasci people resemble you "common lisp ideal" ...but with a different language and better libraries :) And if Julia succeeded, it will also bring decent performance to such workflows. Sure, the way backend devs or devops people use Python is very static and different, but any experienced ML practitioners are at home with "interactive development".
- reikonomusha 4y ago
  Re-define a class and have all objects automatically get updated to reflect the changes. Evaluate arbitrary expressions mid-stack trace. Run into a bug, get caught in an error, re-define the offensive function interactively, and re-run the computation before that point, all without stopping and restarting. (Example real-world use-case: You re-define one of your REST API endpoints without having to restart your server or interrupt the request-flow, be it in test or—if you so desire—in prod.) Change the class of an object at run-time. Re-compile and re-load individual functions from your editor with your production, git-committed code. This one is huge. You use interactive and incremental development to, well, develop! Not side-by-side with your code, but in your code as well, at a fine-enough grained level. Common Lisp was designed to be both interactive and incremental. It's not a bolt-on feature. Jupyter notebooks and auto-reloading have a very different feel, and (dis)incentivize different qualities and processes of development.
  - catchclose8919 4y ago
    Basically all you mentioned above can be done in modern Python. Unfortunately I've seen it done... and God, the results are horrible, in Python. "Wait did calling this method just change the class of the object the method was on??!!!! Fuuuu..." But simple stuff like incrementally building classes method by method in eg. jupyter notebooks can be done and it's even quite clean, it's just that nobody is teaching people the techniques for doing it... (And the jupyter autoreload magic doesn't seem to like it so you'd have to "do it by hand") And sure, since Python got scoping and concurrency quite wrong, if you go full-interactive on things you'll shoot yourself in the foot faaast!
emptysongglass 4y ago
This reads out of touch with why curious people use Hy, like me who wanted to get going with an easily accessible language and ecosystem (Python) while rewriting bits in Hy to grok some lisp magic. No, of course I wouldn't have reached for CL first and it's disappointing to read the top-comment is this.
- IshKebab 4y ago
  But... by using Hy you're already learning a new super niche language. If you're going to learn a new language why not ditch Python and use something good? Ecosystem size is really the only thing I can think of.
  - emptysongglass 4y ago
    Because I write Python to plug into a bigger ecosystem and I can mix and match Python and Hy freely. I can also use Poetry to package and manage dependencies and Poetry2nix to output a Nix derivation that is guaranteed to run on someone else's machine. I can also freely tap Python packages like Beautiful Soup or use frameworks like Django (which I am). The entire Python universe is available to me. The number of accessible tutorials for Django, videos for Pythonistas and community support channels is monumentally large. I even get to do cool things like use Django's ORM for learning how to interact with and manipulate data in Postgres, all right there available to me. I can use Discord (despite my distaste for it as a platform it has a huge Django and Python community), Telegram, IRC, and Matrix to ask for help and get metric tonnes of friendly advice, suggestions and criticisms from Pythonistas. And I can take all of this at my own pace, writing only bits in Hy and branching as I go. Every step is as gradual as I want it to be. Common Lisp is not the answer to my needs. Presenting it as so is tone deaf at the least.
    - vindarel 4y ago
      but who's presenting CL as the answer to your needs? (me?) If you're happy with your mix of Python and Hy, well, great. But you know what, you can do what you described with CL too! Need to do some webscraping? Beautiful Soup -> lquery. Django -> well… no CL framework gives you all these batteries included, but what if you want to extract data from a Postgres DB and display it in a web app? that's very easy [1] in CL too. Django ORM -> postmodern or Mito (or more). IDK about Nix but several people are working with CL and Guix. https://github.com/CodyReichert/awesome-cl https://github.com/CodyReichert/awesome-cl What worked out for me was to write companion services in CL, instead of extending my big Django app. This new service that takes data from a FTP server, transforms it, stores it, displays it in a web app? CL. That web app that serves another kind of customers than the Django one? A CL app. It isn't for everybody and sometimes you wish for a bigger ecosystem, but anyways, many things can be done in CL. [1]: ok honestly the learning process was not that easy. It's a bit easier with my contributions now! (Cookbook etc)
lf-non 4y ago
Can you elaborate on how CL provides more static type checks ? I am not much familiar with CL, but does it provide something similar to typed-racket ? Do major libraries actually use it ?
- vindarel 4y ago
  Not so easy question, especially because it depends of the implementation, but SBCL gives pretty good static type errors and warnings. Mostly, you get warnings at the function boundaries. It isn't complete like type checking in the modern languages, but it helps catch many errors. And quickly & interactively, because we compile our functions as we write them, with a keystroke (or the whole file, and from time to time, we build the whole project from scratch). So, coming from Python, that was superbly useful to me. This, everybody uses it. We can also add gradual typing to our variables and functions. It might help for the static checks, and it also gives hints for the compiler to speed things up. We can also declare our own types, but these won't be used for compile-time inference. For a type-racket equivalent, we now have Coalton, it's like a Haskell on top of CL: https://github.com/coalton-lang/coalton/ https://github.com/coalton-lang/coalton/ Its author says: -- Take this with a grain of salt, because I’m neither a user nor expert of Typed Racket, but: Typed Racket focuses more on the gradual typing of a given program. It has lots of features to make that easier, such as occurrence typing. Coalton is a separate, embedded language. Typed Racket achieves polymorphism through subtyping and and first-order type variables. Coalton achieves polymorphism through type variables, higher-kinded types, and type classes. Coalton, like ML and Haskell, focuses on defining objects by their properties and supported functions. This is a proposed way of having modular, reusable code. Typed Racket, as far as I can tell, has no such features. Coalton code can be fully inferred, so type annotations are not necessary. Typed Racket cannot. All in all, I think the biggest and most important take-away is that Typed Racket goes through great effort to seamlessly blend with ordinary Racket. But that means Typed Racket has to compromise on type system features that can only be supported if you’re willing to change the language itself. Coalton puts the type system first, opting for something close to Haskell, at the expense of not being a system for gradually typing Common Lisp, and instead being a separate language altogether. --- They use it for their open-source quantum compiler and other things.
pmarreck 4y ago
possibly because i've been playing with Guix lately, and also because all the smartest people I know seem to use lisp-likes, I'm on team "just use a Lisp" (possibly with some sort of API interface to Python)
gilch 4y ago
No closures? Hasn't it always had them?

peatmoss 4y ago

Hy is cool because it allows people to explore some lisp concepts in context of libraries they may already be familiar with. And Python has an enviable set of batteries. But compared to CL or Racket, Python’s performance is a bitter pill to swallow.

Kodiologist 4y ago
Even with PyPy, or Python libraries that are themselves written in C?
- mrtranscendence 4y ago
  PyPy won't be a candidate in some scenarios (nor is it always that much faster), and even libraries written in C leave some performance on the table when Python is the glue language. Don't get me wrong, I'm hardly ever in a position where Python limits me from a performance perspective ... there's always Cython. But it could be better.
  - masklinn 4y ago
    There's also a new kind on the block with mypyc, though it's pretty much pre-alpha and I assume it comes with a very long list of limitations.
- peatmoss 4y ago
  I ended up replying in a different thread on Python Performance: https://news.ycombinator.com/item?id=31272681#31275210 https://news.ycombinator.com/item?id=31272681#31275210

ducktective 4y ago

What about other LISPs? Racket? Clojure? What is your view on Red language?

vindarel 4y ago
They feel all limited to me :D I only did serious CL development but neither of those two especially attract me: they are less interactive (and that's my ideal and a daily joy), I don't need the Java ecosystem, I want to be lighter on resources… I take news about Red from time to time and it's still in its early phases it seems.
- didibus 4y ago
  I watched the video from your article: https://youtu.be/CNFr7zIfyeM https://youtu.be/CNFr7zIfyeM And as someone who programs in Clojure, I feel the interactivity is the same. Maybe I'm missing a detail of what he's doing though? The part you don't have in Clojure (without using a library), is the restart options, but he didn't seem to ever use them, seems he just looked at the error message, and went back to fix the code and re-evaluate his test again, and that would be exactly the interactive flow I would follow in Clojure. Am I missing something?
  - vindarel 4y ago
    ah, I see. There's another shorter one below: https://www.youtube.com/watch?v=KsHxgP3SRTs https://www.youtube.com/watch?v=KsHxgP3SRTs In that one I get the debugger, I go to the erroneous function, I re-compile it, come back to the debugger and resume execution (seeing everything pass). Can you do that in Clojure? There's also the typical class change: in CL you can alter a class definition and the existing objects are (lazily) updated. Also (and it's like the only time I tried Clojure) you can't install a Clojure library from the Clojure REPL, right? And then, what's explained here: https://mikelevins.github.io/posts/2020-12-18-repl-driven/ https://mikelevins.github.io/posts/2020-12-18-repl-driven/
    - didibus 4y ago

Kodiologist 4y ago

Hy has closures. They work just like Python's. (Common Lisp is super cool, though. Perhaps one day Hy will be as cool.)

vindarel 4y ago
oopsy, thanks, I was thinking about let (discussed below). (yep, I tried Hy but not for long, since I needed something more than Python)

mrtranscendence 4y ago

Common Lisp is great, I'm sure, but it's a bit of a nonstarter for me because most of my work is Spark ETL and machine learning. I can theoretically sneak some hy in for low-visibility code but I can't do the same for Common Lisp. > no closures in Hy last time I checked I don't know when the last time you checked was, but as far as I can tell hy definitely has closures. It largely follows Python semantics, after all.

vindarel 4y ago
If you're (very) motivated you could try MGL or CLML! https://github.com/melisgl/mgl https://github.com/melisgl/mgl - https://github.com/mmaul/clml https://github.com/mmaul/clml - https://github.com/CodyReichert/awesome-cl#machine-learning https://github.com/CodyReichert/awesome-cl#machine-learning MGL's auhor [won](https://github.com/melisgl/higgsml https://github.com/melisgl/higgsml) the Higgs Boson ML Challenge.

amelius 4y ago

What I look for in a language first is how good the support is for concurrency, and especially if it has a good, modern concurrent garbage collector. A quick search brought up this: https://lisp-journey.gitlab.io/blog/pro-mailing-list-on-common-lisp-and-parallel-gc/ https://lisp-journey.gitlab.io/blog/pro-mailing-list-on-comm... > This was primarily for the lack of good parallel, concurrent garbage collectors in Common Lisp implementations. The CL version of elPrep was actually still a tad faster than any of the C++, Go, or Java versions, but we had to work hard to avoid long GC pauses. elPrep allocates a lot of memory, and the pause time hurts a lot. We solved this by, basically, disabling the garbage collector, and reusing memory manually as much as possible, which turned the program into almost a manually memory-managed affair.

vindarel 4y ago
They didn't try Allegro CL's new parallel GC (see comments). Otherwise, yeah. It depends on one's use case, I guess.
throwaway894345 4y ago
With respect to concurrency, after a lot of experience operating async Python services, the thing that I look for is a single, standard, ubiquitous concurrency model. One of the biggest pain points for operating Python systems is some transitive dependency making a synchronous I/O call and blocking the whole event loop, causing timeouts in unrelated endpoints and often bringing the whole system down (healthchecks start to fail because the endpoint is blocked and then the supervisor kills the whole process/pod/etc). This doesn't happen in Go or (presumably) languages that run on Erlang's BEAM VM because there is no sync I/O (from the event loop's perspective, anyway).
- gilch 4y ago
  Python has pre-emptive threading too! The GIL makes it all but useless for CPU-bound tasks, but it works fine for IO. Keep your health-check tasks in a non-IO thread and communicate with queues.
throw10920 4y ago

catchclose8919 4y ago

Which CL "interactive development" feature can't you get with either a jupyter notebook or ipython + autoreload enables? With a smart editor like VSCode or Spyder you also get visual variable inspections, history of executed code in one way or another etc. Only that can't easily save an "image" auto-magically, but data science and ML people wouldn't want to do this other than explicitly (eg. using fast loading formats like feather to reload semi-processed vars etc.). I don't think you are aware how much the Python workflows of AI / ML / datasci people resemble you "common lisp ideal" ...but with a different language and better libraries :) And if Julia succeeded, it will also bring decent performance to such workflows. Sure, the way backend devs or devops people use Python is very static and different, but any experienced ML practitioners are at home with "interactive development".

reikonomusha 4y ago
Re-define a class and have all objects automatically get updated to reflect the changes. Evaluate arbitrary expressions mid-stack trace. Run into a bug, get caught in an error, re-define the offensive function interactively, and re-run the computation before that point, all without stopping and restarting. (Example real-world use-case: You re-define one of your REST API endpoints without having to restart your server or interrupt the request-flow, be it in test or—if you so desire—in prod.) Change the class of an object at run-time. Re-compile and re-load individual functions from your editor with your production, git-committed code. This one is huge. You use interactive and incremental development to, well, develop! Not side-by-side with your code, but in your code as well, at a fine-enough grained level. Common Lisp was designed to be both interactive and incremental. It's not a bolt-on feature. Jupyter notebooks and auto-reloading have a very different feel, and (dis)incentivize different qualities and processes of development.
- catchclose8919 4y ago
  Basically all you mentioned above can be done in modern Python. Unfortunately I've seen it done... and God, the results are horrible, in Python. "Wait did calling this method just change the class of the object the method was on??!!!! Fuuuu..." But simple stuff like incrementally building classes method by method in eg. jupyter notebooks can be done and it's even quite clean, it's just that nobody is teaching people the techniques for doing it... (And the jupyter autoreload magic doesn't seem to like it so you'd have to "do it by hand") And sure, since Python got scoping and concurrency quite wrong, if you go full-interactive on things you'll shoot yourself in the foot faaast!

emptysongglass 4y ago

This reads out of touch with why curious people use Hy, like me who wanted to get going with an easily accessible language and ecosystem (Python) while rewriting bits in Hy to grok some lisp magic. No, of course I wouldn't have reached for CL first and it's disappointing to read the top-comment is this.

IshKebab 4y ago
But... by using Hy you're already learning a new super niche language. If you're going to learn a new language why not ditch Python and use something good? Ecosystem size is really the only thing I can think of.
- emptysongglass 4y ago
  Because I write Python to plug into a bigger ecosystem and I can mix and match Python and Hy freely. I can also use Poetry to package and manage dependencies and Poetry2nix to output a Nix derivation that is guaranteed to run on someone else's machine. I can also freely tap Python packages like Beautiful Soup or use frameworks like Django (which I am). The entire Python universe is available to me. The number of accessible tutorials for Django, videos for Pythonistas and community support channels is monumentally large. I even get to do cool things like use Django's ORM for learning how to interact with and manipulate data in Postgres, all right there available to me. I can use Discord (despite my distaste for it as a platform it has a huge Django and Python community), Telegram, IRC, and Matrix to ask for help and get metric tonnes of friendly advice, suggestions and criticisms from Pythonistas. And I can take all of this at my own pace, writing only bits in Hy and branching as I go. Every step is as gradual as I want it to be. Common Lisp is not the answer to my needs. Presenting it as so is tone deaf at the least.
  - vindarel 4y ago
    but who's presenting CL as the answer to your needs? (me?) If you're happy with your mix of Python and Hy, well, great. But you know what, you can do what you described with CL too! Need to do some webscraping? Beautiful Soup -> lquery. Django -> well… no CL framework gives you all these batteries included, but what if you want to extract data from a Postgres DB and display it in a web app? that's very easy [1] in CL too. Django ORM -> postmodern or Mito (or more). IDK about Nix but several people are working with CL and Guix. https://github.com/CodyReichert/awesome-cl https://github.com/CodyReichert/awesome-cl What worked out for me was to write companion services in CL, instead of extending my big Django app. This new service that takes data from a FTP server, transforms it, stores it, displays it in a web app? CL. That web app that serves another kind of customers than the Django one? A CL app. It isn't for everybody and sometimes you wish for a bigger ecosystem, but anyways, many things can be done in CL. [1]: ok honestly the learning process was not that easy. It's a bit easier with my contributions now! (Cookbook etc)

lf-non 4y ago

Can you elaborate on how CL provides more static type checks ? I am not much familiar with CL, but does it provide something similar to typed-racket ? Do major libraries actually use it ?

vindarel 4y ago
Not so easy question, especially because it depends of the implementation, but SBCL gives pretty good static type errors and warnings. Mostly, you get warnings at the function boundaries. It isn't complete like type checking in the modern languages, but it helps catch many errors. And quickly & interactively, because we compile our functions as we write them, with a keystroke (or the whole file, and from time to time, we build the whole project from scratch). So, coming from Python, that was superbly useful to me. This, everybody uses it. We can also add gradual typing to our variables and functions. It might help for the static checks, and it also gives hints for the compiler to speed things up. We can also declare our own types, but these won't be used for compile-time inference. For a type-racket equivalent, we now have Coalton, it's like a Haskell on top of CL: https://github.com/coalton-lang/coalton/ https://github.com/coalton-lang/coalton/ Its author says: -- Take this with a grain of salt, because I’m neither a user nor expert of Typed Racket, but: Typed Racket focuses more on the gradual typing of a given program. It has lots of features to make that easier, such as occurrence typing. Coalton is a separate, embedded language. Typed Racket achieves polymorphism through subtyping and and first-order type variables. Coalton achieves polymorphism through type variables, higher-kinded types, and type classes. Coalton, like ML and Haskell, focuses on defining objects by their properties and supported functions. This is a proposed way of having modular, reusable code. Typed Racket, as far as I can tell, has no such features. Coalton code can be fully inferred, so type annotations are not necessary. Typed Racket cannot. All in all, I think the biggest and most important take-away is that Typed Racket goes through great effort to seamlessly blend with ordinary Racket. But that means Typed Racket has to compromise on type system features that can only be supported if you’re willing to change the language itself. Coalton puts the type system first, opting for something close to Haskell, at the expense of not being a system for gradually typing Common Lisp, and instead being a separate language altogether. --- They use it for their open-source quantum compiler and other things.

pmarreck 4y ago

possibly because i've been playing with Guix lately, and also because all the smartest people I know seem to use lisp-likes, I'm on team "just use a Lisp" (possibly with some sort of API interface to Python)

gilch 4y ago

No closures? Hasn't it always had them?