6 ms·
I can gloss over LISP's lack of static strong typing and if I force myself a bit, I can also ignore it not producing small efficient native binaries... but the
by pdimitar 4y ago
I can gloss over LISP's lack of static strong typing and if I force myself a bit, I can also ignore it not producing small efficient native binaries... but the lack of (semi-)transparent concurrency / parallelism is my deal-breaker. Multicore CPUs are a fact for life for a long time now.
Where is the stuff like Erlang/Elixir's green threads or Golang's goroutines/channels and Rust's async runtimes workers/channels, and OCaml 5.0's recent multicore runtime (and the emerging libraries around it)?
People have been quick to point out various old-school multithreading libraries in the past but as a guy who was working with `pthreads` almost instantly after he started his career 21+ years ago... yeah, nope, I want the higher-level stuff. I want to solve problems and not invent a new async orchestration runtime for every project I work on.
If LISP is so fantastic why isn't there a serious effort to make it multicore-friendly? Where's the transparent concurrency and parallelism? The actor runtime? The [stateful] effect handlers?
---
Believe it or not, I understand the value of inventing your own DSL and that solving business problems with it is much easier compared to coding it in another programming language but (a) nobody wants to pay me for that because that way I make myself irreplaceable and business is extremely against this and (b) that solution might end up being non-extendable, rigid and a dead-end. So it's a high-risk endeavor in more ways than one.
So OK, LISP is awesome. I tried it, liked it, couldn't find a way to make money with it. I suspect many other programmers are in my boat.
If you want more adoption then do the grunt work and bring it to 2023. Give us an HTTP (1.x & 2.0) and WebSockets libraries, give us an actor runtime (or a goroutine-like one), maybe an optional type checker -- and I am sure the community can very easily pick it up afterwards and lift it up to big heights.
No? Then it remains a niche curiosity. Enlightenment does not have to always go hand-in-hand with being a spartan. I want libraries.
- jabradoodle 4y agoThere are staticly typed and gradually typed lisps. Clojure also has a similar concurrency to go with channels.
- Blackthorn 4y agoClojure's core.async channel concurrency is nothing like go's. You need to wrap your concurrent block in a macro so it can code walk to find explicit blocking points. That means these points need to be marked, and can't be in already compiled functions. That's an enormous limitation. Compare to go's which just works, no caveat.
- deleted 4y ago[deleted]
- ravi-delia 4y agoPlus Go is just so nice with the automatic IO handoff. But in a functional language I really don't think it's as big a limitation- still large, but not as bad as it could be. The pattern of writing mostly small, pure functions and then piping them together with channels all at the end is pretty workable.
- dataangel 4y agoCoalton doesn't really work (https://github.com/coalton-lang/coalton/issues/84?s=09 https://github.com/coalton-lang/coalton/issues/84?s=09) and Typed Racket is only typed outside macros (and I don't just mean that macros are hard to type, I mean the imperative code you write to generate code is not type checked). Any others?
- reikonomusha 4y agoI can't speak for dataangel's requirements but Coalton does work, as I discussed here [1]. [1] https://news.ycombinator.com/item?id=35863429 https://news.ycombinator.com/item?id=35863429
- Blackthorn 4y agoI'd kill for common lisp with transparent green threads. It would be so great. As you noted once you use them you can never really go back. Of course, it will never happen because it can only be done in the compiler/runtime, not in a library, and the standard is frozen in stone, so that's that. I've got some hope for clojure with that at least, thanks to it being jvm hosted.
- bcrosby95 4y agoHope no more - Java 21 will have virtual threads: https://openjdk.org/projects/jdk/21/ https://openjdk.org/projects/jdk/21/ Speaking of, I have yet to test how agents with virtual threads perform vs core.async for my use cases.
- varjag 4y agoBack in the 1990s the usual in CL implementations was green threads. Everyone kinda wished for POSIX threads support instead, which was eventually gained on most runtimes. You can still see the glimpse of them as atavisms in old code and documentation.
- Blackthorn 4y agoI indeed saw this years ago mentioned in clozurecl's documentation. I was very upset that all the support for it got ripped out of the language, because as shown with https://blog.linuxplumbersconf.org/2013/ocw/proposals/1653 https://blog.linuxplumbersconf.org/2013/ocw/proposals/1653 (which inspired what is now making its way into Java) this is the way to go for performant solutions with very very high thread counts.
- varjag 4y agoshrug I personally hate cooperative multitasking, and so far never ran into a problem that couldn't be scaled with system threads and use of select/poll. But am sure Google sees its uses.
- 4y ago
- lisper 4y agoClozure CL and Allegro CL have native threads. I think SBCL and Wraith Scheme do as well, but I'm not sure.
- pdimitar 4y agoWell, that's exactly what I don't care about. Most languages have native threads. I want an actor runtime with transparent green threads, for example. Like Erlang/Elixir.
- lisper 4y agoI don't think you understand what green threads are. Green threads are simulated threads that run on a single core. https://en.wikipedia.org/wiki/Green_thread https://en.wikipedia.org/wiki/Green_thread But you wrote: Multicore CPUs are a fact for life for a long time now. So I assumed that you didn't really want green threads and were just using the wrong terminology. In any case, you can get a Lisp with green threads too, though as you yourself point out that is considered pretty dated technology nowadays.
- pdimitar 4y agoIf you want to be a pedant ¯\_(ツ)_/¯ then OK (and I am not sure how "native threads" == "green threads" but I won't argue) -- look up Erlang/Elixir's model, or Golang's, or Rust's async workers. That's what I am looking for; basically M:N parallel execution model where M can be 50_000 and N (the number of CPU cores/threads) is no more than 32-64. The rest is a suboptimal mess and, as already mentioned above, I am not looking to invent my own async [task] orchestration runtime for every project I participate in.
- medo-bear 4y agohttps://github.com/CodyReichert/awesome-cl#parallelism-and-concurrency https://github.com/CodyReichert/awesome-cl#parallelism-and-c... https://github.com/thezerobit/green-threads https://github.com/thezerobit/green-threads edit: static typing https://github.com/coalton-lang/coalton https://github.com/coalton-lang/coalton
- tikhonj 4y ago> that solution might end up being non-extendable, rigid and a dead-end This is definitely something businesses are concerned about, but my experience has been that bog-standard enterprise-style codebases are way more likely to calcify than custom DSLs... but, somehow, it's never seen as a problem with standard technologies and approaches.
- pdimitar 4y agoYou won't catch me arguing with that. Like in communism: some [programming paradigmae] are more equal than others. I am sick of being managed by clueless MBAs but I also don't want to piss against the wind anymore.
- deleted 4y ago[deleted]
- vindarel 4y ago> static strong typing Alright, here is it: https://github.com/coalton-lang/coalton/ https://github.com/coalton-lang/coalton/ > small efficient native binaries The numbers are: with SBCL's core-compression, a web app with dozens on dependencies will weight ±30 to 40MB. This includes the compiler, the debugger, etc. Without core compression, we reach ±150MB. > The actor runtime? the actor library: https://github.com/mdbergmann/cl-gserver https://github.com/mdbergmann/cl-gserver > couldn't find a way to make money with it. I suspect many other programmers are in my boat. Alright. Some do, that's life. Yes, some companies go with CL even in 2023 (https://lisp-journey.gitlab.io/blog/lisp-interview-kina/ https://lisp-journey.gitlab.io/blog/lisp-interview-kina/, they released https://github.com/KinaKnowledge/juno-lang https://github.com/KinaKnowledge/juno-lang lately; Feetr (finance): https://twitter.com/feetr_io/status/1587182923911991303 https://twitter.com/feetr_io/status/1587182923911991303) https://github.com/azzamsa/awesome-lisp-companies/ https://github.com/azzamsa/awesome-lisp-companies/ > Give us an HTTP (1.x & 2.0) and WebSockets libraries How so? We have those libraries. HTTP/2: https://github.com/zellerin/http2/ https://github.com/zellerin/http2/ https://github.com/CodyReichert/awesome-cl https://github.com/CodyReichert/awesome-cl
- pdimitar 4y agoThanks.
- dataangel 4y agoCoalton doesn't actually work: https://github.com/coalton-lang/coalton/issues/84?s=09 https://github.com/coalton-lang/coalton/issues/84?s=09 This is why it's important to always look at the issues on a repo instead of just believing what's in the README. It fails to detect type errors in some of the most basic situations
- vindarel 4y agoInteresting. Fortunately, they are making clear Coalton didn't reach "v1.0".
- medo-bear 4y ago> This is why it's important to always look at the issues on a repo instead of just believing what's in the README this makes it sound like the developers are lying whereas the issue was raised by one of the core developers > fails to detect type errors in some of the most basic situations i dont think this is a basic situation. i wouldnt do mutation at coalton level, instead at lisp level
- lispm 4y agoMany Lisps (and Common Lisp) had 'green threads' decades ago. Green threads are not preemptively scheduled by the OS/hardware. Multi-Processor Lisps were new and very rare (like the Lisp&Scheme for BBN Butterfly https://en.wikipedia.org/wiki/BBN_Butterfly https://en.wikipedia.org/wiki/BBN_Butterfly or the Lisp for massive parallel Connection Machine). So the green threads then were mostly on a single core. Many Common Lisps moved to native threads in the last two decades because that provided preemptively scheduled threads on multi-core machines. Early examples were Corman Lisp (on Windows) and Scieneer CL (a commercial fork of CMUCL). Native threads have also advantages when interfacing to the OS. Implementations like SBCL, LispWorks, Allegro CL and others provide native threads and support multi-core machines. In the LispWorks development environment I use, every IDE tool runs in its own thread on a multicore CPU. The big problem most implementations face is getting a parallel and/or concurrent garbage collector. For the JVM such things exist in various forms. Allegro CL has something like a parallel GC. I would maybe use something like lparallel. https://lparallel.org/overview/ https://lparallel.org/overview/ Other libs are listed here: https://cliki.net/concurrency https://cliki.net/concurrency
- Paul-Craft 4y agoIt's funny, even Emacs lisp has threads now: https://www.gnu.org/software/emacs/manual/html_node/elisp/Threads.html https://www.gnu.org/software/emacs/manual/html_node/elisp/Th...
- deleted 4y ago[deleted]
- packetlost 4y agoGerbil Scheme has some of these: https://cons.io/ https://cons.io/
- pdimitar 4y agoTBF I liked Gerbil quite a bit when I tried it roughly a year ago. Don't remember why I gave up on it at this point though. Thanks for reminding me.
- packetlost 4y agoProbably the random missing documentation. To be fair, it's easy enough to navigate the source code but the real problem for me is discovering what's what and where to import it from.
- cutler 4y ago> If LISP is so fantastic why isn't there a serious effort to make it multicore-friendly? There is. It's called Clojure.
- pdimitar 4y agoOther commenters say it's nowhere close to what I'm looking for, btw. And let me repeat that I am not looking for the old-school OS threads support. Almost all programming languages have that. It's nowhere nearly good enough.
- kraf 4y agoIt has Go style concurrency, look for `core.async`. Maybe it could have been said that it wasn't lightweight enough and that would have been due to the JVM not providing the primitives but they're here now under the name "Virtual threads".
- pdimitar 4y agoIIRC somebody in this thread said it's not preemptive and doesn't have enough functionality. Quickly looking at it, it seems to be fully opt-in / cooperative parallelism solution which is IMO not good enough.
- ravi-delia 4y agoIt's...close-ish? It could really use default non-blocking IO, but honestly the reason no one's made one integrated solution yet is that giving a callback that pushes to a channel is mostly fine. I'd like something a little more robust obviously, but it's by no means a toy.
- pdimitar 4y agoOh definitely. I wouldn't call it a toy either, it's just weird that nobody bothered to make something more ready to consume.
- hot_gril 4y agoI agree about high-level parallelism. Having done it every which way, from C to Python (bleh) to Rust to Java to Erlang: NodeJS is actually nice with throng, at least for a webserver. And the built-in async/await stuff is your greenthreading. Easy to avoid race conditions because you know the event loop stays in context until you hit an `await`.
- thou24343434 4y agoCommon Lisp (CL) is not the entirety of the Lisp world. However, CL does have a lot of limitations given its 30-year old design. 1. The typing is vague, annoying and its weird half-assed existence to a CLOS is a constant source of trouble 2. Code-walking is not completely deterministic and is not implementation independent. 3. The environment is not part of the standard. CL was written in a completely different time when there was money to be made selling various bespoke implementations, and therefore there was a big need to vagueness in the spec. Today this need is much less so: there's 1-2 big open source impl. and about 2-3 commercial vendors.
- skissane 4y ago> CL was written in a completely different time when there was money to be made selling various bespoke implementations, and therefore there was a big need to vagueness in the spec. Today this need is much less so Part of the issue here is that the spec has essentially been abandoned, and frozen for the last almost 30 years. The committee which produced it decided to dissolve itself. If the surviving major commercial vendors and open source implementations thought it was in their interest, they could get together and update the standard for the 21st century, removing a lot of the vagueness and standardising the most widely implemented extensions. ANSI/INCITS is commonly criticised as an overly cumbersome and bureaucratic process, so maybe it would best be done by starting a new bespoke standardisation committee, like what WHAT WG did for HTML. However, I don’t think enough of the major players see it as sufficiently worth their while to invest in, which is why I doubt it will ever actually happen.
- thou24343434 4y agoAt this point you can't fix CL without creating a new language.
- skissane 4y agoWhy not just something like (require ‘cl-2029) to indicate one’s code is written to the new “Common Lisp 2029” standard? Which does its best to retain backward compatibility with existing code, but breaks compatibility whenever doing so is really worth it.
- LispSporks22 4y agoAll that stuff you're worried about is just implemented in libraries. Lisp lets you do that. You don't have to wait around years for a ECMA committee or whatnot to provide you a "async" keyword, or any other keyword. The language is flexible enough to let you add it yourself. It's a whole change of mindset. In practice however, several people probably implemented "async" independently and the community as a whole just settles around a favorite.
- pdimitar 4y agoAlso in practice: no ready-baked solution and I am not gonna move the ecosystem when I just want to do some commercial work with it, because nobody is gonna pay me to evolve the ecosystem. Maybe an article or two demonstrating step by step how does one invent their own async runtime exactly would be hugely helpful in terms of credibility. Otherwise it's just handwaving and I am sure you can see it.