6 ms·

I write Clojure for food, and Common Lisp for fun. One reason for the latter is CL's speed -- awhile back I compared a bit of (non-optimized) Clojure code I wro

I write Clojure for food, and Common Lisp for fun. One reason for the latter is CL's speed -- awhile back I compared a bit of (non-optimized) Clojure code I wrote for a blog post with a rough equivalent in CL, and was stunned that the Common Lisp code ran about 10x faster. This made me curious as to how fast it could be made if I really tried, and was able to get nearly 30x more[1] by optimizing it. Clojure is definitely fast enough for everything I've done professionally for six years. But Common Lisp, while having plenty of rough edges, intrigues on the basis of performance alone. (This is on SBCL -- I have yet to play with a commercial implementation.) [1] http://johnj.com/from-elegance-to-speed.html http://johnj.com/from-elegance-to-speed.html

lmilcin 5y ago
I have implemented real algotrading framework in Common Lisp (SBCL) that connected directly to Warsaw Stock Exchange. The application was receiving binary messages from the exchange over multicast, rebuilding state of the market, running various (simple) algorithms and responding with orders within 5 microseconds of the original message, at up to 10k messages per second. With SBCL you can write a DSL and have ability to fully control the resulting instructions (through vops). It is just the question how dedicated you are to writing macros upon macros upon macros. I used this to the fullest extent and the result was as good as any hand optimized C/assembly. For example, the binary message parser would receive a stack of complicated specifications for the messages in the form of XML files (see here if you are curious: https://www.gpw.pl/pub/GPW/files/WSE_CDE_XDP_Message_Specifications_-_v5.9b.pdf https://www.gpw.pl/pub/GPW/files/WSE_CDE_XDP_Message_Specifi...), converted XML to DSL and then, through magic of macros, the DSL was converted to accessors that allowed optimal access to the fields. Optimal here mans the assembly could not be improved upon any further. Large parts of the application (especially any communication and device control as it was done with full kernel bypass) was written in ANSI C but the Foreign Function Interface makes integrating it into the rest of application a breeze. I write all of this, because I frequently meet complete disbelief from people that Lisp can be used in production. I personally think it is exactly the opposite. Lisps offer fantastic development environment. The problem are developers who are mostly unable to use Lisp for work effectively, I guess due to too much power, freedom and complete lack of direction on how to structure your application.
- jamtho 5y ago
  This is fascinating. Did you come across any good resources to help learn how to optimise aggressively on SBCL, as you're describing?
  - lmilcin 5y ago
    This was many years ago and I don't remember exact sources. It was more of a research than a normal development project. I spent most of my time researching techniques needed to write that kind of application. The largest influence were definitely LMAX articles and Disruptor pattern. SBCL was comparatively easy. Basically, if anything caused problems I just moved it to C and called using FFI. Think in terms of writing a C program but using pieces of assembly when C is not enough for some reason. Even though a large part of small pieces was moved to C, the application still felt like Lisp. It just orchestrated a large library of utilities to do various things. I still had fully functional REPL, for example. The hard part of the project was full kernel bypass. After the initial setup, the application stopped talking to Linux kernel except for one CPU core that was devoted to running OS threads, some necessary applications and some non-performance-critical threads of the algotrading framework (like REPL, persistence, etc). All except for one core were completely owned each by a single thread of the application and never did any context switch after initial setup.
    - jamtho 5y ago
      Thank you, that's very helpful. I've found the cepl videos instructive in a simlar area https://www.youtube.com/user/CBaggers/videos https://www.youtube.com/user/CBaggers/videos I hadn't appreciated that level of thread isolation was possible in SBCL, but of course it makes sense. Presumably you had some kind of instrumentation to let you know if a stray syscall had slipped in?
      - lmilcin 5y ago
        > Presumably you had some kind of instrumentation to let you know if a stray syscall had slipped in? Don't know if you can call it instrumentation... I wrote an extremely hacky patch for the kernel to detect when any piece of kernel is running on anything than core 0 after certain flag was set. Remember, it is not just syscalls. Even something as simple as accessing memory can cause switch to kernel to resolve TLB entry if you don't set up your memory correctly to prevent this from happening. Yeah... I know. Now a bunch of people will come and explain how this could be done the right way. I just didn't care at the time to invest more time than necessary to get this right.
NoahTheDuke 5y ago
Your post is a lot of fun! I have a fondness for these kinds of deep dives. That being said, I feel like comparing the initial unoptimized Clojure code to highly optimized Common Lisp is kind of unfair. I wonder how fast the Clojure code could run if given the same care. Maybe I'll give that a try tonight!
- eigenhombre 5y ago
  That would be great! I agree it's not a fair comparison -- the post was meant more along the lines of, "how far can I push Common Lisp and learn some things?" rather than a strict comparison of performance of optimized code in each language. As I said, Clojure is fast enough for my purposes nearly all the time.
  - NoahTheDuke 5y ago
    I wrote it up here: http://noahtheduke.github.io/posts/2021-10-02-from-elegance-to-speed-with-clojure/ http://noahtheduke.github.io/posts/2021-10-02-from-elegance-... I'm not a great writer, but hopefully you enjoy it! I also found a small bug, that you'll want to use `(elt s (+ x 7))`, not `(+ x 8)`. `elt` is 0-indexed, so first and last of an 8 element list will be 0 and 7.
- ludston 5y ago
  You're going to be severely constrained by the fact that idiomatic clojure code uses immutable data structures which cannot theoretically be as fast as mutable ones in sbcl. Even with transience, the Clojure hash map trie is an order of magnitude slower than sbcl's hash table.
  - NoahTheDuke 5y ago
    Of course. I don't mean to imply that optimized Clojure could match optimized Common Lisp, just that the disparity wouldn't be quite as stark. For truly speed-focused code on the JVM, you have to write speed-focused Java.
vindarel 5y ago
It would be cool if you updated the link to the Cookbook in your article from the old one (on Sourceforge) to the newer one (on Github): https://lispcookbook.github.io/cl-cookbook/ https://lispcookbook.github.io/cl-cookbook/ Best,
- eigenhombre 5y ago
  I will, thank you!
gleenn 5y ago
I also write Clojure for food, but also for fun. I found that a lot of Clojure is fast enough but also speeding it up is relatively easy with some type hinting and perhaps dipping into mutable data structures or other tricks. Relevant Stackoverflow which contains a lot of simple suggestions to hopefully shore up the deficit compared to SBCL: https://stackoverflow.com/questions/14115980/clojure-performance-really-bad-on-simple-loop-versus-java#14116215 https://stackoverflow.com/questions/14115980/clojure-perform...
earthscienceman 5y ago
Your website inspires me, I've been on a similar career trajectory to you in some ways although at the opposite end of the earth. Starting in Physics and moving away in time, although I'm only 33 now. I love that you're putting your art on display. It's very rare to find scientists pursuing art in such a direct way.
didibus 5y ago
I'm sorry to say, but I'm not a fan of articles like this, you're not using the same data-structures and algorithms, so what's the point? To compare language compiler and runtime performance you should at least use similar data-structures and algorithms. > but, you can generate very fast (i.e., more energy efficient) I'm actually not sure this is true, I was surprised the other day to find a study on this and to find that the JVM was one of the most energy efficient runtime. This was the link: https://thenewstack.io/which-programming-languages-use-the-least-electricity/ https://thenewstack.io/which-programming-languages-use-the-l... And what you can see is that execution time doesn't always mean more energy efficient. For example you can look at Go and see that it beats a lot of things in execution time, but loses to those in energy efficiency. Like how Go was faster than Lisp yet less energy efficient than Lisp.
- hajile 5y ago
  Quite a few lisp implementations allow you to drop into inline assembly. That makes them theoretically per close to maximum efficiency.
  - didibus 5y ago
    I'm not sure in what way you're using the word "efficiency" here? The research showed that the Go version of the programs were in average faster in terms of execution time and consumed less memory than the alternate Lisp versions, yet the Lisp versions consumed less electricity and were thus more energy efficient. So the interesting bit here is that better performance and lower memory consumption doesn't always mean more energy efficient. Now, there is definitely a link between performance and energy use, the research did show that for the most part, faster execution often reflected in lower energy spent, but there were surprising variation within that range where it wasn't always clear cut.
_bsless 5y ago
Others have made similar comments on comparing apples to oranges when comparing optimized CL to idiomatic Clojure, but what they didn't address was "idiomatic Clojure". Threading a sequence through a bunch of lazy operations is good for development time, but at this point I wouldn't consider it idiomatic for "production" code. Two things in the implementation are performance killers: Laziness and boxed maths. - baseline: Taking your original implementation and running it on a sequence or 1e6 elements I generated, I start off at 1.2 seconds. - Naive transducer: Needs a transducer of a sliding window which doesn't exist in the core yet[0], 470ms - Throw away function composition, use peek and nth to access the vector: 54ms - Map & filter -> keep: 49ms - Vectors -> arrays: 29ms I'd argue only the last step makes the code slightly less idiomatic. Might even say that aggressively using juxt, partial and apply is less idiomatic than simple lambdas You can see the implementation here [0] https://gist.github.com/nornagon/03b85fbc22b3613087f6 https://gist.github.com/nornagon/03b85fbc22b3613087f6 [1] https://gist.github.com/bsless/0d9863424a00abbd325327cff1ea0a04 https://gist.github.com/bsless/0d9863424a00abbd325327cff1ea0... Edit: formatting
kaba0 5y ago
I would be interested in at least the performance of type hinted clojure code.
fulafel 5y ago
The closest translation of the code translation, having already dropped the laziness of the Clojure version, was 4x faster. The rest of the speedups came from rewriting the code!

lmilcin 5y ago

I have implemented real algotrading framework in Common Lisp (SBCL) that connected directly to Warsaw Stock Exchange. The application was receiving binary messages from the exchange over multicast, rebuilding state of the market, running various (simple) algorithms and responding with orders within 5 microseconds of the original message, at up to 10k messages per second. With SBCL you can write a DSL and have ability to fully control the resulting instructions (through vops). It is just the question how dedicated you are to writing macros upon macros upon macros. I used this to the fullest extent and the result was as good as any hand optimized C/assembly. For example, the binary message parser would receive a stack of complicated specifications for the messages in the form of XML files (see here if you are curious: https://www.gpw.pl/pub/GPW/files/WSE_CDE_XDP_Message_Specifications_-_v5.9b.pdf https://www.gpw.pl/pub/GPW/files/WSE_CDE_XDP_Message_Specifi...), converted XML to DSL and then, through magic of macros, the DSL was converted to accessors that allowed optimal access to the fields. Optimal here mans the assembly could not be improved upon any further. Large parts of the application (especially any communication and device control as it was done with full kernel bypass) was written in ANSI C but the Foreign Function Interface makes integrating it into the rest of application a breeze. I write all of this, because I frequently meet complete disbelief from people that Lisp can be used in production. I personally think it is exactly the opposite. Lisps offer fantastic development environment. The problem are developers who are mostly unable to use Lisp for work effectively, I guess due to too much power, freedom and complete lack of direction on how to structure your application.

jamtho 5y ago
This is fascinating. Did you come across any good resources to help learn how to optimise aggressively on SBCL, as you're describing?
- lmilcin 5y ago
  This was many years ago and I don't remember exact sources. It was more of a research than a normal development project. I spent most of my time researching techniques needed to write that kind of application. The largest influence were definitely LMAX articles and Disruptor pattern. SBCL was comparatively easy. Basically, if anything caused problems I just moved it to C and called using FFI. Think in terms of writing a C program but using pieces of assembly when C is not enough for some reason. Even though a large part of small pieces was moved to C, the application still felt like Lisp. It just orchestrated a large library of utilities to do various things. I still had fully functional REPL, for example. The hard part of the project was full kernel bypass. After the initial setup, the application stopped talking to Linux kernel except for one CPU core that was devoted to running OS threads, some necessary applications and some non-performance-critical threads of the algotrading framework (like REPL, persistence, etc). All except for one core were completely owned each by a single thread of the application and never did any context switch after initial setup.
  - jamtho 5y ago
    Thank you, that's very helpful. I've found the cepl videos instructive in a simlar area https://www.youtube.com/user/CBaggers/videos https://www.youtube.com/user/CBaggers/videos I hadn't appreciated that level of thread isolation was possible in SBCL, but of course it makes sense. Presumably you had some kind of instrumentation to let you know if a stray syscall had slipped in?
    - lmilcin 5y ago
      > Presumably you had some kind of instrumentation to let you know if a stray syscall had slipped in? Don't know if you can call it instrumentation... I wrote an extremely hacky patch for the kernel to detect when any piece of kernel is running on anything than core 0 after certain flag was set. Remember, it is not just syscalls. Even something as simple as accessing memory can cause switch to kernel to resolve TLB entry if you don't set up your memory correctly to prevent this from happening. Yeah... I know. Now a bunch of people will come and explain how this could be done the right way. I just didn't care at the time to invest more time than necessary to get this right.

NoahTheDuke 5y ago

Your post is a lot of fun! I have a fondness for these kinds of deep dives. That being said, I feel like comparing the initial unoptimized Clojure code to highly optimized Common Lisp is kind of unfair. I wonder how fast the Clojure code could run if given the same care. Maybe I'll give that a try tonight!

eigenhombre 5y ago
That would be great! I agree it's not a fair comparison -- the post was meant more along the lines of, "how far can I push Common Lisp and learn some things?" rather than a strict comparison of performance of optimized code in each language. As I said, Clojure is fast enough for my purposes nearly all the time.
- NoahTheDuke 5y ago
  I wrote it up here: http://noahtheduke.github.io/posts/2021-10-02-from-elegance-to-speed-with-clojure/ http://noahtheduke.github.io/posts/2021-10-02-from-elegance-... I'm not a great writer, but hopefully you enjoy it! I also found a small bug, that you'll want to use `(elt s (+ x 7))`, not `(+ x 8)`. `elt` is 0-indexed, so first and last of an 8 element list will be 0 and 7.
ludston 5y ago
You're going to be severely constrained by the fact that idiomatic clojure code uses immutable data structures which cannot theoretically be as fast as mutable ones in sbcl. Even with transience, the Clojure hash map trie is an order of magnitude slower than sbcl's hash table.
- NoahTheDuke 5y ago
  Of course. I don't mean to imply that optimized Clojure could match optimized Common Lisp, just that the disparity wouldn't be quite as stark. For truly speed-focused code on the JVM, you have to write speed-focused Java.

vindarel 5y ago

It would be cool if you updated the link to the Cookbook in your article from the old one (on Sourceforge) to the newer one (on Github): https://lispcookbook.github.io/cl-cookbook/ https://lispcookbook.github.io/cl-cookbook/ Best,

eigenhombre 5y ago
I will, thank you!

gleenn 5y ago

I also write Clojure for food, but also for fun. I found that a lot of Clojure is fast enough but also speeding it up is relatively easy with some type hinting and perhaps dipping into mutable data structures or other tricks. Relevant Stackoverflow which contains a lot of simple suggestions to hopefully shore up the deficit compared to SBCL: https://stackoverflow.com/questions/14115980/clojure-performance-really-bad-on-simple-loop-versus-java#14116215 https://stackoverflow.com/questions/14115980/clojure-perform...

earthscienceman 5y ago

Your website inspires me, I've been on a similar career trajectory to you in some ways although at the opposite end of the earth. Starting in Physics and moving away in time, although I'm only 33 now. I love that you're putting your art on display. It's very rare to find scientists pursuing art in such a direct way.

didibus 5y ago

I'm sorry to say, but I'm not a fan of articles like this, you're not using the same data-structures and algorithms, so what's the point? To compare language compiler and runtime performance you should at least use similar data-structures and algorithms. > but, you can generate very fast (i.e., more energy efficient) I'm actually not sure this is true, I was surprised the other day to find a study on this and to find that the JVM was one of the most energy efficient runtime. This was the link: https://thenewstack.io/which-programming-languages-use-the-least-electricity/ https://thenewstack.io/which-programming-languages-use-the-l... And what you can see is that execution time doesn't always mean more energy efficient. For example you can look at Go and see that it beats a lot of things in execution time, but loses to those in energy efficiency. Like how Go was faster than Lisp yet less energy efficient than Lisp.

hajile 5y ago
Quite a few lisp implementations allow you to drop into inline assembly. That makes them theoretically per close to maximum efficiency.
- didibus 5y ago
  I'm not sure in what way you're using the word "efficiency" here? The research showed that the Go version of the programs were in average faster in terms of execution time and consumed less memory than the alternate Lisp versions, yet the Lisp versions consumed less electricity and were thus more energy efficient. So the interesting bit here is that better performance and lower memory consumption doesn't always mean more energy efficient. Now, there is definitely a link between performance and energy use, the research did show that for the most part, faster execution often reflected in lower energy spent, but there were surprising variation within that range where it wasn't always clear cut.

_bsless 5y ago

Others have made similar comments on comparing apples to oranges when comparing optimized CL to idiomatic Clojure, but what they didn't address was "idiomatic Clojure". Threading a sequence through a bunch of lazy operations is good for development time, but at this point I wouldn't consider it idiomatic for "production" code. Two things in the implementation are performance killers: Laziness and boxed maths. - baseline: Taking your original implementation and running it on a sequence or 1e6 elements I generated, I start off at 1.2 seconds. - Naive transducer: Needs a transducer of a sliding window which doesn't exist in the core yet[0], 470ms - Throw away function composition, use peek and nth to access the vector: 54ms - Map & filter -> keep: 49ms - Vectors -> arrays: 29ms I'd argue only the last step makes the code slightly less idiomatic. Might even say that aggressively using juxt, partial and apply is less idiomatic than simple lambdas You can see the implementation here [0] https://gist.github.com/nornagon/03b85fbc22b3613087f6 https://gist.github.com/nornagon/03b85fbc22b3613087f6 [1] https://gist.github.com/bsless/0d9863424a00abbd325327cff1ea0a04 https://gist.github.com/bsless/0d9863424a00abbd325327cff1ea0... Edit: formatting

kaba0 5y ago

I would be interested in at least the performance of type hinted clojure code.

fulafel 5y ago

The closest translation of the code translation, having already dropped the laziness of the Clojure version, was 4x faster. The rest of the speedups came from rewriting the code!