4 ms·

Common Lisp also allows you to redefine everything at runtime but doesn't suffer from the same performance issues that Python has, does it? Doe anyone have ins

Common Lisp also allows you to redefine everything at runtime but doesn't suffer from the same performance issues that Python has, does it? Doe anyone have insight into this?

pjmlp 1y ago
Just like Smalltalk and SELF, also Lisp Machines and Interlisp-D. Usually comes down from a urban myth that Python is special and there was no other dynamic language before it came to be. The JIT research on those platforms is what gave us leading JIT capabilities on modern runtimes, OpenJDK HotSpot traces back to Smalltalk and StrongTalk, while V8 traces back to SELF. Especially in Smalltalk and SELF, you can change anything at any time across the whole image, and have the JIT pick up on that and re-optimize. Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available. A reason why JVM, CLR, V8, ART make use of handles and have marshaling layers not allowing such kind of liberties with native extensions.
- gjvc 1y ago
  Great explanation. Five years ago I did the genealogical work to discover that StrongTalk begat HotSpot (by virtue of having some of the same authors) It was quite a joy to discover!
- vidarh 1y ago
  > Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available. I'm doing a Ruby compiler (very, very slowly, though faster again now with Claude Code doing most of the heavy lifting), and the same issue with Ruby has made me seriously toy with the idea of one day embedding a C compiler so that the Ruby compiler can optimise across both (it'd still have to deal with linking to third party C code, of course, which is one reason I'm hesitating) as a simple not-so-optimizing C compiler is like a trivial toy compared to compiling Ruby where just the parser makes you want to claw your eyes out, but it'd at least widen the surface a bit.
  - pjmlp 1y ago
    Have you seen how TruffleRuby makes use of LLVM bitcode, to ship C compiler as well?
    - vidarh 1y ago
      I haven't, but not surprised given some of the very aggressive (in a good way) things they've done to address performance. It's impressive. I spoke to Chris Seaton back when it was getting started, and loved many of the things they were doing. Personally I want something leaner, and self-hosting, but that's also why mine is still experimental and wildly incomplete and TruffleRuby works for a lot of things.
      - pjmlp 1y ago
        Here, I don't know all the details, just remember having seen some talks about this in the past. It builds on top of this effort, https://medium.com/graalvm/graalvm-llvm-toolchain-f606f995bf https://medium.com/graalvm/graalvm-llvm-toolchain-f606f995bf "From Interpreting C Extensions to Compiling Them" https://eregon.me/blog/assets/research/from-interpreting-c-extensions-to-compiling-them.pdf https://eregon.me/blog/assets/research/from-interpreting-c-e...
        vidarh 1y ago
        Thanks, will definitely give it a read.
a-french-anon 1y ago
Common Lisp doesn't use (expensive) CLOS dispatch in the core language, e.g. to add two numbers or find the right equality operator. That's one known pain point due to CLOS having been "bolted-on" rather than part of the language which makes the divide between internal (using typecase and similar) and external (generic functions) dispatch pretty ugly; and gave use the eql/equal/equalp/etc... hell. Thing is that you need a complex JIT like Julia's or stuff like https://github.com/marcoheisig/fast-generic-functions https://github.com/marcoheisig/fast-generic-functions to offset the cost of constant dynamic dispatch. I actually had such a conversation on that comparison earlier this year: https://lwn.net/Articles/1032617/ https://lwn.net/Articles/1032617/
- psychoslave 1y ago
  What is CLOS in this context?
  - bradrn 1y ago
    The Common Lisp Object System: https://en.wikipedia.org/wiki/Common_Lisp_Object_System https://en.wikipedia.org/wiki/Common_Lisp_Object_System
  - brabel 1y ago
    Common Lisp Object System. The language amazing version of what OOP can be.
- martinflack 1y ago
  > and gave use the eql/equal/equalp/etc... hell. You don't like those? I've always considered them a fairly elegant deconstruction of the problem domain of equality checking. DWIM languages can get very confusing when they DWIM or don't DWIM.
  - a-french-anon 1y ago
    Where's the elegance? Equality is well defined on everything handled by those three, since they only compare the same types without doing coercion. Plus, you can't extend these to handle user types. Which is why https://cdr.common-lisp.dev/document/8/cleqcmp.html https://cdr.common-lisp.dev/document/8/cleqcmp.html exists, really; all the "copy-x" would benefit from the same fix, in my opinion.
brabel 1y ago
Common Lisp is not a runtime, it’s a specification. Implementations are free to compile everything to fast native code, or to interpret everything. Various available implementations do that and everything in between. That said , SBCL and the commercial implementations can be extremely fast, especially if you specify types on tight loops. SBCL comes with a disassembler that shows you right in the REPL the Assembly a function compiles to so you can even get close to C performance.
- pjmlp 1y ago
  Addedum that having a disassembler is quite common language primitive in most compiled Lisps, since early days.

Common Lisp also allows you to redefine everything at runtime but doesn't suffer from the same performance issues that Python has, does it? Doe anyone have insight into this?

pjmlp 1y ago

Just like Smalltalk and SELF, also Lisp Machines and Interlisp-D. Usually comes down from a urban myth that Python is special and there was no other dynamic language before it came to be. The JIT research on those platforms is what gave us leading JIT capabilities on modern runtimes, OpenJDK HotSpot traces back to Smalltalk and StrongTalk, while V8 traces back to SELF. Especially in Smalltalk and SELF, you can change anything at any time across the whole image, and have the JIT pick up on that and re-optimize. Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available. A reason why JVM, CLR, V8, ART make use of handles and have marshaling layers not allowing such kind of liberties with native extensions.

gjvc 1y ago
Great explanation. Five years ago I did the genealogical work to discover that StrongTalk begat HotSpot (by virtue of having some of the same authors) It was quite a joy to discover!
vidarh 1y ago
> Granted what messes up Python, or better said CPython implemenation, is that C extensions are allowed to mess up with its internals thus making void many possible optimizations that would be otherwise available. I'm doing a Ruby compiler (very, very slowly, though faster again now with Claude Code doing most of the heavy lifting), and the same issue with Ruby has made me seriously toy with the idea of one day embedding a C compiler so that the Ruby compiler can optimise across both (it'd still have to deal with linking to third party C code, of course, which is one reason I'm hesitating) as a simple not-so-optimizing C compiler is like a trivial toy compared to compiling Ruby where just the parser makes you want to claw your eyes out, but it'd at least widen the surface a bit.
- pjmlp 1y ago
  Have you seen how TruffleRuby makes use of LLVM bitcode, to ship C compiler as well?
  - vidarh 1y ago
    I haven't, but not surprised given some of the very aggressive (in a good way) things they've done to address performance. It's impressive. I spoke to Chris Seaton back when it was getting started, and loved many of the things they were doing. Personally I want something leaner, and self-hosting, but that's also why mine is still experimental and wildly incomplete and TruffleRuby works for a lot of things.
    - pjmlp 1y ago
      Here, I don't know all the details, just remember having seen some talks about this in the past. It builds on top of this effort, https://medium.com/graalvm/graalvm-llvm-toolchain-f606f995bf https://medium.com/graalvm/graalvm-llvm-toolchain-f606f995bf "From Interpreting C Extensions to Compiling Them" https://eregon.me/blog/assets/research/from-interpreting-c-extensions-to-compiling-them.pdf https://eregon.me/blog/assets/research/from-interpreting-c-e...
      - vidarh 1y ago
        Thanks, will definitely give it a read.

a-french-anon 1y ago

Common Lisp doesn't use (expensive) CLOS dispatch in the core language, e.g. to add two numbers or find the right equality operator. That's one known pain point due to CLOS having been "bolted-on" rather than part of the language which makes the divide between internal (using typecase and similar) and external (generic functions) dispatch pretty ugly; and gave use the eql/equal/equalp/etc... hell. Thing is that you need a complex JIT like Julia's or stuff like https://github.com/marcoheisig/fast-generic-functions https://github.com/marcoheisig/fast-generic-functions to offset the cost of constant dynamic dispatch. I actually had such a conversation on that comparison earlier this year: https://lwn.net/Articles/1032617/ https://lwn.net/Articles/1032617/

psychoslave 1y ago
What is CLOS in this context?
- bradrn 1y ago
  The Common Lisp Object System: https://en.wikipedia.org/wiki/Common_Lisp_Object_System https://en.wikipedia.org/wiki/Common_Lisp_Object_System
- brabel 1y ago
  Common Lisp Object System. The language amazing version of what OOP can be.
martinflack 1y ago
> and gave use the eql/equal/equalp/etc... hell. You don't like those? I've always considered them a fairly elegant deconstruction of the problem domain of equality checking. DWIM languages can get very confusing when they DWIM or don't DWIM.
- a-french-anon 1y ago
  Where's the elegance? Equality is well defined on everything handled by those three, since they only compare the same types without doing coercion. Plus, you can't extend these to handle user types. Which is why https://cdr.common-lisp.dev/document/8/cleqcmp.html https://cdr.common-lisp.dev/document/8/cleqcmp.html exists, really; all the "copy-x" would benefit from the same fix, in my opinion.

brabel 1y ago

Common Lisp is not a runtime, it’s a specification. Implementations are free to compile everything to fast native code, or to interpret everything. Various available implementations do that and everything in between. That said , SBCL and the commercial implementations can be extremely fast, especially if you specify types on tight loops. SBCL comes with a disassembler that shows you right in the REPL the Assembly a function compiles to so you can even get close to C performance.

pjmlp 1y ago
Addedum that having a disassembler is quite common language primitive in most compiled Lisps, since early days.