8 ms·

This project looks awesome, though I'm a bit sad that "performance" is a non-goal as stated in the README. Clojure being able to compile to a single shippable b

by masijo 5y ago

This project looks awesome, though I'm a bit sad that "performance" is a non-goal as stated in the README. Clojure being able to compile to a single shippable binary is just what I need in my life. Kudos to the author.

bpicolo 5y ago
Can do that with Graal these days: https://github.com/BrunoBonacci/graalvm-clojure/blob/master/doc/clojure-graalvm-native-binary.md https://github.com/BrunoBonacci/graalvm-clojure/blob/master/...
- uDontKnowMe 5y ago
  I explored this briefly but it ended up taking about 25 seconds to compile hello-world in Clojure. That struck me as causing a lot of pain to develop for. That, combined with how you'd lose access to a lot of the java library ecosystem, makes it less appealing to me to develop on.
  - dig1 5y ago
    True, but you don't develop that way using graal. You develop java apps the usual way (eclipse, emacs/vim/maven, whatever) and create a fat jar (or uberjar). When you make sure the application is working, compile it with graal, creating a static binary. Clojure cycle is even faster because you keep REPL open and evaluate code directly while application is running. After that, you build uberjar, then compile it with graal.
    - uDontKnowMe 5y ago
      Would there not be a risk that you may get something working nicely in openjdk and then you go to do a native compilation, and it turns out that it doesn't work due to some new method call which uses reflection or something?
      - didibus 5y ago
        Reflection is supported by GraalVM native compilation, but you need to declare what classes you need reflection support for at compilation. What you must understand though, is that you're asking for something contradictory. You cannot get runtime eval and code linking and also deliver a statically linked compiled machine code binary. That's why statically linked to machine code binary languages don't offer dynamic code generation, linking and reflection that can't be declared or performed at compile time. This is true for GraalVM. It means when you develop Clojure for GraalVM native compilation you can't leverage such dynamic behavior as well. Though you can embed the SCI Clojure interpreter inside your application and do dynamic code evaluation with it at runtime.
      - deleted 5y ago
        [deleted]
      - jgrodziski 5y ago
        GraalVM issue an error if it can't resolve the proper method call or if another Thread is involved in the computation so I find it pretty safe to use. Of course, GraalVM doesn't exempt you from doing some tests.
chitowneats 5y ago
Have you tried Babashka? https://github.com/babashka/babashka https://github.com/babashka/babashka
- dimitar 5y ago
  babashka also has a performance non-goal
  - chitowneats 5y ago
    Ah. You are correct.
  - didibus 5y ago
    That's true, but in my poor man tests, Babashka was in the ballpark of Python for me.
    - chitowneats 5y ago
      I was surprised when I realized I had overlooked the performance non-goal considering how fast it has been in my limited experience.
      - didibus 5y ago
        Ya, it even supports multi-core programming, so I bet for some use case it can outperform Python by parallelizing.
  - Borkdude 5y ago
    The performance non-goal is there to lower expectations compared to compiled Clojure. Babashka's author (me) does try to make it as fast as possible within the constraints it's implemented in.
vnorilo 5y ago
I have long had the ambition to be the N:th guy to try Clojure on C++. For similar reasons. I have a pretty good idea how to do it, already got HAMT and some macroexpanded code to work. But then you remember how even ClojureCLR struggled at times, and that you'd be lightyears behind that, not to speak of CLJ/CLJS, and alone. Maybe one day? :)
- jb1991 5y ago
  It would seem to me the most significant challenge would be having functional data structures in C++, that would be a non-trivial effort I would think. Even if you were to use other libraries that experiment with this. Without those, the language just doesn’t make any sense.
  - toymachine1974 5y ago
    My Park language does this https://github.com/toymachine/park-lang https://github.com/toymachine/park-lang Currently the syntax is mostly JS but the semantics is all clojure. I recreated the map and vector datatypes in c++ by adapting clojures Java code. it needs a GC though which is another big part of the implemention of Park
  - vnorilo 5y ago
    Yeah, that was the problem with most attempts I saw, unordered_map won't work. I did write a persistent hash map and vector [1], but ended up mostly using them in C++ projects. I wouldn't advise anyone else to use them at this level of testing/maturity though. Writing them was super interesting. 1: https://hg.sr.ht/~vnorilo/pcoll/browse https://hg.sr.ht/~vnorilo/pcoll/browse (not production quality)
  - bertmuthalaly 5y ago
    Immer [0][1] seems like a really serious attempt at bringing high-quality persistent data structures to C++. [0]: https://www.youtube.com/watch?v=sPhpelUfu8Q https://www.youtube.com/watch?v=sPhpelUfu8Q [1]: https://sinusoid.es/immer/ https://sinusoid.es/immer/
- geokon 5y ago
zerr 5y ago
What would be the advantage over other Lisps if you won't be able to leverage JVM ecosystem?
- leephillips 5y ago
  Some people like Clojure's data structures, which add a couple of things to lists. Also, there's ClojureScript, which is huge. And, ideas such as software transactional memory.
- didibus 5y ago
  Depending who you ask, Clojure provides some benefits over other Lisps unrelated to its ability to leverage the JVM. Some of those are: - Default efficient immutable collections - Good support for safe concurrency and multi-core programming - Extended homoiconicity to include maps, vectors and sets as well as lists - Convenient short lambda syntax - Extensible data representation - Distinguishes between nil and empty list - Distinguishes between keywords and symbols - Real boolean types - Function arity overloading - Well thought out abstractions and set of standard functions: collections are abstractions, lazy sequences, transducers, relational set manipulation, etc. - Only two kinds of equality: by value or by reference, with the former being the default - Is a lisp-1 like Scheme, but also with more batteries included like Common Lisp - Slightly different visual appearance in its syntax, due to support for literal maps and vectors as well as lists, which can make things easier to visually parse.
skybrian 5y ago
It’s difficult to write a fast interpreter in Go. It doesn’t optimize interpreter inner loops particularly well, you can’t do the data structure tricks you can do in C, and there is no JIT compiler available like in Java. The best way forward might be an option to compile some Joker code by generating Go code for it. (This is based on writing an interpreter a few years ago. Perhaps Go’s compiler has improved?)
- AtlasBarfed 5y ago
  What C tricks are used? I'd be fascinated to know some details.
  - spijdar 5y ago
    Caveat: I've barely used Go personally. Just speculating on what GP specifically meant, but Go lacks the ability to pack structs, and probably lacks the equivalent of tricks like computed gotos [0] to increase bytecode interpreter speeds. In general, Go seems to (intentionally) lack a lot of low-level control of code generation, preferring a "there's one way to do things, and it either works, or it's a bug we'll fix" approach. Which is probably for the best in "most" software, but interpreters typically use weird hacks to squeeze more performance out of the rock. Wren is a good example [1] of some of these optimizations, and has splendid comments/documentation. [0] https://eli.thegreenplace.net/2012/07/12/computed-goto-for-efficient-dispatch-tables https://eli.thegreenplace.net/2012/07/12/computed-goto-for-e... [1] https://github.com/wren-lang/wren/blob/main/src/vm/wren_vm.c https://github.com/wren-lang/wren/blob/main/src/vm/wren_vm.c
  - skybrian 5y ago
    For one thing you have C unions. You can store different types in the same place and use information stored elsewhere to know what the bits really mean. It’s horribly unsafe compared to a Go interface or the tagged unions in other languages, but uses less space. Also, there is the NaN boxing trick [1], which allows you to store a value that is essentially a tagged union that is either a double, a pointer, or an integer, in the same space as a double. To get these in a safe language, they essentially have to be built in already. Go’s interface type is very high overhead compared to the union types used in other languages because it uses an entire pointer as a tag. Though, it only matters in special cases like interpreters. Most performance-intensive code isn’t nearly so dynamic. [1] https://sean.cm/a/nan-boxing https://sean.cm/a/nan-boxing
didibus 5y ago
You can just use GraalVM for that. Also, Joker does not let you do what you want, it is not Clojure hosted on the Go runtime, it is a Clojure interpreter runtime implemented in Go, so with Joker you always need the joker binary. If you want to produce native shippable statically linked binaries with Clojure look at GraalVM native compilation: https://github.com/lread/clj-graal-docs https://github.com/lread/clj-graal-docs

by masijo 5y ago

bpicolo 5y ago

Can do that with Graal these days: https://github.com/BrunoBonacci/graalvm-clojure/blob/master/doc/clojure-graalvm-native-binary.md https://github.com/BrunoBonacci/graalvm-clojure/blob/master/...

uDontKnowMe 5y ago
I explored this briefly but it ended up taking about 25 seconds to compile hello-world in Clojure. That struck me as causing a lot of pain to develop for. That, combined with how you'd lose access to a lot of the java library ecosystem, makes it less appealing to me to develop on.
- dig1 5y ago
  True, but you don't develop that way using graal. You develop java apps the usual way (eclipse, emacs/vim/maven, whatever) and create a fat jar (or uberjar). When you make sure the application is working, compile it with graal, creating a static binary. Clojure cycle is even faster because you keep REPL open and evaluate code directly while application is running. After that, you build uberjar, then compile it with graal.
  - uDontKnowMe 5y ago
    Would there not be a risk that you may get something working nicely in openjdk and then you go to do a native compilation, and it turns out that it doesn't work due to some new method call which uses reflection or something?
    - didibus 5y ago
      Reflection is supported by GraalVM native compilation, but you need to declare what classes you need reflection support for at compilation. What you must understand though, is that you're asking for something contradictory. You cannot get runtime eval and code linking and also deliver a statically linked compiled machine code binary. That's why statically linked to machine code binary languages don't offer dynamic code generation, linking and reflection that can't be declared or performed at compile time. This is true for GraalVM. It means when you develop Clojure for GraalVM native compilation you can't leverage such dynamic behavior as well. Though you can embed the SCI Clojure interpreter inside your application and do dynamic code evaluation with it at runtime.
    - deleted 5y ago
      [deleted]
    - jgrodziski 5y ago
      GraalVM issue an error if it can't resolve the proper method call or if another Thread is involved in the computation so I find it pretty safe to use. Of course, GraalVM doesn't exempt you from doing some tests.

chitowneats 5y ago

Have you tried Babashka? https://github.com/babashka/babashka https://github.com/babashka/babashka

dimitar 5y ago
babashka also has a performance non-goal
- chitowneats 5y ago
  Ah. You are correct.
- didibus 5y ago
  That's true, but in my poor man tests, Babashka was in the ballpark of Python for me.
  - chitowneats 5y ago
    I was surprised when I realized I had overlooked the performance non-goal considering how fast it has been in my limited experience.
    - didibus 5y ago
      Ya, it even supports multi-core programming, so I bet for some use case it can outperform Python by parallelizing.
- Borkdude 5y ago
  The performance non-goal is there to lower expectations compared to compiled Clojure. Babashka's author (me) does try to make it as fast as possible within the constraints it's implemented in.

vnorilo 5y ago

I have long had the ambition to be the N:th guy to try Clojure on C++. For similar reasons. I have a pretty good idea how to do it, already got HAMT and some macroexpanded code to work. But then you remember how even ClojureCLR struggled at times, and that you'd be lightyears behind that, not to speak of CLJ/CLJS, and alone. Maybe one day? :)

jb1991 5y ago
It would seem to me the most significant challenge would be having functional data structures in C++, that would be a non-trivial effort I would think. Even if you were to use other libraries that experiment with this. Without those, the language just doesn’t make any sense.
- toymachine1974 5y ago
  My Park language does this https://github.com/toymachine/park-lang https://github.com/toymachine/park-lang Currently the syntax is mostly JS but the semantics is all clojure. I recreated the map and vector datatypes in c++ by adapting clojures Java code. it needs a GC though which is another big part of the implemention of Park
- vnorilo 5y ago
  Yeah, that was the problem with most attempts I saw, unordered_map won't work. I did write a persistent hash map and vector [1], but ended up mostly using them in C++ projects. I wouldn't advise anyone else to use them at this level of testing/maturity though. Writing them was super interesting. 1: https://hg.sr.ht/~vnorilo/pcoll/browse https://hg.sr.ht/~vnorilo/pcoll/browse (not production quality)
- bertmuthalaly 5y ago
  Immer [0][1] seems like a really serious attempt at bringing high-quality persistent data structures to C++. [0]: https://www.youtube.com/watch?v=sPhpelUfu8Q https://www.youtube.com/watch?v=sPhpelUfu8Q [1]: https://sinusoid.es/immer/ https://sinusoid.es/immer/
geokon 5y ago

zerr 5y ago

What would be the advantage over other Lisps if you won't be able to leverage JVM ecosystem?

leephillips 5y ago
Some people like Clojure's data structures, which add a couple of things to lists. Also, there's ClojureScript, which is huge. And, ideas such as software transactional memory.
didibus 5y ago
Depending who you ask, Clojure provides some benefits over other Lisps unrelated to its ability to leverage the JVM. Some of those are: - Default efficient immutable collections - Good support for safe concurrency and multi-core programming - Extended homoiconicity to include maps, vectors and sets as well as lists - Convenient short lambda syntax - Extensible data representation - Distinguishes between nil and empty list - Distinguishes between keywords and symbols - Real boolean types - Function arity overloading - Well thought out abstractions and set of standard functions: collections are abstractions, lazy sequences, transducers, relational set manipulation, etc. - Only two kinds of equality: by value or by reference, with the former being the default - Is a lisp-1 like Scheme, but also with more batteries included like Common Lisp - Slightly different visual appearance in its syntax, due to support for literal maps and vectors as well as lists, which can make things easier to visually parse.

skybrian 5y ago

It’s difficult to write a fast interpreter in Go. It doesn’t optimize interpreter inner loops particularly well, you can’t do the data structure tricks you can do in C, and there is no JIT compiler available like in Java. The best way forward might be an option to compile some Joker code by generating Go code for it. (This is based on writing an interpreter a few years ago. Perhaps Go’s compiler has improved?)

AtlasBarfed 5y ago
What C tricks are used? I'd be fascinated to know some details.
- spijdar 5y ago
  Caveat: I've barely used Go personally. Just speculating on what GP specifically meant, but Go lacks the ability to pack structs, and probably lacks the equivalent of tricks like computed gotos [0] to increase bytecode interpreter speeds. In general, Go seems to (intentionally) lack a lot of low-level control of code generation, preferring a "there's one way to do things, and it either works, or it's a bug we'll fix" approach. Which is probably for the best in "most" software, but interpreters typically use weird hacks to squeeze more performance out of the rock. Wren is a good example [1] of some of these optimizations, and has splendid comments/documentation. [0] https://eli.thegreenplace.net/2012/07/12/computed-goto-for-efficient-dispatch-tables https://eli.thegreenplace.net/2012/07/12/computed-goto-for-e... [1] https://github.com/wren-lang/wren/blob/main/src/vm/wren_vm.c https://github.com/wren-lang/wren/blob/main/src/vm/wren_vm.c
- skybrian 5y ago
  For one thing you have C unions. You can store different types in the same place and use information stored elsewhere to know what the bits really mean. It’s horribly unsafe compared to a Go interface or the tagged unions in other languages, but uses less space. Also, there is the NaN boxing trick [1], which allows you to store a value that is essentially a tagged union that is either a double, a pointer, or an integer, in the same space as a double. To get these in a safe language, they essentially have to be built in already. Go’s interface type is very high overhead compared to the union types used in other languages because it uses an entire pointer as a tag. Though, it only matters in special cases like interpreters. Most performance-intensive code isn’t nearly so dynamic. [1] https://sean.cm/a/nan-boxing https://sean.cm/a/nan-boxing

didibus 5y ago

You can just use GraalVM for that. Also, Joker does not let you do what you want, it is not Clojure hosted on the Go runtime, it is a Clojure interpreter runtime implemented in Go, so with Joker you always need the joker binary. If you want to produce native shippable statically linked binaries with Clojure look at GraalVM native compilation: https://github.com/lread/clj-graal-docs https://github.com/lread/clj-graal-docs