4 ms·

I have this crazy idea that you could build Clojure on Rust the same way Clojure sits on top of JVM right now and it could provide you with some interesting ben

I have this crazy idea that you could build Clojure on Rust the same way Clojure sits on top of JVM right now and it could provide you with some interesting benefits. First of all Rust doesn't implement a GC but it has very precise memory management semantics which means integrating in to a GC should be possible and safe. I believe Clojure immutable data structures would allow you to (theoretically) build a fast/simplified GC because immutable data has some very nice properties : * no cycles except trough reference types which are explicitly marked - this means you never need to scan old generation to collect the new generation * data is immutable which means copying can be done in parallel * mutable references are explicitly marked and isolated which means you could have a very minimal pause to update references And Rust could safely encode all those requirements in it's type system to make the interop safe. No need to sprinkle write barriers all over the place, localized and isolated mutable references (you can modify the update semantics and track them easily), minimal pause times, parallel collection, etc. I think there are other interesting things you could do, for example Clojure is pretty static for a dynamically typed language and protocols map nicely to traits I wonder if you could somehow remove the dynamic typing and just have compile time type inference figure out the types (requiring explicit types when the system can't infer the types) - this would allow you to send Clojure down to LLVM along with Rust. Or maybe you could have an interpreter for the truly dynamic stuff - but then you'd need to figure out how to export metadata. Also I'm sure that the only sane/safe way to do it would be to restrict the kinds of objects and that would require serious thinking too.

stcredzero 11y ago
I have this crazy idea that you could build Clojure on Rust the same way Clojure sits on top of JVM right now and it could provide you with some interesting benefits. If you read some of the design docs and articles, you'll find that Clojure is designed to be a "hosted" language from the ground up. It would be a challenge to implement some of Clojure's dynamic features. You could very well do a Clojure-like language which has a different approach to being functional. Clojure is very Lisp-y in that it's actually a dynamic runtime fronted by a language that makes it look functional.
tsmarsh 11y ago
I had similar thoughts but wanted to take it a step further and just have a GC less Clojure built on Rust. You would have to do things like nest functions inside of (ns), you couldn't have (def) but this is all before I've spent any real hammock time on it. If you get started it would be fun to follow / contribute to.
pron 11y ago
> this means you never need to scan old generation to collect the new generation Even in HotSpot you don't need to scan the old generation if references in it don't change. You only scan (roughly) those old objects who hold references that have been mutated since the last collection. > data is immutable which means copying can be done in parallel This is also possible when mutation is allowed (see Red Hat's Shenandoah[1][2], A new GC for HotSpot), and immutability doesn't make the challenge significantly easier (there is added overhead for mutation only). > mutable references are explicitly marked and isolated which means you could have a very minimal pause to update references... No need to sprinkle write barriers all over the place Again, mutating reference fields has a special barrier in HotSpot's GCs, but they're not "sprinkled all over the place" -- they're as good as being specially marked. To summarize, current (and future) HotSpot GCs punish you for mutation, but not for the possibility of mutation; you get all the benefits of immutability automatically when you simply don't mutate references. Immutability -- as you note -- might indeed simplify the GC (though not in the case of Clojure, which does allow quite a lot of mutability -- not everywhere, but enough that you have to account for it not as an outlier), but it doesn't make it faster. ---- > for example Clojure is pretty static for a dynamically typed language and protocols map nicely to traits I wonder if you could somehow remove the dynamic typing and just have compile time type inference figure out the types (requiring explicit types when the system can't infer the types) - this would allow you to send Clojure down to LLVM along with Rust. A good JIT (like HotSpot) does this a lot better. It is true that Clojure is nowhere near as crazily-dispatched as, say Ruby, but it does rely a lot on Java-like interface-based dynamic dispatch. If your type inference is really, really good, you might be able to identify a great deal of the monomorphic call-sites, but a good JIT finds all of them and makes those calls virtually free, something an AOT compiler can never hope to achieve. Besides, HotSpot's optimizing JIT is a state-of-the-art compiler. You won't do any better with LLVM. There are approaches that are good for some languages, and others that are good for others. Language's that rely on dynamic dispatch are better off using a JIT than an AOT compiler (actually, pretty much every language would get better code generation with a JIT -- except C which hardly does any dynamic dispatch -- but a JIT has some runtime overheads in warmup, RAM and energy, and languages that don't do too much dynamic dispatch can get excellent machine code generated by an AOT compiler, so that a JIT isn't worth it). ---- In short, while you could (maybe) compile Clojure down to Rust, you'll see no benefit, and probably quite a few disadvantages to that approach. If you want Clojure-Rust interoperability, you can get it today with RustJNI[3]. [1]: http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189 [2]: http://www.jclarity.com/2014/02/19/shenandoah-a-new-low-pause-garbage-collection-algorithm-for-the-java-hotspot-jvm/ http://www.jclarity.com/2014/02/19/shenandoah-a-new-low-paus... [3]: https://github.com/Monnoroch/RustJni https://github.com/Monnoroch/RustJni
kyllo 11y ago
It's an interesting idea, but the fact that the Rust compiler is an AOT compiler with an extremely minimal runtime makes it seem like a poor fit for Clojure which is very much an interpreter/REPL-driven language.

stcredzero 11y ago

I have this crazy idea that you could build Clojure on Rust the same way Clojure sits on top of JVM right now and it could provide you with some interesting benefits. If you read some of the design docs and articles, you'll find that Clojure is designed to be a "hosted" language from the ground up. It would be a challenge to implement some of Clojure's dynamic features. You could very well do a Clojure-like language which has a different approach to being functional. Clojure is very Lisp-y in that it's actually a dynamic runtime fronted by a language that makes it look functional.

tsmarsh 11y ago

I had similar thoughts but wanted to take it a step further and just have a GC less Clojure built on Rust. You would have to do things like nest functions inside of (ns), you couldn't have (def) but this is all before I've spent any real hammock time on it. If you get started it would be fun to follow / contribute to.

pron 11y ago

> this means you never need to scan old generation to collect the new generation Even in HotSpot you don't need to scan the old generation if references in it don't change. You only scan (roughly) those old objects who hold references that have been mutated since the last collection. > data is immutable which means copying can be done in parallel This is also possible when mutation is allowed (see Red Hat's Shenandoah[1][2], A new GC for HotSpot), and immutability doesn't make the challenge significantly easier (there is added overhead for mutation only). > mutable references are explicitly marked and isolated which means you could have a very minimal pause to update references... No need to sprinkle write barriers all over the place Again, mutating reference fields has a special barrier in HotSpot's GCs, but they're not "sprinkled all over the place" -- they're as good as being specially marked. To summarize, current (and future) HotSpot GCs punish you for mutation, but not for the possibility of mutation; you get all the benefits of immutability automatically when you simply don't mutate references. Immutability -- as you note -- might indeed simplify the GC (though not in the case of Clojure, which does allow quite a lot of mutability -- not everywhere, but enough that you have to account for it not as an outlier), but it doesn't make it faster. ---- > for example Clojure is pretty static for a dynamically typed language and protocols map nicely to traits I wonder if you could somehow remove the dynamic typing and just have compile time type inference figure out the types (requiring explicit types when the system can't infer the types) - this would allow you to send Clojure down to LLVM along with Rust. A good JIT (like HotSpot) does this a lot better. It is true that Clojure is nowhere near as crazily-dispatched as, say Ruby, but it does rely a lot on Java-like interface-based dynamic dispatch. If your type inference is really, really good, you might be able to identify a great deal of the monomorphic call-sites, but a good JIT finds all of them and makes those calls virtually free, something an AOT compiler can never hope to achieve. Besides, HotSpot's optimizing JIT is a state-of-the-art compiler. You won't do any better with LLVM. There are approaches that are good for some languages, and others that are good for others. Language's that rely on dynamic dispatch are better off using a JIT than an AOT compiler (actually, pretty much every language would get better code generation with a JIT -- except C which hardly does any dynamic dispatch -- but a JIT has some runtime overheads in warmup, RAM and energy, and languages that don't do too much dynamic dispatch can get excellent machine code generated by an AOT compiler, so that a JIT isn't worth it). ---- In short, while you could (maybe) compile Clojure down to Rust, you'll see no benefit, and probably quite a few disadvantages to that approach. If you want Clojure-Rust interoperability, you can get it today with RustJNI[3]. [1]: http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189 [2]: http://www.jclarity.com/2014/02/19/shenandoah-a-new-low-pause-garbage-collection-algorithm-for-the-java-hotspot-jvm/ http://www.jclarity.com/2014/02/19/shenandoah-a-new-low-paus... [3]: https://github.com/Monnoroch/RustJni https://github.com/Monnoroch/RustJni

kyllo 11y ago

It's an interesting idea, but the fact that the Rust compiler is an AOT compiler with an extremely minimal runtime makes it seem like a poor fit for Clojure which is very much an interpreter/REPL-driven language.