4 ms·

Clojure: Creating a reducible repeat

57 points by r4um 12y ago

samatman 12y ago
This reminds me of a daydream I've been having, that goes something like this: Rewrite the core Clojure data structures in Rust. Link in LuaJIT. Write a bunch of finalizers and a reader in Lua, skin the Clojure library in the proper syntax, and Bob's your uncle. I'm far too busy to take ownership on such a project, but if anyone starts it or finds it, please contact me.
- Skinney 12y ago
  The core datastructures are immutable and uses structural sharing. This would require reference counting (slower) or a proper GC to be efficient (Rust doesn't have one). Doing this in Rust would probably give you worse results than the JVM. Also, why LuaJIT? LuaJIT (or Lua) is not threadsafe, so you wouldn't be able to share state between threads safely, which would conflict with the way Clojure handles concurrency. I'm not trying to be negative. But unless this is a project for fun or excercise, the JVM is a better host.
  - Jweb_Guru 12y ago
    Rust doesn't require reference counting to make sharing immutable datastructures safe. The interface would require lifetimes for safety, though, and might be generally unpleasant to use.
    - Skinney 12y ago
      Oh? Lifetimes are that flexible? It's been a long time since I looked into Rust, so I had no idea. I would thought it was difficult to decide when to free an object, if it is linked to by two separate data structures and one of them goes out of scope.
      - Matthias247 12y ago
        They are not. Sending something to another thread requires the thing to send to implement some kinds (I think Send + Sync). To have those guarantees the data would need to stored in an atomically reference counted structure (or something else with the same guarantees), which is just like you said. And you would need to watch out for reference cycles. I guess they are quite easy to create with Clojure.
        Jweb_Guru 12y ago
        > To have those guarantees the data would need to stored in an atomically reference counted structure (or something else with the same guarantees), which is just like you said. That is not true. `Send` is having its `'static` bound removed, so sharing data will no longer require reference counting for threads with bounded lifetime: https://github.com/rust-lang/rfcs/pull/458 https://github.com/rust-lang/rfcs/pull/458 (the RFC hasn't been officially accepted but it's been referenced as a foregone conclusion several times by Rust core team members). There is already an experimental library exploiting this for `Sync` data, which includes immutable substructures: https://github.com/nikomatsakis/rayon https://github.com/nikomatsakis/rayon The only reason people aren't more aware of this is because the functionality isn't exposed in the standard library yet. The language is perfectly capable.
      - Jweb_Guru 12y ago
        Rust solves this dilemma with lifetimes. As long as a reference exists to data, Rust statically ensures that you cannot perform any operation that frees or moves it in memory. In the case of multiple threads, one can also assign each thread a lifetime, and perform the same analysis to maintain existence guarantees. Things get a bit trickier if you want to actually mutate the shared reference, but in the case of truly immutable data that's really all you need.
- nickik 12y ago
  Not exactly what you want but Im writing a LuaJit inspired VM (hope to use some of its code as well) for clojure and Im using rust to do it. That said, I have no plans of writting any persistent datastrucutres in rust, I would like to keep that code in clojure. https://github.com/clojit/clojit-vm https://github.com/clojit/clojit-vm Rust updates fucked me over and I have not time to fix it atm, but work will go on.
- yogthos 12y ago
  I think Pixie is a very promising attempt at something like that https://github.com/pixie-lang/pixie https://github.com/pixie-lang/pixie

57 points by r4um 12y ago

This reminds me of a daydream I've been having, that goes something like this: Rewrite the core Clojure data structures in Rust. Link in LuaJIT. Write a bunch of finalizers and a reader in Lua, skin the Clojure library in the proper syntax, and Bob's your uncle. I'm far too busy to take ownership on such a project, but if anyone starts it or finds it, please contact me.

Skinney 12y ago
The core datastructures are immutable and uses structural sharing. This would require reference counting (slower) or a proper GC to be efficient (Rust doesn't have one). Doing this in Rust would probably give you worse results than the JVM. Also, why LuaJIT? LuaJIT (or Lua) is not threadsafe, so you wouldn't be able to share state between threads safely, which would conflict with the way Clojure handles concurrency. I'm not trying to be negative. But unless this is a project for fun or excercise, the JVM is a better host.
- Jweb_Guru 12y ago
  Rust doesn't require reference counting to make sharing immutable datastructures safe. The interface would require lifetimes for safety, though, and might be generally unpleasant to use.
  - Skinney 12y ago
    Oh? Lifetimes are that flexible? It's been a long time since I looked into Rust, so I had no idea. I would thought it was difficult to decide when to free an object, if it is linked to by two separate data structures and one of them goes out of scope.
    - Matthias247 12y ago
      They are not. Sending something to another thread requires the thing to send to implement some kinds (I think Send + Sync). To have those guarantees the data would need to stored in an atomically reference counted structure (or something else with the same guarantees), which is just like you said. And you would need to watch out for reference cycles. I guess they are quite easy to create with Clojure.
      - Jweb_Guru 12y ago
        > To have those guarantees the data would need to stored in an atomically reference counted structure (or something else with the same guarantees), which is just like you said. That is not true. `Send` is having its `'static` bound removed, so sharing data will no longer require reference counting for threads with bounded lifetime: https://github.com/rust-lang/rfcs/pull/458 https://github.com/rust-lang/rfcs/pull/458 (the RFC hasn't been officially accepted but it's been referenced as a foregone conclusion several times by Rust core team members). There is already an experimental library exploiting this for `Sync` data, which includes immutable substructures: https://github.com/nikomatsakis/rayon https://github.com/nikomatsakis/rayon The only reason people aren't more aware of this is because the functionality isn't exposed in the standard library yet. The language is perfectly capable.
    - Jweb_Guru 12y ago
      Rust solves this dilemma with lifetimes. As long as a reference exists to data, Rust statically ensures that you cannot perform any operation that frees or moves it in memory. In the case of multiple threads, one can also assign each thread a lifetime, and perform the same analysis to maintain existence guarantees. Things get a bit trickier if you want to actually mutate the shared reference, but in the case of truly immutable data that's really all you need.
nickik 12y ago
Not exactly what you want but Im writing a LuaJit inspired VM (hope to use some of its code as well) for clojure and Im using rust to do it. That said, I have no plans of writting any persistent datastrucutres in rust, I would like to keep that code in clojure. https://github.com/clojit/clojit-vm https://github.com/clojit/clojit-vm Rust updates fucked me over and I have not time to fix it atm, but work will go on.
yogthos 12y ago
I think Pixie is a very promising attempt at something like that https://github.com/pixie-lang/pixie https://github.com/pixie-lang/pixie