3 ms·

What a great writeup, thank you! I think I followed all that . Two things leap out at me: - I can see arenas for a compiler, but I'm not so sure for an interpr

What a great writeup, thank you! I think I followed all that . Two things leap out at me: - I can see arenas for a compiler, but I'm not so sure for an interpreter. While most of the memory allocated in Dark will be done within a single HTTP call (and so an arena might actually be appropriate), there are contexts where a GC cycle or two to cleanup might be appropriate. I'm guessing that arena's are an all-or-nothing approach? - I'm using `im`, which I think also gets in the way of my nice pattern matching, right? Any solutions to that?

Jweb_Guru 6y ago
- Well, it depends on how you intend your interpreter to be used, but if it's for long running programs, a garbage collector of some kind is essential. Rc<T> is a primitive garbage collector that happens to be provided by Rust, but it's not very efficient, so if you're concerned with performance you'd probably want to build your own (or try your luck with a community GC--I haven't tried them in a while, so maybe they've improved since I last looked, but there didn't seem to be a lot of promising options for automating this). - I don't think so, unfortunately. Your best bet would probably be a macro, which I'm sure you've already considered. You might want to talk to the author though, as I'm sure she has thought about this a lot more than I have.
Jweb_Guru 6y ago
I also realized I should specifically address your "all or nothing" question, though: arenas absolutely aren't all or nothing :) In particular, a very common (well, by writing-a-compiler-in-Rust standards) pattern is to have two representations for your elements: one that uses Rc, that you use for longer-lived stuff, and another based on references, that you use for arena-scoped stuff. An extra term is added to the younger version that marks the use of a wrapper around an Rc type (recall that you can downcast Rc to reference, so you do not necessarily have to lose Copy to be able to do this, depending on your implementation). You can then implement a variant of generational collection by (prior to dropping your arena, i.e. the young generation), scanning any live objects using the young term type, and converting whatever you find to Rc nodes. The Rc nodes then persist as the older generation. This kind of hybrid strategy is much more effective than using Rc for everything, and while it's still not as efficient as a proper garbage collector it still has roughly the kinds of properties you want from one. My experience (like that of many others) is that almost any workload you want to use this with will satisfy the generational hypothesis, especially in Rust where you only use GC for things that really need it, so hopefully you can see why it's sometimes worth going to the extra trouble to maintain several versions of the same type. (Obviously, in OCaml, you can piggyback off the existing garbage collector and this is a nonissue).

Jweb_Guru 6y ago

- Well, it depends on how you intend your interpreter to be used, but if it's for long running programs, a garbage collector of some kind is essential. Rc<T> is a primitive garbage collector that happens to be provided by Rust, but it's not very efficient, so if you're concerned with performance you'd probably want to build your own (or try your luck with a community GC--I haven't tried them in a while, so maybe they've improved since I last looked, but there didn't seem to be a lot of promising options for automating this). - I don't think so, unfortunately. Your best bet would probably be a macro, which I'm sure you've already considered. You might want to talk to the author though, as I'm sure she has thought about this a lot more than I have.

Jweb_Guru 6y ago

I also realized I should specifically address your "all or nothing" question, though: arenas absolutely aren't all or nothing :) In particular, a very common (well, by writing-a-compiler-in-Rust standards) pattern is to have two representations for your elements: one that uses Rc, that you use for longer-lived stuff, and another based on references, that you use for arena-scoped stuff. An extra term is added to the younger version that marks the use of a wrapper around an Rc type (recall that you can downcast Rc to reference, so you do not necessarily have to lose Copy to be able to do this, depending on your implementation). You can then implement a variant of generational collection by (prior to dropping your arena, i.e. the young generation), scanning any live objects using the young term type, and converting whatever you find to Rc nodes. The Rc nodes then persist as the older generation. This kind of hybrid strategy is much more effective than using Rc for everything, and while it's still not as efficient as a proper garbage collector it still has roughly the kinds of properties you want from one. My experience (like that of many others) is that almost any workload you want to use this with will satisfy the generational hypothesis, especially in Rust where you only use GC for things that really need it, so hopefully you can see why it's sometimes worth going to the extra trouble to maintain several versions of the same type. (Obviously, in OCaml, you can piggyback off the existing garbage collector and this is a nonissue).