4 ms·

Rust noob here so please be gentle - can someone explain why in `Vm<'a>` the lifetime is needed? I see that in tests, `Vm<'static>` gets created, presumably bec

Rust noob here so please be gentle - can someone explain why in `Vm<'a>` the lifetime is needed? I see that in tests, `Vm<'static>` gets created, presumably because there's nowhere to grab a better lifetime from? But if that's the case, would it be possible to express this in a different way? ISTM that Vm owns everything it needs, so is there a way to avoid bubbling the lifetime up? Similar question applies to ClassManager - it already owns all the classes (they are in an arena), so it looks like a lifetime is needed because there's no way to say "things live as long as ClassManager lives" w/o introducing the lifetime at `ClassManager<'a>` level...

andreabergia 3y ago
The problem arises by the fact that `Class` needs to refer to other classes: pub struct Class<'a> { pub superclass: Option<ClassRef<'a>>, } pub type ClassRef<'a> = &'a Class<'a>; Given that classes are managed by the arena, I know the reference will not be dangling thanks to the 'a lifetime. Initially I had implemented this with raw pointers, without lifetimes, but then I switched to the reference because it felt more "idiomatic" and I had to put the lifetimes just about everywhere to make the compiler happy. If there are better ways to do this, I would be really happy to learn, though!
- bialpio 3y ago
  Yeah, I think I understand why you had to write it this way, but the fact that this lifetime needs to be bubbled up so far up is really non-intuitive to me. > If there are better ways to do this, I would be really happy to learn, though! Me, too. :) I find myself limited by my way of thinking here, coming from C++ ("I _know_ it lives long enough, why don't you let me express this?").
  - andreabergia 3y ago
    Agreed, it _is_ annoying. It bubbles up everywhere and it feels like "something to silence the compiler" more than "something to express the safety of the code", as other people have pointed out.
- 0b01 3y ago
  Same way to write self referential structs - use index types into the whatever arena you are using. Indexes are usually 32 bit so they are a bit faster than pointers. If you are building one off trees such as for parsing and ast transforms, bumpalo is your friend. In your case, you can look into generational arenas and slabs which are useful for graphs.
- pie_flavor 3y ago
  The better way to do this is to use an indexing arena such as `slotmap` or `generational-arena`, so `ClassRef` is not actually a memory pointer to anything but can still be looked up in the context of the class collection. Or, if you never need to remove any values, you can use Vec as your arena and usize as your key.
  - fleventynine 3y ago
    IMHO, using an vector index as a pointer should be a last resort (as an alternative to unsafe). Although it's memory safe, you still have many of the traditional issues like stale pointers / use-after-free, plus you also have the cost of a bounds-check on every access and making sure you use your stored indexes with the correct container instance. If you can solve your problem by using safe code with lifetimes, that should be preferred.
    - pie_flavor 3y ago
      What is presented here is a self-referential type, and Rust does not allow you to declare self-referential types, and an indexed arena is a solution to that. You are correct that you still have all the issues you described - that doesn't mean it should be a 'last resort' to stop using an incorrect system of lifetimes, because there is no 'safe code with lifetimes' that encodes a self-referential type. The type safety issue is solved by using slotmap or typed-generational-arena, which use unique key types.
- kevincox 3y ago
  For things like classes that aren't going to be created and destroyed quickly it likely is best to just use a Rc. Then you know it will be available and don't need to bother with lifetimes. Of course there is a good chance that you will eventually want a VM-scoped lifetime for something else so maybe it is best to just start now.
skitter 3y ago
You're right there – It looks like the author is trying to express "this lifetime lasts as long as the VM exists", which isn't possible to express in Rust. For VM::new, the callsite just picks a lifetime that lasts long enough – even if that outlasts the VM, making the lifetime useless. This is a bug and unsound.
- bialpio 3y ago
  My understanding is that if Rust let you compile it, then there is no bug with a lifetime (the code doesn't use `unsafe`). There may be something magical happening under the hood that I do not fully understand, but in Rust I trust. ;)
  - skitter 3y ago
    Your understanding of Rust is correct, but the code uses unsafe: https://github.com/andreabergia/rjvm/blob/93e7e48db085e780b0787aaf37bb3c77d6cfa7b8/vm/src/gc.rs#L39 https://github.com/andreabergia/rjvm/blob/93e7e48db085e780b0... As a result, when the VM gets dropped, the memory gets deallocated even if there are still objects that reference it. Somewhat related, there also is the issue that when you create multiple VMs, you can use classes/objects/… created by one VM with another.
    - bialpio 3y ago
      IIUC this should be fine - MemoryChunk doesn't seem to deallocate the memory (so we leak), unless there's a dealloc somewhere (sorry, not logged in to GH so cannot search). Which is not the worst outcome here, because as you said, we don't want to deallocate this memory since there may be places that still refer to it... As for where the static lifetime gets into play, VM creation doesn't use unsafe & compiler is happy about it, even though ISTM that there could be a tighter bound on the lifetime? https://github.com/andreabergia/rjvm/blob/93e7e48db085e780b0787aaf37bb3c77d6cfa7b8/vm/tests/integration/real_code_tests.rs#L10 https://github.com/andreabergia/rjvm/blob/93e7e48db085e780b0...
      - 3y ago

andreabergia 3y ago

The problem arises by the fact that `Class` needs to refer to other classes: pub struct Class<'a> { pub superclass: Option<ClassRef<'a>>, } pub type ClassRef<'a> = &'a Class<'a>; Given that classes are managed by the arena, I know the reference will not be dangling thanks to the 'a lifetime. Initially I had implemented this with raw pointers, without lifetimes, but then I switched to the reference because it felt more "idiomatic" and I had to put the lifetimes just about everywhere to make the compiler happy. If there are better ways to do this, I would be really happy to learn, though!

bialpio 3y ago
Yeah, I think I understand why you had to write it this way, but the fact that this lifetime needs to be bubbled up so far up is really non-intuitive to me. > If there are better ways to do this, I would be really happy to learn, though! Me, too. :) I find myself limited by my way of thinking here, coming from C++ ("I _know_ it lives long enough, why don't you let me express this?").
- andreabergia 3y ago
  Agreed, it _is_ annoying. It bubbles up everywhere and it feels like "something to silence the compiler" more than "something to express the safety of the code", as other people have pointed out.
0b01 3y ago
Same way to write self referential structs - use index types into the whatever arena you are using. Indexes are usually 32 bit so they are a bit faster than pointers. If you are building one off trees such as for parsing and ast transforms, bumpalo is your friend. In your case, you can look into generational arenas and slabs which are useful for graphs.
pie_flavor 3y ago
The better way to do this is to use an indexing arena such as `slotmap` or `generational-arena`, so `ClassRef` is not actually a memory pointer to anything but can still be looked up in the context of the class collection. Or, if you never need to remove any values, you can use Vec as your arena and usize as your key.
- fleventynine 3y ago
  IMHO, using an vector index as a pointer should be a last resort (as an alternative to unsafe). Although it's memory safe, you still have many of the traditional issues like stale pointers / use-after-free, plus you also have the cost of a bounds-check on every access and making sure you use your stored indexes with the correct container instance. If you can solve your problem by using safe code with lifetimes, that should be preferred.
  - pie_flavor 3y ago
    What is presented here is a self-referential type, and Rust does not allow you to declare self-referential types, and an indexed arena is a solution to that. You are correct that you still have all the issues you described - that doesn't mean it should be a 'last resort' to stop using an incorrect system of lifetimes, because there is no 'safe code with lifetimes' that encodes a self-referential type. The type safety issue is solved by using slotmap or typed-generational-arena, which use unique key types.
kevincox 3y ago
For things like classes that aren't going to be created and destroyed quickly it likely is best to just use a Rc. Then you know it will be available and don't need to bother with lifetimes. Of course there is a good chance that you will eventually want a VM-scoped lifetime for something else so maybe it is best to just start now.

skitter 3y ago

You're right there – It looks like the author is trying to express "this lifetime lasts as long as the VM exists", which isn't possible to express in Rust. For VM::new, the callsite just picks a lifetime that lasts long enough – even if that outlasts the VM, making the lifetime useless. This is a bug and unsound.

bialpio 3y ago
My understanding is that if Rust let you compile it, then there is no bug with a lifetime (the code doesn't use `unsafe`). There may be something magical happening under the hood that I do not fully understand, but in Rust I trust. ;)
- skitter 3y ago
  Your understanding of Rust is correct, but the code uses unsafe: https://github.com/andreabergia/rjvm/blob/93e7e48db085e780b0787aaf37bb3c77d6cfa7b8/vm/src/gc.rs#L39 https://github.com/andreabergia/rjvm/blob/93e7e48db085e780b0... As a result, when the VM gets dropped, the memory gets deallocated even if there are still objects that reference it. Somewhat related, there also is the issue that when you create multiple VMs, you can use classes/objects/… created by one VM with another.
  - bialpio 3y ago
    IIUC this should be fine - MemoryChunk doesn't seem to deallocate the memory (so we leak), unless there's a dealloc somewhere (sorry, not logged in to GH so cannot search). Which is not the worst outcome here, because as you said, we don't want to deallocate this memory since there may be places that still refer to it... As for where the static lifetime gets into play, VM creation doesn't use unsafe & compiler is happy about it, even though ISTM that there could be a tighter bound on the lifetime? https://github.com/andreabergia/rjvm/blob/93e7e48db085e780b0787aaf37bb3c77d6cfa7b8/vm/tests/integration/real_code_tests.rs#L10 https://github.com/andreabergia/rjvm/blob/93e7e48db085e780b0...
    - 3y ago