6 ms·

> I can't reply directly to your other comment Next time just wait 5-10 minutes. > Most data structures in Rust require unsafe for performance or memory acces

> I can't reply directly to your other comment Next time just wait 5-10 minutes. > Most data structures in Rust require unsafe for performance or memory access patterns. And when I want to compose 2 data structures into my own higher-level one, for performance and memory access patterns I need these two lower-level structures to expose unsafe stuff at their API boundaries. The data structures I saw don’t do that, they’re designed to be consumed from safe Rust instead. > Rust doesn't just throw out all the rules I think in modern C++, with these iterators and smart pointers, you’re less likely to screw up dereferencing a wrong pointer.

pcwalton 9y ago
> And when I want to compose 2 data structures into my own higher-level one, for performance and memory access patterns I need these two lower-level structures to expose unsafe stuff at their API boundaries. The data structures I saw don’t do that, they’re designed to be consumed from safe Rust instead. Can you give a specific example of something you want to do that you can't? > I think in modern C++, with these iterators and smart pointers, you’re less likely to screw up dereferencing a wrong pointer. I don't think this is empirically true relative to C, but even if it is, use after free is still far too common in C++ code.
- Const-me 9y ago
  > Can you give a specific example of something you want to do that you can't? Compose a hash map + linked list into an LRU cache. Rust how has that in standard library, but they had to implement their own linked list for that. In C++ it’s just a few lines of code, because standard maps+lists compose just fine. Or (more generic example and thus harder to put in the standard library), add an index to existing collection. I have a large collection of some values. I want to build an index allowing to lookup values by some key. Values are not small, can’t afford duplicating them. If you’ll tell “just move values into a hashmap”, my response is “and I also want another, different index of the same set of values by different key”. Again, very easy in C++, encapsulate both the original collection, and a hashmap from key to value pointer. > use after free is still far too common in C++ code. In my experience, use after free = instant crash in debug build. Quite easy to detect and fix.
  - burntsushi 9y ago
    Use an Rc? If you can't afford the reference count, then use unsafe/raw pointers, which it sounds like you'd do in C++ anyway.
    - Const-me 9y ago
      > then use unsafe/raw pointers Even if I manage to extract an unsafe pointer from that Rust collection, I don’t know for how long will it work. For C++ collections, iterator invalidation rules tell me that.
      - burntsushi 9y ago
        You'd have to box your values. And if you didn't want to do that, then I'd just used the indexing method mentioned elsewhere in this thread. I've used such things in performance critical code.
        Const-me 9y ago
        > the indexing method mentioned elsewhere in this thread https://news.ycombinator.com/item?id=15180649 https://news.ycombinator.com/item?id=15180649
      - cesarb 9y ago
        > Even if I manage to extract an unsafe pointer from that Rust collection, It's easy, just get the & or &mut to the value (as if you were acessing it), and cast it to respectively * const or * mut. > I don’t know for how long will it work. For C++ collections, iterator invalidation rules tell me that. It's the same in Rust. Whenever the iterator would be invalidated in C++, the pointer you stashed above might point to the wrong place. This is not usually documented in Rust, because its borrow rules prevent you from stashing a reference while the collection mutates, but once you start playing with raw pointers, the borrow checker gets out of the way (references have a lifetime, pointers don't). You just have to be careful when casting the pointer back to a mutable ref ("unsafe { &mut *ptr }" is the trick, see the documentation for std::mem::transmute): mutable references are like C99's "restrict", so you should make sure to only ever have one live for each pointer at every moment, otherwise you're in undefined behavior land. ---- Anyway, going back to the parent comment, you said "Values are not small, can’t afford duplicating them". Might I suggest keeping the values in a Box<T> then, and making both collections point to the box? That way, you don't have to worry about a mutation in one of the collections invalidating the pointer, since the contents of a Box won't move in memory. And in fact, the usual Rust style for keeping a value in more than one collection would be to use a Rc<T>, which is basically a Box with a reference counter. That way, you don't need to play with raw pointers, and have no risk of a misstep. You pay the cost of incrementing/decrementing the reference counter only when adding/removing from the collection, and the reference counter is small.
burntsushi 9y ago
> And when I want to compose 2 data structures into my own higher-level one Could you give a specific example please? I compose data structures in Rust all the time.
- Const-me 9y ago
  https://news.ycombinator.com/item?id=15180500 https://news.ycombinator.com/item?id=15180500
bluejekyll 9y ago
More like 20 it seems ;) > I need these two lower-level structures to expose unsafe stuff at the API boundary Two thoughts: 1) I think you can always use unsafe to get access to a raw pointer (I honestly don't use unsafe often) 2) you need someway to express ownership between both data structures, this can be annoying, no doubt. > C++, with these iterators and smart pointers Does that make it safer than unsafe Rust? Maybe, but there's a lot less unsafe Rust even in these graphs...
- dikaiosune 9y ago
  If you click directly on the time of a comment, it's a link to a direct reply page.
- Const-me 9y ago
  > I think you can always use unsafe to get access to a raw pointer I’m not sure about that. Also I don’t know for how long will it work, C++ has iterator invalidation rules. > you need someway to express ownership between both data structures Not every relation is ownership. Graph modes don’t own each other, an external index doesn’t own the indexed items, etc.
  - oconnor663 9y ago
    I'm really not sure, but I think there's a miscommunication about raw pointers in this thread. I think other folks are suggesting that you can solve the container composition problems you're talking about by inserting raw pointers into a HashMap. But I think you're reading that as obtaining raw pointers to the storage that a HashMap owns, which is why you're worried about iterator invalidation rules and stuff like that? (My understanding is that any reference into storage that a container owns is / could be completely invalidated by any &mut operation on that container.)
    - Const-me 9y ago
      To create an external index for the existing collection, I need to get pointer of the item stored in that existing collection, compute a key, and put both into the HashMap<tKey, tValue*> So, I need to do both. And also, I need to know when these pointer expire so I can rebuild my index when it happens.

pcwalton 9y ago

> And when I want to compose 2 data structures into my own higher-level one, for performance and memory access patterns I need these two lower-level structures to expose unsafe stuff at their API boundaries. The data structures I saw don’t do that, they’re designed to be consumed from safe Rust instead. Can you give a specific example of something you want to do that you can't? > I think in modern C++, with these iterators and smart pointers, you’re less likely to screw up dereferencing a wrong pointer. I don't think this is empirically true relative to C, but even if it is, use after free is still far too common in C++ code.

Const-me 9y ago
> Can you give a specific example of something you want to do that you can't? Compose a hash map + linked list into an LRU cache. Rust how has that in standard library, but they had to implement their own linked list for that. In C++ it’s just a few lines of code, because standard maps+lists compose just fine. Or (more generic example and thus harder to put in the standard library), add an index to existing collection. I have a large collection of some values. I want to build an index allowing to lookup values by some key. Values are not small, can’t afford duplicating them. If you’ll tell “just move values into a hashmap”, my response is “and I also want another, different index of the same set of values by different key”. Again, very easy in C++, encapsulate both the original collection, and a hashmap from key to value pointer. > use after free is still far too common in C++ code. In my experience, use after free = instant crash in debug build. Quite easy to detect and fix.
- burntsushi 9y ago
  Use an Rc? If you can't afford the reference count, then use unsafe/raw pointers, which it sounds like you'd do in C++ anyway.
  - Const-me 9y ago
    > then use unsafe/raw pointers Even if I manage to extract an unsafe pointer from that Rust collection, I don’t know for how long will it work. For C++ collections, iterator invalidation rules tell me that.
    - burntsushi 9y ago
      You'd have to box your values. And if you didn't want to do that, then I'd just used the indexing method mentioned elsewhere in this thread. I've used such things in performance critical code.
      - Const-me 9y ago
        > the indexing method mentioned elsewhere in this thread https://news.ycombinator.com/item?id=15180649 https://news.ycombinator.com/item?id=15180649
    - cesarb 9y ago
      > Even if I manage to extract an unsafe pointer from that Rust collection, It's easy, just get the & or &mut to the value (as if you were acessing it), and cast it to respectively * const or * mut. > I don’t know for how long will it work. For C++ collections, iterator invalidation rules tell me that. It's the same in Rust. Whenever the iterator would be invalidated in C++, the pointer you stashed above might point to the wrong place. This is not usually documented in Rust, because its borrow rules prevent you from stashing a reference while the collection mutates, but once you start playing with raw pointers, the borrow checker gets out of the way (references have a lifetime, pointers don't). You just have to be careful when casting the pointer back to a mutable ref ("unsafe { &mut *ptr }" is the trick, see the documentation for std::mem::transmute): mutable references are like C99's "restrict", so you should make sure to only ever have one live for each pointer at every moment, otherwise you're in undefined behavior land. ---- Anyway, going back to the parent comment, you said "Values are not small, can’t afford duplicating them". Might I suggest keeping the values in a Box<T> then, and making both collections point to the box? That way, you don't have to worry about a mutation in one of the collections invalidating the pointer, since the contents of a Box won't move in memory. And in fact, the usual Rust style for keeping a value in more than one collection would be to use a Rc<T>, which is basically a Box with a reference counter. That way, you don't need to play with raw pointers, and have no risk of a misstep. You pay the cost of incrementing/decrementing the reference counter only when adding/removing from the collection, and the reference counter is small.

burntsushi 9y ago

> And when I want to compose 2 data structures into my own higher-level one Could you give a specific example please? I compose data structures in Rust all the time.

Const-me 9y ago
https://news.ycombinator.com/item?id=15180500 https://news.ycombinator.com/item?id=15180500

bluejekyll 9y ago

More like 20 it seems ;) > I need these two lower-level structures to expose unsafe stuff at the API boundary Two thoughts: 1) I think you can always use unsafe to get access to a raw pointer (I honestly don't use unsafe often) 2) you need someway to express ownership between both data structures, this can be annoying, no doubt. > C++, with these iterators and smart pointers Does that make it safer than unsafe Rust? Maybe, but there's a lot less unsafe Rust even in these graphs...

dikaiosune 9y ago
If you click directly on the time of a comment, it's a link to a direct reply page.
Const-me 9y ago
> I think you can always use unsafe to get access to a raw pointer I’m not sure about that. Also I don’t know for how long will it work, C++ has iterator invalidation rules. > you need someway to express ownership between both data structures Not every relation is ownership. Graph modes don’t own each other, an external index doesn’t own the indexed items, etc.
- oconnor663 9y ago
  I'm really not sure, but I think there's a miscommunication about raw pointers in this thread. I think other folks are suggesting that you can solve the container composition problems you're talking about by inserting raw pointers into a HashMap. But I think you're reading that as obtaining raw pointers to the storage that a HashMap owns, which is why you're worried about iterator invalidation rules and stuff like that? (My understanding is that any reference into storage that a container owns is / could be completely invalidated by any &mut operation on that container.)
  - Const-me 9y ago
    To create an external index for the existing collection, I need to get pointer of the item stored in that existing collection, compute a key, and put both into the HashMap<tKey, tValue*> So, I need to do both. And also, I need to know when these pointer expire so I can rebuild my index when it happens.