4 ms·

Surely this is true, but I still have the feeling that libraries in Rust tend to have more unsafe code than Java, Python, C# or others, maybe even more unsafe c

Surely this is true, but I still have the feeling that libraries in Rust tend to have more unsafe code than Java, Python, C# or others, maybe even more unsafe code than needed. Perhaps this is related to the problem domain.

burntsushi 4y ago
Maybe. I don't know. You'd have to collect some data. > Perhaps this is related to the problem domain. Yes, I included that possibility in my comment here: https://news.ycombinator.com/item?id=33821787 https://news.ycombinator.com/item?id=33821787
saagarjha 4y ago
Rust code frequently is used in a systems programming context, where it interoperates with unsafe code or needs to occasionally overrule the compiler to satisfy performance requirements.
tialaramex 4y ago
You would definitely need to control for domain. A Rust library for some sort of mathematical modelling might well need no unsafe at all, while a Java library for controlling some hardware might soon turn into JNI talking to some C++ code and oops you're unsafe. In C# you need to reach for unsafe to do some of the stuff Rust can just do safely anyway. Did you know a C# struct with an array of 8 ints in it, doesn't actually have the eight ints baked inside the struct? It was easier in the CLR not to do that, so they didn't. Which means C# structs which look like a compact single object that surely lives in a single cache line don't actually do that in safe C#. You need unsafe.
- kaba0 4y ago
  > You need unsafe. You beed it for that feature. It is questionable whether you really want to mandate a special memory layout (because you can’t really do that even in Rust, you don’t have explicit control of struct alignments, paddings, order(!) )
  - steveklabnik 4y ago
    Rust absolutely gives you control over alignment, padding, and ordering. It’s just not the default. Ask for those things and you shall be given it.
    - kaba0 4y ago
      Could you point me to some resources on that? I only know about #[repr] options, but that isn’t absolute control (e.g. for having structs usable from rust and internal asm)
      - steveklabnik 4y ago
        What is “internal assembly”? I’m not familiar with that term. Is there anything else that the various repr options don’t give you? My team at work does OS dev in Rust, and haven’t ever run into cases where Rust can’t do what we need it to do in these cases.
        kaba0 4y ago
        * inline assembly, just my brain stopped working for a sec :D Well, my specific case is writing a fast interpreter in Rust, where I would like to use elements like a stackframe from both inline asm and proper Rust code. In my first iteration I chose a dynamically sized u64 array, wrapped in a safe API, because I couldn’t be more specific. But even with known size elements the best I can do - to my knowledge - is Layout? Or just a raw pointer and a wrapper with helper functions, as otherwise I can’t modify the object in question from both places.
        steveklabnik 4y ago
        Ah yeah no worries :) It’s sort of tough because I am only familiar in passing with the patterns in that type of code, but Layout is an allocator API, so I’m not 100% sure why it would be used here. I’d guess that if I was doing something like this, I’d be casting it to and from a struct that’s defined correctly. This is one area where stuff is a little simpler than C, thanks to the lack of TBAA, though many projects do turn that off.
- pjmlp 4y ago
  It does if you learn to use C# properly, https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.structlayoutattribute.pack?view=net-7.0 https://learn.microsoft.com/en-us/dotnet/api/system.runtime.... In actual native code produced by RyuJit, you don't need to worry about cache lines for single instances, because the struct might not even exist at all, the Jit having mapped fields into CPU registers instead. When it matters, like the struct being part of an array, use StructLayout.
  - tialaramex 4y ago
    That link seems like it's about alignment rather than about arrays inside structures?
    - pjmlp 4y ago
      No, it is about alignment and packing, you use StructLayout attribute alongside LayoutKind and FieldOffsetAttribute. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.layoutkind?source=recommendations&view=net-7.0 https://learn.microsoft.com/en-us/dotnet/api/system.runtime.... You main issue was how structures arrange their fields. Also regarding arrays and structs, as of C# 7 you can use fixed to declare static arrays inside structs, however these structs need to be marked as unsafe.
      - tialaramex 4y ago
        > as of C# 7 you can use fixed to declare static arrays inside structs, however these structs need to be marked as unsafe. That is exactly what I was talking about.
        pjmlp 4y ago
        Ah ok, somehow misunderstood that. However there are actually good reasons for it to be unsafe, although it is debatable if that alone should be it. One is due to the interactions with the GC, in case it moves the data and there are references to its elements, and stack size. One way to get around it is to use AoS instead of SoA, which is any the best option if performance is the ultimate goal.
- raxxorraxor 4y ago
  > It was easier in the CLR not to do that This has also advantages in that you don't need to allocate the struct in a coherent memory block. Edge case of course, but there are domains where this is relevant. There was an allocation bug once because unsafe code needs to be allocated consecutively but most memory checks that only returned available memory failed to account for fragmented memory.

burntsushi 4y ago

Maybe. I don't know. You'd have to collect some data. > Perhaps this is related to the problem domain. Yes, I included that possibility in my comment here: https://news.ycombinator.com/item?id=33821787 https://news.ycombinator.com/item?id=33821787

saagarjha 4y ago

Rust code frequently is used in a systems programming context, where it interoperates with unsafe code or needs to occasionally overrule the compiler to satisfy performance requirements.

tialaramex 4y ago

You would definitely need to control for domain. A Rust library for some sort of mathematical modelling might well need no unsafe at all, while a Java library for controlling some hardware might soon turn into JNI talking to some C++ code and oops you're unsafe. In C# you need to reach for unsafe to do some of the stuff Rust can just do safely anyway. Did you know a C# struct with an array of 8 ints in it, doesn't actually have the eight ints baked inside the struct? It was easier in the CLR not to do that, so they didn't. Which means C# structs which look like a compact single object that surely lives in a single cache line don't actually do that in safe C#. You need unsafe.

kaba0 4y ago
> You need unsafe. You beed it for that feature. It is questionable whether you really want to mandate a special memory layout (because you can’t really do that even in Rust, you don’t have explicit control of struct alignments, paddings, order(!) )
- steveklabnik 4y ago
  Rust absolutely gives you control over alignment, padding, and ordering. It’s just not the default. Ask for those things and you shall be given it.
  - kaba0 4y ago
    Could you point me to some resources on that? I only know about #[repr] options, but that isn’t absolute control (e.g. for having structs usable from rust and internal asm)
    - steveklabnik 4y ago
      What is “internal assembly”? I’m not familiar with that term. Is there anything else that the various repr options don’t give you? My team at work does OS dev in Rust, and haven’t ever run into cases where Rust can’t do what we need it to do in these cases.
      - kaba0 4y ago
        * inline assembly, just my brain stopped working for a sec :D Well, my specific case is writing a fast interpreter in Rust, where I would like to use elements like a stackframe from both inline asm and proper Rust code. In my first iteration I chose a dynamically sized u64 array, wrapped in a safe API, because I couldn’t be more specific. But even with known size elements the best I can do - to my knowledge - is Layout? Or just a raw pointer and a wrapper with helper functions, as otherwise I can’t modify the object in question from both places.
        steveklabnik 4y ago
        Ah yeah no worries :) It’s sort of tough because I am only familiar in passing with the patterns in that type of code, but Layout is an allocator API, so I’m not 100% sure why it would be used here. I’d guess that if I was doing something like this, I’d be casting it to and from a struct that’s defined correctly. This is one area where stuff is a little simpler than C, thanks to the lack of TBAA, though many projects do turn that off.
pjmlp 4y ago
It does if you learn to use C# properly, https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.structlayoutattribute.pack?view=net-7.0 https://learn.microsoft.com/en-us/dotnet/api/system.runtime.... In actual native code produced by RyuJit, you don't need to worry about cache lines for single instances, because the struct might not even exist at all, the Jit having mapped fields into CPU registers instead. When it matters, like the struct being part of an array, use StructLayout.
- tialaramex 4y ago
  That link seems like it's about alignment rather than about arrays inside structures?
  - pjmlp 4y ago
    No, it is about alignment and packing, you use StructLayout attribute alongside LayoutKind and FieldOffsetAttribute. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.layoutkind?source=recommendations&view=net-7.0 https://learn.microsoft.com/en-us/dotnet/api/system.runtime.... You main issue was how structures arrange their fields. Also regarding arrays and structs, as of C# 7 you can use fixed to declare static arrays inside structs, however these structs need to be marked as unsafe.
    - tialaramex 4y ago
      > as of C# 7 you can use fixed to declare static arrays inside structs, however these structs need to be marked as unsafe. That is exactly what I was talking about.
      - pjmlp 4y ago
        Ah ok, somehow misunderstood that. However there are actually good reasons for it to be unsafe, although it is debatable if that alone should be it. One is due to the interactions with the GC, in case it moves the data and there are references to its elements, and stack size. One way to get around it is to use AoS instead of SoA, which is any the best option if performance is the ultimate goal.
raxxorraxor 4y ago
> It was easier in the CLR not to do that This has also advantages in that you don't need to allocate the struct in a coherent memory block. Edge case of course, but there are domains where this is relevant. There was an allocation bug once because unsafe code needs to be allocated consecutively but most memory checks that only returned available memory failed to account for fragmented memory.