4 ms·

It's pretty difficult to have no panics, because many functions allocate memory and what are they supposed to do when there is no memory left? Also many functio

It's pretty difficult to have no panics, because many functions allocate memory and what are they supposed to do when there is no memory left? Also many functions use addition and what is one supposed to do in case of overflow?

nicce 1y ago
Additions are easy. By default they are wrapped, and you can make them explicit with checked_ methods. Assuming that you are not using much recursion, you can eliminate most of the heap related memory panics by adding limited reservation checks for dynamic data, which is allocated based on user input/external data. You should also use statically sized types whennever possible. They are also faster.
- codedokode 1y ago
  Wrapping on overflow is wrong because this is not the math we expect. As a result, errors and vulnerabilities occur (look at Linux kernel for example).
  - nicce 1y ago
    It depends on the context. Of course the result may cause vulnerabilities if the program logic in bad context depends on it. But yeah, generally I would agree.
pdimitar 1y ago
Don't know about your parent poster but I didn't take it 100% literally. Obviously if there's no memory left then you crash; the kernel would likely murder your program half a second later anyway. But for arithmetics Rust has non-aborting bound checking API, if my memory serves. And that's what I'm trying hard to do in my Rust code f.ex. don't frivolously use `unwrap` or `expect`, ever. And just generally try hard to never use an API that can crash. You can write a few error branches that might never get triggered. It's not the end of the world.
- wahern 1y ago
  Dealing with integer overflow is much more burdensome than dealing with allocation failure, IME. Relatively speaking, allocation failure is closer to file descriptor limits in terms of how it effects code structure. But then I mostly use C when I'm not using a scripting language. In languages like Rust and C++ there's alot of hidden allocation in the high-level libraries that seem to be popular, perhaps because the notion that "there's nothing you can do" has infected too many minds. Of course, just like with opening files or integer arithmetic, if you don't pay any attention to handling the errors up front when writing your code, it can be an onerous if not impossible to task to refactor things after the fact.
  - pdimitar 1y ago
    Oh I agree, don't get me wrong. Both are pretty gnarly. I was approaching these problems strictly from the point of view of what can Rust do today really, nothing else. To me having checked and non-panicking API for integer overflows / underflows at least gives you some agency. If you don't have memory, well, usually you are cooked. Though one area where Rust can become even better there is to give us some API to reserve more memory upfront, maybe? Or I don't know, maybe adopt some of the memory-arena crates in stdlib. But yeah, agreed. Not the types of problems I want to have anymore (because I did have them in the past).
  - uecker 1y ago
    In C I simply use -fsanitize=signed-integer-overflow if I expect no overflow and checked arithmetic when I need to handle overflow. I do not think this is worse than in any other languages and seems less annoying than Rust. If I am lazy, I let allocation failure trap on null pointer dereference which is also safe, out-of-bounds accesses are avoided by -fsanitize=bounds (I avoid pointer arithmetic and unsafe casts where I can and essentially treat it like Rust's "unsafe").
- tialaramex 1y ago
  Rust provides a default integer of each common size and signedness, for which overflow is prohibited [but this prohibition may not be enforced in release compiled binaries depending on your chosen settings for the compiler, in this case what happens is not promised but today it will wrap - it's wrong to write code which does this on purpose - see the wrapping types below if you want that - but it won't cause UB if you do it anyway] Rust also provides Wrapping and Saturating wrapper types for these integers, which wrap (255 + 1 == 0) or saturate (255 + 1 == 255). Depending on your CPU either or both of these might just be "how the computer works anyway" and will accordingly be very fast. Neither of them is how humans normally think about arithmetic. Furthermore, Rust also provides operations which do all of the above, as well as the more fundamental "with carry" type operations where you get two results from the operation and must write your algorithms accordingly, and explicitly fallible operations where if you would overflow your operation reports that it did not succeed.
Arnavion 1y ago
>many functions allocate memory and what are they supposed to do when there is no memory left? Return an AllocationError. Rust unfortunately picked the wrong default here for the sake of convenience, along with the default of assuming a global allocator. It's now trying to add in explicit allocators and allocation failure handling (A:Allocator type param) at the cost of splitting the ecosystem (all third-party code, including parts of libstd itself like std::io::Read::read_to_end, only work with A=GlobalAlloc). Zig for example does it right by having explicit allocators from the start, plus good support for having the allocator outside the type (ArrayList vs ArrayListUnmanaged) so that multiple values within a composite type can all use the same allocator. >Also many functions use addition and what is one supposed to do in case of overflow? Return an error ( https://doc.rust-lang.org/stable/std/primitive.i64.html#method.checked_add https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ) or a signal that overflow occurred ( https://doc.rust-lang.org/stable/std/primitive.i64.html#method.overflowing_add https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ). Or use wrapping addition ( https://doc.rust-lang.org/stable/std/primitive.i64.html#method.wrapping_add https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ) if that was intended. Note that for the checked case, it is possible to have a newtype wrapper that impls std::ops::Add etc, so that you can continue using the compact `+` etc instead of the cumbersome `.checked_add(...)` etc. For the wrapping case libstd already has such a newtype: std::num::Wrapping. Also, there is a clippy lint for disallowing `+` etc ( https://rust-lang.github.io/rust-clippy/master/index.html#arithmetic_side_effects https://rust-lang.github.io/rust-clippy/master/index.html#ar... ), though I assume only the most masochistic people enable it. I actually tried to enable it once for some parsing code where I wanted to enforce checked arithmetic, but it pointlessly triggered on my Checked wrapper (as described in the previous paragraph) so I ended up disabling it.
- smj-edison 1y ago
  > Rust unfortunately picked the wrong default here I partially disagree with this. Using Zig style allocators doesn't really fit with Rust ergonomics, as it would require pretty extensive lifetime annotations. With no_std, you absolutely can roll your own allocation styles, at the price of more manual lifetime annotations. I do hope though that some library comes along that allows for Zig style collections, with the associated lifetimes... (It's been a bit painful rolling my own local allocator for audio processing).
PhilipRoman 1y ago
>what are they supposed to do when there is no memory left Well on Linux they are apparently supposed to return memory anyway and at some point in the future possibly SEGV your process when you happen to dereference some unrelated pointer.
- tialaramex 1y ago
  You can tell Linux that you don't want overcommit. You will probably discover that you're now even more miserable and change it back, but it's an option.
  - wizzwizz4 1y ago
    Whenever I switch off overcommitting, every program on my system (that I'm using) dies, one by one, over the course of 2–5 seconds, followed by Xorg. It's quite pretty.
  - Shorel 1y ago
    I did that and even with enormous amounts of free memory, Chrome and other Chromium browsers just die. They require overcommit just to open an empty window.
kllrnohj 1y ago
> Also many functions use addition and what is one supposed to do in case of overflow? Honestly this is where you'd throw an exception. It's a shame Rust refuses to have them, they are absolutely perfect for things like this...
- brooke2k 1y ago
  I'm confused by this, because a panic is essentially an exception. They can be thrown and caught (although it's extremely discouraged to do so). The only place where it would be different is if you explicitly set panics to abort instead of unwind, but that's not default behavior.
kevin_thibedeau 1y ago
> what are they supposed to do when there is no memory left? You abandon the current activity and bubble up the error to a stage where that effort can be tossed out or retried sometime later. i.e. Use the same error handling approach you would have to use for any other unreliable operation like networking.

nicce 1y ago

Additions are easy. By default they are wrapped, and you can make them explicit with checked_ methods. Assuming that you are not using much recursion, you can eliminate most of the heap related memory panics by adding limited reservation checks for dynamic data, which is allocated based on user input/external data. You should also use statically sized types whennever possible. They are also faster.

codedokode 1y ago
Wrapping on overflow is wrong because this is not the math we expect. As a result, errors and vulnerabilities occur (look at Linux kernel for example).
- nicce 1y ago
  It depends on the context. Of course the result may cause vulnerabilities if the program logic in bad context depends on it. But yeah, generally I would agree.

pdimitar 1y ago

Don't know about your parent poster but I didn't take it 100% literally. Obviously if there's no memory left then you crash; the kernel would likely murder your program half a second later anyway. But for arithmetics Rust has non-aborting bound checking API, if my memory serves. And that's what I'm trying hard to do in my Rust code f.ex. don't frivolously use `unwrap` or `expect`, ever. And just generally try hard to never use an API that can crash. You can write a few error branches that might never get triggered. It's not the end of the world.

wahern 1y ago
Dealing with integer overflow is much more burdensome than dealing with allocation failure, IME. Relatively speaking, allocation failure is closer to file descriptor limits in terms of how it effects code structure. But then I mostly use C when I'm not using a scripting language. In languages like Rust and C++ there's alot of hidden allocation in the high-level libraries that seem to be popular, perhaps because the notion that "there's nothing you can do" has infected too many minds. Of course, just like with opening files or integer arithmetic, if you don't pay any attention to handling the errors up front when writing your code, it can be an onerous if not impossible to task to refactor things after the fact.
- pdimitar 1y ago
  Oh I agree, don't get me wrong. Both are pretty gnarly. I was approaching these problems strictly from the point of view of what can Rust do today really, nothing else. To me having checked and non-panicking API for integer overflows / underflows at least gives you some agency. If you don't have memory, well, usually you are cooked. Though one area where Rust can become even better there is to give us some API to reserve more memory upfront, maybe? Or I don't know, maybe adopt some of the memory-arena crates in stdlib. But yeah, agreed. Not the types of problems I want to have anymore (because I did have them in the past).
- uecker 1y ago
  In C I simply use -fsanitize=signed-integer-overflow if I expect no overflow and checked arithmetic when I need to handle overflow. I do not think this is worse than in any other languages and seems less annoying than Rust. If I am lazy, I let allocation failure trap on null pointer dereference which is also safe, out-of-bounds accesses are avoided by -fsanitize=bounds (I avoid pointer arithmetic and unsafe casts where I can and essentially treat it like Rust's "unsafe").
tialaramex 1y ago
Rust provides a default integer of each common size and signedness, for which overflow is prohibited [but this prohibition may not be enforced in release compiled binaries depending on your chosen settings for the compiler, in this case what happens is not promised but today it will wrap - it's wrong to write code which does this on purpose - see the wrapping types below if you want that - but it won't cause UB if you do it anyway] Rust also provides Wrapping and Saturating wrapper types for these integers, which wrap (255 + 1 == 0) or saturate (255 + 1 == 255). Depending on your CPU either or both of these might just be "how the computer works anyway" and will accordingly be very fast. Neither of them is how humans normally think about arithmetic. Furthermore, Rust also provides operations which do all of the above, as well as the more fundamental "with carry" type operations where you get two results from the operation and must write your algorithms accordingly, and explicitly fallible operations where if you would overflow your operation reports that it did not succeed.

Arnavion 1y ago

>many functions allocate memory and what are they supposed to do when there is no memory left? Return an AllocationError. Rust unfortunately picked the wrong default here for the sake of convenience, along with the default of assuming a global allocator. It's now trying to add in explicit allocators and allocation failure handling (A:Allocator type param) at the cost of splitting the ecosystem (all third-party code, including parts of libstd itself like std::io::Read::read_to_end, only work with A=GlobalAlloc). Zig for example does it right by having explicit allocators from the start, plus good support for having the allocator outside the type (ArrayList vs ArrayListUnmanaged) so that multiple values within a composite type can all use the same allocator. >Also many functions use addition and what is one supposed to do in case of overflow? Return an error ( https://doc.rust-lang.org/stable/std/primitive.i64.html#method.checked_add https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ) or a signal that overflow occurred ( https://doc.rust-lang.org/stable/std/primitive.i64.html#method.overflowing_add https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ). Or use wrapping addition ( https://doc.rust-lang.org/stable/std/primitive.i64.html#method.wrapping_add https://doc.rust-lang.org/stable/std/primitive.i64.html#meth... ) if that was intended. Note that for the checked case, it is possible to have a newtype wrapper that impls std::ops::Add etc, so that you can continue using the compact `+` etc instead of the cumbersome `.checked_add(...)` etc. For the wrapping case libstd already has such a newtype: std::num::Wrapping. Also, there is a clippy lint for disallowing `+` etc ( https://rust-lang.github.io/rust-clippy/master/index.html#arithmetic_side_effects https://rust-lang.github.io/rust-clippy/master/index.html#ar... ), though I assume only the most masochistic people enable it. I actually tried to enable it once for some parsing code where I wanted to enforce checked arithmetic, but it pointlessly triggered on my Checked wrapper (as described in the previous paragraph) so I ended up disabling it.

smj-edison 1y ago
> Rust unfortunately picked the wrong default here I partially disagree with this. Using Zig style allocators doesn't really fit with Rust ergonomics, as it would require pretty extensive lifetime annotations. With no_std, you absolutely can roll your own allocation styles, at the price of more manual lifetime annotations. I do hope though that some library comes along that allows for Zig style collections, with the associated lifetimes... (It's been a bit painful rolling my own local allocator for audio processing).

PhilipRoman 1y ago

>what are they supposed to do when there is no memory left Well on Linux they are apparently supposed to return memory anyway and at some point in the future possibly SEGV your process when you happen to dereference some unrelated pointer.

tialaramex 1y ago
You can tell Linux that you don't want overcommit. You will probably discover that you're now even more miserable and change it back, but it's an option.
- wizzwizz4 1y ago
  Whenever I switch off overcommitting, every program on my system (that I'm using) dies, one by one, over the course of 2–5 seconds, followed by Xorg. It's quite pretty.
- Shorel 1y ago
  I did that and even with enormous amounts of free memory, Chrome and other Chromium browsers just die. They require overcommit just to open an empty window.

kllrnohj 1y ago

> Also many functions use addition and what is one supposed to do in case of overflow? Honestly this is where you'd throw an exception. It's a shame Rust refuses to have them, they are absolutely perfect for things like this...

brooke2k 1y ago
I'm confused by this, because a panic is essentially an exception. They can be thrown and caught (although it's extremely discouraged to do so). The only place where it would be different is if you explicitly set panics to abort instead of unwind, but that's not default behavior.

kevin_thibedeau 1y ago

> what are they supposed to do when there is no memory left? You abandon the current activity and bubble up the error to a stage where that effort can be tossed out or retried sometime later. i.e. Use the same error handling approach you would have to use for any other unreliable operation like networking.