6 ms·

> "what does this language make easy" (C: raw memory manipulation!) Depending on what you mean by "raw memory manipulation" that can actually be surprisingly d

> "what does this language make easy" (C: raw memory manipulation!) Depending on what you mean by "raw memory manipulation" that can actually be surprisingly difficult with C, without running into undefined behavior, at least technically-speaking. e.g. type-punning through unions is defined by all implementations I can think of, but technically UB according to the standard. That said, if you know the UB rules, C/C++ are still the "nicest" mainstream languages to use for that sort of super-low-level manipulation of raw object representations. The non-UB type-punning methods in C/C++ (`memcpy(3)`, casting addresses to pointer-to-`char`), finicky as they are, are still more ergonomic and, ironically, safer than Rust's poorly-designed `mem::transmute()` API. Language-lawyering is quite the rabbit hole, BTW; for example, it was discovered a few years ago that the wording of the formal C++ memory model defined in the C++11 and later standards means all non-trivial C programs (technically-speaking, not according to any existing or sane implementation) invoke undefined behavior in C++. We'll be lucky if the bikeshedding over the fix for this gets resolved in time to make C++20. #include <stdlib.h> void *safe_ptr(void *p) { if (!p) exit(1); return p; } #define INIT(TYPE) safe_ptr(malloc(sizeof(TYPE))) int *get_intbuffer(void) { enum {N=3}; int *const intbuffer = (int*)INIT(int[N]); int i = N; /* Technically UB in C++; lifetime starts * w/ declaration or initializing w/ new * or placement-new, so reads/writes thru * the result of malloc w/o initializing * the objects w/ placement-new are UB */ while (i--) intbuffer[i] = i; return intbuffer; }

dbaupp 9y ago
> are still more ergonomic and, ironically, safer than Rust's poorly-designed `mem::transmute()` API. Eh, I disagree. I find: let x: f32 = ...; let y: i32 = mem::transmute(x); is nicer than float x = ...; int y; memcpy(&y, &x, sizeof x); And, in terms of safety, transmute manages the sizes of everything and makes sure they match. There's no risk of passing the wrong size, or having types with mismatched sizes (resulting in either "slicing", or buffer overflows). The only way I can see transmute being more dangerous than the equivalent is that one can allow the types to be inferred, which can result in badness if the inferred type isn't what the programmer is expecting. This is especially dangerous when type involves a lifetime. (I suppose one could argue that it also makes this dangerous operation easier to do, since there's an easy-to-find function for it. But... this goes both ways, since it also stops people being tempted into using undefined pointer casting like ^(float ^)(x) (using ^ for * since HN likes italics too much).) > Language-lawyering is quite the rabbit hole, BTW; for example, it was discovered a few years ago that the wording of the formal C++ memory model defined in the C++11 and later standards means all non-trivial C programs (technically-speaking, not according to any existing or sane implementation) invoke undefined behavior in C++. We'll be lucky if the bikeshedding over the fix for this gets resolved in time to make C++20. I'm curious if you've got a defect report link or similar, because my reading of "6.8 Object Lifetime" in the C++17 draft implies that the lifetime has started for those int objects since the two conditions are both satisfied: - "storage with the proper alignment and size for type T is obtained": the malloc'd pointer is fine. - "if the object has non-vacuous initialization, its initialization is complete": ints have vacuous initialization. But, I'm probably wrong, and would appreciate being corrected.
- smitherfield 9y ago
  > Eh, I disagree... The reason transmute is (particularly) unsafe is because it can be quite difficult to tell whether a variable is bound to a value, a (possibly implicitly-dereferenced) reference or a slice, and so, given transmute's implicit type inference, it's very easy to quietly get either the source or destination type very wrong. The size-checking is also I would argue useless bordering on worse than useless (false sense of security) since any two references or two slices are the same size, but are unlikely to actually be interconvertible, and it also fails to check alignment. C and C++ (particularly the latter) make it much more difficult to inadvertently cast between values and pointers, and the strict aliasing rules (aliasing almost always illegal except through `char`) are draconian enough to discourage the practice of type-punning-through-indirection (and ensuing alignment bugs) altogether. `memcpy` (and unions-as-implemented) is nice because it's explicit and it just works. > I'm curious if you've got a defect report link or similar I'm having a hard time finding the detailed original paper pointing it out, but see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0593r0.html http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p059...
  - dbaupp 9y ago
    > The reason transmute is (particularly) unsafe is because it can be quite difficult to tell whether a variable is bound to a value, a (possibly implicitly-dereferenced) reference or a slice, and so, given transmute's implicit type inference, it's very easy to quietly get either the source or destination type very wrong. Ah, so just the type inference I mentioned? I agree, and certainly try to never let transmute infer when I use it. However, I don't think one can end up with implicit dereferences or wildly unexpected types since there always has to be some context: the return value of transmute is completely unconstrained, meaning there needs to be something that suggests the type. Additionally, C++ can suffer in a similar way (but not quite identical) due to auto: auto source = ...; T dest; memcpy(&dest, &source, sizeof source); Or, in extreme cases: `auto dest = ...;`. > The size-checking is also I would argue useless bordering on worse than useless (false sense of security) since any two references or two slices are the same size, but are unlikely to actually be interconvertible, and it also fails to check alignment. ... C-style memcpy is strictly worse than all of this. But yes, maybe a false sense of security. But that's a bit like arguing that a safety guard on a buzz-saw doesn't stop all problems, and so is pointless. I agree that failing to check alignment of pointer destinations is unfortunate, but it's only one of many many problems that can occur with `unsafe` and even `transmute` itself. However, it's worth nothing that differing alignment between values being transmuted is not a problem: transmute::<[u8; 4], u32>(byte_array) is fine, despite the byte array only having alignment 1. > C and C++ (particularly the latter) make it much more difficult to inadvertently cast between values and pointers, and the strict aliasing rules (aliasing almost always illegal except through `char`) are draconian enough to discourage the practice of type-punning-through-indirection (and ensuing alignment bugs) altogether. I don't agree: - there's little difference between C/C++ and Rust other than the inference thing. C++ has C-style casts, and reinterpret_cast. - strict aliasing ends up being commonly violated for transmute-style casts (it's just so temptingly easy, and there's nothing that actually stops it compiling), and `unsafe` in Rust is also a fairly major discouragement to doing bad things. > `memcpy` (and unions-as-implemented) is nice because it's explicit and it just works. transmute is only fractionally less explicit (and it's well known bad practice to let it be completely implicit), and also just works. > I'm having a hard time finding the detailed original paper pointing it out, but see ... Thanks; interesting!

dbaupp 9y ago

> are still more ergonomic and, ironically, safer than Rust's poorly-designed `mem::transmute()` API. Eh, I disagree. I find: let x: f32 = ...; let y: i32 = mem::transmute(x); is nicer than float x = ...; int y; memcpy(&y, &x, sizeof x); And, in terms of safety, transmute manages the sizes of everything and makes sure they match. There's no risk of passing the wrong size, or having types with mismatched sizes (resulting in either "slicing", or buffer overflows). The only way I can see transmute being more dangerous than the equivalent is that one can allow the types to be inferred, which can result in badness if the inferred type isn't what the programmer is expecting. This is especially dangerous when type involves a lifetime. (I suppose one could argue that it also makes this dangerous operation easier to do, since there's an easy-to-find function for it. But... this goes both ways, since it also stops people being tempted into using undefined pointer casting like ^(float ^)(x) (using ^ for * since HN likes italics too much).) > Language-lawyering is quite the rabbit hole, BTW; for example, it was discovered a few years ago that the wording of the formal C++ memory model defined in the C++11 and later standards means all non-trivial C programs (technically-speaking, not according to any existing or sane implementation) invoke undefined behavior in C++. We'll be lucky if the bikeshedding over the fix for this gets resolved in time to make C++20. I'm curious if you've got a defect report link or similar, because my reading of "6.8 Object Lifetime" in the C++17 draft implies that the lifetime has started for those int objects since the two conditions are both satisfied: - "storage with the proper alignment and size for type T is obtained": the malloc'd pointer is fine. - "if the object has non-vacuous initialization, its initialization is complete": ints have vacuous initialization. But, I'm probably wrong, and would appreciate being corrected.

smitherfield 9y ago
> Eh, I disagree... The reason transmute is (particularly) unsafe is because it can be quite difficult to tell whether a variable is bound to a value, a (possibly implicitly-dereferenced) reference or a slice, and so, given transmute's implicit type inference, it's very easy to quietly get either the source or destination type very wrong. The size-checking is also I would argue useless bordering on worse than useless (false sense of security) since any two references or two slices are the same size, but are unlikely to actually be interconvertible, and it also fails to check alignment. C and C++ (particularly the latter) make it much more difficult to inadvertently cast between values and pointers, and the strict aliasing rules (aliasing almost always illegal except through `char`) are draconian enough to discourage the practice of type-punning-through-indirection (and ensuing alignment bugs) altogether. `memcpy` (and unions-as-implemented) is nice because it's explicit and it just works. > I'm curious if you've got a defect report link or similar I'm having a hard time finding the detailed original paper pointing it out, but see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0593r0.html http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p059...
- dbaupp 9y ago
  > The reason transmute is (particularly) unsafe is because it can be quite difficult to tell whether a variable is bound to a value, a (possibly implicitly-dereferenced) reference or a slice, and so, given transmute's implicit type inference, it's very easy to quietly get either the source or destination type very wrong. Ah, so just the type inference I mentioned? I agree, and certainly try to never let transmute infer when I use it. However, I don't think one can end up with implicit dereferences or wildly unexpected types since there always has to be some context: the return value of transmute is completely unconstrained, meaning there needs to be something that suggests the type. Additionally, C++ can suffer in a similar way (but not quite identical) due to auto: auto source = ...; T dest; memcpy(&dest, &source, sizeof source); Or, in extreme cases: `auto dest = ...;`. > The size-checking is also I would argue useless bordering on worse than useless (false sense of security) since any two references or two slices are the same size, but are unlikely to actually be interconvertible, and it also fails to check alignment. ... C-style memcpy is strictly worse than all of this. But yes, maybe a false sense of security. But that's a bit like arguing that a safety guard on a buzz-saw doesn't stop all problems, and so is pointless. I agree that failing to check alignment of pointer destinations is unfortunate, but it's only one of many many problems that can occur with `unsafe` and even `transmute` itself. However, it's worth nothing that differing alignment between values being transmuted is not a problem: transmute::<[u8; 4], u32>(byte_array) is fine, despite the byte array only having alignment 1. > C and C++ (particularly the latter) make it much more difficult to inadvertently cast between values and pointers, and the strict aliasing rules (aliasing almost always illegal except through `char`) are draconian enough to discourage the practice of type-punning-through-indirection (and ensuing alignment bugs) altogether. I don't agree: - there's little difference between C/C++ and Rust other than the inference thing. C++ has C-style casts, and reinterpret_cast. - strict aliasing ends up being commonly violated for transmute-style casts (it's just so temptingly easy, and there's nothing that actually stops it compiling), and `unsafe` in Rust is also a fairly major discouragement to doing bad things. > `memcpy` (and unions-as-implemented) is nice because it's explicit and it just works. transmute is only fractionally less explicit (and it's well known bad practice to let it be completely implicit), and also just works. > I'm having a hard time finding the detailed original paper pointing it out, but see ... Thanks; interesting!