3 ms·

> Even an obvious optimisation of moving the buffer declaration outside of the loop isn’t available to the compiler. Why? Can't the programmer just do this him

> Even an obvious optimisation of moving the buffer declaration outside of the loop isn’t available to the compiler. Why? Can't the programmer just do this himself?

Arch-TK 2y ago
The compiler cannot assume that the read call won't read from the mutable reference (well, it might be able to given a sufficiently sophisticated optimizer and/or aggressive inlining). The programmer, on the other hand, can do this, but the point is to make this implicitly possible by making it more explicit that read does not read from the buffer (and therefore allowing it to accept uninitialized memory).
- dietr1ch 2y ago
  And I don't think it can ensure that all the bits were written. I've been bit by people trying to reuse buffers/objects like this that were not fully rewritten in one of the possible re-uses. It's a bit puzzling how a change that just adds a new continue leads to memory corruption.
  - hansvm 2y ago
    Zig has a couple features to help with that. I assume Rust should too (probably not directly applicable to BorrowedBuf, but for the case of a reusable pool of objects)? It comes down to a piece of syntactic sugar, plus "result location semantics" guaranteeing that you won't have a copy. E.g.: my_ptr.* = .{ .x = 42, .y = 53 }; No matter how you choose to construct the intermediate fields (x and y in this example) with continues or other control flow, the very last step should be something that sets every field at once. If you miss one, the compiler will yell at you. If it compiles, the assembly is as if you filled in each field by hand.
    - dietr1ch 2y ago
      It's cool to have syntax for that, but I feel that the optimisation there works at a too high level. I'd like the backend to notice that a variable is being dropped and "re-created" on every iteration, and then figuring out how to initialise over a "dropped" zombie value. It'd be nice to have something like this, because I'm pretty sure people don't do this kind of optimisation all the time (it's annoying to leak the variable that should survive loops outside of the loop becase it's not mentioned elsewhere).
      - hansvm 2y ago
        That's another possible tradeoff. Suppose you want that feature and aren't giving any more information to the compiler than you normally do, two extremes exist: 1. Add any such pools you can reason about at compile-time (Rust does this, or at least it did the last time I profiled anything rusty at my last job). 2. Dynamically infer how such pools should behave at runtime. (1) has the same sorts of problems you see with auto-vectorization. It gives a modest boost for 'free' (Rust's implementation when I benchmarked it was sub-par, and a naive user-space pool would have been better), but it won't kick in that often. Perf wins for 'free' often aren't a bad thing. (2) isn't terrible if you're okay with a GC language. Many GC's effectively do something similar internally. The code complexity is similar. However, both ideas also have the potential to fall prey to the issue we were trying to circumvent in the first place. The "backend notic[ing]" only covers the performance aspect of the equation. The real value is in having a dedicated syntax for compiler-erroring partial initializations, combined with the knowledge that such code is free (compared to less-safe state-of-the-art options) at runtime. I think you're observing that APIs like in TFA are easy to mis-use, and my proposal says "make them easy to use," where you're proposing "make the compiler smart enough to not need them"?
Waterluvian 2y ago
Arguably the buffer belongs in the loop scope because it is only relevant there. It’s probably also safer from wrong use. This feels like exactly what you want the compiler to think about: a case where optimization comes at the cost of organization.
- immibis 2y ago
  Organization is basically completely irrelevant for a small piece of code like this, which fits in your head all at once.
lmm 2y ago
> Why? Because as far as the compiler is concerned it appears to change the behaviour, unless the compiler gets very fancy. > Can't the programmer just do this himself? Yes, but it's not really desirable for them to have to (and would arguably make the code less maintainable if they did). Doing the right thing should be easy.
auggierose 2y ago
Not if they are using Rust ... which is why I am not.
- vlovich123 2y ago
  What language would automatically be able to hoist the array outside the loop in that kind of code?
  - jhasse 2y ago
    C, because reading uninitialized memory is undefined behavior so the compile can assume it never happens.
    - jeroenhd 2y ago
      But wouldn't that change behavior? An empty, zero initialised array will contain data and a bunch of zeroes after a read, but if the next read only partially fills the buffer you end up with a buffer containing data from two reads. In this specific example there's no issue because the result of read() is being used to only write as much data as was read, but to me this seems like a pretty complicated and unlikely assumption to write optimizations for.
    - vlovich123 2y ago
      Care to present some proof? Here’s an counter proof that the compiler isn’t able to reason about the memory in that way https://godbolt.org/z/x7j8xoMxY https://godbolt.org/z/x7j8xoMxY There are cases where C can do loop hoisting, but the cases are a subset of what Rust does and this isn’t one of those.
      - uecker 2y ago
        You example doesn't show this. Also the allocation is hoisted out of the loop. The initialization is not and this would be invalid in general. It could eliminated in this case, but this would be dead store elimination.
vlovich123 2y ago
Because the compiler doesn’t know what read/write are doing to the buffer. And since it’s declared as [0; 4096], the compiler wouldn’t be able to do anything other than 0’ing the entire 4kib region on every read instead of what’s dirtied if it attempted to automatically hoist. BorrowedBuf is an attempt to let you declare [MaybeUninit::uninit(); 4096] which the compiler could hoist although there it doesn’t matter either since the allocation of the uninit array is just an adjustment of the stack pointer.

> Even an obvious optimisation of moving the buffer declaration outside of the loop isn’t available to the compiler. Why? Can't the programmer just do this himself?

Arch-TK 2y ago

The compiler cannot assume that the read call won't read from the mutable reference (well, it might be able to given a sufficiently sophisticated optimizer and/or aggressive inlining). The programmer, on the other hand, can do this, but the point is to make this implicitly possible by making it more explicit that read does not read from the buffer (and therefore allowing it to accept uninitialized memory).

dietr1ch 2y ago
And I don't think it can ensure that all the bits were written. I've been bit by people trying to reuse buffers/objects like this that were not fully rewritten in one of the possible re-uses. It's a bit puzzling how a change that just adds a new continue leads to memory corruption.
- hansvm 2y ago
  Zig has a couple features to help with that. I assume Rust should too (probably not directly applicable to BorrowedBuf, but for the case of a reusable pool of objects)? It comes down to a piece of syntactic sugar, plus "result location semantics" guaranteeing that you won't have a copy. E.g.: my_ptr.* = .{ .x = 42, .y = 53 }; No matter how you choose to construct the intermediate fields (x and y in this example) with continues or other control flow, the very last step should be something that sets every field at once. If you miss one, the compiler will yell at you. If it compiles, the assembly is as if you filled in each field by hand.
  - dietr1ch 2y ago
    It's cool to have syntax for that, but I feel that the optimisation there works at a too high level. I'd like the backend to notice that a variable is being dropped and "re-created" on every iteration, and then figuring out how to initialise over a "dropped" zombie value. It'd be nice to have something like this, because I'm pretty sure people don't do this kind of optimisation all the time (it's annoying to leak the variable that should survive loops outside of the loop becase it's not mentioned elsewhere).
    - hansvm 2y ago
      That's another possible tradeoff. Suppose you want that feature and aren't giving any more information to the compiler than you normally do, two extremes exist: 1. Add any such pools you can reason about at compile-time (Rust does this, or at least it did the last time I profiled anything rusty at my last job). 2. Dynamically infer how such pools should behave at runtime. (1) has the same sorts of problems you see with auto-vectorization. It gives a modest boost for 'free' (Rust's implementation when I benchmarked it was sub-par, and a naive user-space pool would have been better), but it won't kick in that often. Perf wins for 'free' often aren't a bad thing. (2) isn't terrible if you're okay with a GC language. Many GC's effectively do something similar internally. The code complexity is similar. However, both ideas also have the potential to fall prey to the issue we were trying to circumvent in the first place. The "backend notic[ing]" only covers the performance aspect of the equation. The real value is in having a dedicated syntax for compiler-erroring partial initializations, combined with the knowledge that such code is free (compared to less-safe state-of-the-art options) at runtime. I think you're observing that APIs like in TFA are easy to mis-use, and my proposal says "make them easy to use," where you're proposing "make the compiler smart enough to not need them"?

Waterluvian 2y ago

Arguably the buffer belongs in the loop scope because it is only relevant there. It’s probably also safer from wrong use. This feels like exactly what you want the compiler to think about: a case where optimization comes at the cost of organization.

immibis 2y ago
Organization is basically completely irrelevant for a small piece of code like this, which fits in your head all at once.

lmm 2y ago

> Why? Because as far as the compiler is concerned it appears to change the behaviour, unless the compiler gets very fancy. > Can't the programmer just do this himself? Yes, but it's not really desirable for them to have to (and would arguably make the code less maintainable if they did). Doing the right thing should be easy.

auggierose 2y ago

Not if they are using Rust ... which is why I am not.

vlovich123 2y ago
What language would automatically be able to hoist the array outside the loop in that kind of code?
- jhasse 2y ago
  C, because reading uninitialized memory is undefined behavior so the compile can assume it never happens.
  - jeroenhd 2y ago
    But wouldn't that change behavior? An empty, zero initialised array will contain data and a bunch of zeroes after a read, but if the next read only partially fills the buffer you end up with a buffer containing data from two reads. In this specific example there's no issue because the result of read() is being used to only write as much data as was read, but to me this seems like a pretty complicated and unlikely assumption to write optimizations for.
  - vlovich123 2y ago
    Care to present some proof? Here’s an counter proof that the compiler isn’t able to reason about the memory in that way https://godbolt.org/z/x7j8xoMxY https://godbolt.org/z/x7j8xoMxY There are cases where C can do loop hoisting, but the cases are a subset of what Rust does and this isn’t one of those.
    - uecker 2y ago
      You example doesn't show this. Also the allocation is hoisted out of the loop. The initialization is not and this would be invalid in general. It could eliminated in this case, but this would be dead store elimination.

vlovich123 2y ago

Because the compiler doesn’t know what read/write are doing to the buffer. And since it’s declared as [0; 4096], the compiler wouldn’t be able to do anything other than 0’ing the entire 4kib region on every read instead of what’s dirtied if it attempted to automatically hoist. BorrowedBuf is an attempt to let you declare [MaybeUninit::uninit(); 4096] which the compiler could hoist although there it doesn’t matter either since the allocation of the uninit array is just an adjustment of the stack pointer.