5 ms·

This would be a bad thing. It's taking code written in terms of value semantics, copying stuff around, and replacing it with more efficient code that avoids th

This would be a bad thing. It's taking code written in terms of value semantics, copying stuff around, and replacing it with more efficient code that avoids the copy. Doing that by improving the compiler is a win. Doing it by changing the source to be easier to compile is a loss. You're trading readability for performance when instead you should fix the compiler and get both.

loeg 3y ago
Disagree. Using reference (pointer) syntax explicitly to avoid relying on a non-deterministic compiler optimization doesn't decrease readability. It is extremely naive to assume that a heuristic-guided compiler optimization will always work or that you never need to write your code explicitly in a system that aims to be high-performance, like Tiger Beetle. Also, some of the optimizations they are hunting are implicit and surprising. Probably not something a new compiler optimization is going to automatically fix.
- mhh__ 3y ago
  Compilers in general aren't always perfect so if you really genuinely care about copying that much you should be inspecting and profiling the binary they output e.g. unless you have some truly massive value types (at which point my eyebrows would be raised anyway) being copied, the cost is probably going to be in the same ballpark as some other compiler-fuckups too
  - loeg 3y ago
    > Compilers in general aren't always perfect Right. > so if you really genuinely care about copying that much you should be inspecting and profiling the binary they output Sure, but manual inspection is burdensome and unreliable. Are you advocating against just use the language-level constructs (pointers/references) that guarantee a copy isn't performed?
  - OfferFun6595 3y ago
    True, not all companies have a perfect data to be relied upon. Some of them are old and redundant
- anonymoushn 3y ago
  Using the pointer syntax means "I want to mutate this" and not using it means "I don't need to mutate this." So it's unfortunate if readers cannot count on this meaning because the compiler is defective.
  - loeg 3y ago
    Most languages have something like the ‘const’ keyword (or in Rust, eliding ‘mut’).
    - anonymoushn 3y ago
      The post is about a specific language. The way that you write the loop without being able to mutate the elements in this language is like this: for (items) |item| { } The way that you write the loop so that you can mutate the elements is like this: for (items) |*item| { } A way that you could try writing the loop so that it's clear that the elements are not mutated, even though you are using the pointer syntax, is maybe like this: for (items) |*_item| { const item: *const @typeInfo(@TypeOf(_item)).Pointer.child = _item; // there's no "drop _item" so unfortunately _item remains in scope but we promise not to use it. } I wasn't sure this would work, but it looks like you can also manage with this: const _items: []const @typeInfo(@TypeOf(items)).Pointer.child = items; for (_items) |*item| { // now item is of the correct type, even if items is a slice of mutable things. }
      - loeg 3y ago
        If Zig does not have a |*const item| syntax, maybe adding it would be a good idea (and also satisfy your concern)? (Zig already has a concept of mutable and const pointers.) For example, in C++ I would distinguish between mutating iteration and non-mutating iteration like this: for (const auto& item : collection) {} // non-mutating for (auto& item : collection) {} // mutating
      - jrockway 3y ago
        > there's no "drop _item" so unfortunately _item remains in scope but we promise not to use it This is actually a very intriguing programming language keyword. I feel like there are lots of cases where you don't want to split something into a separate function just to avoid being able to use a variable that was used previously, and these things often get refactored over time to the point where the not-to-be-used variable gets used unexpectedly.
- JonChesterfield 3y ago
  > ... avoid relying on a non-deterministic compiler optimization doesn't decrease readability. It is extremely naive to assume that a heuristic-guided ... This is a post about crawling the compiler IR to find things the compiler missed so that the programmer can change their code to work around the compiler. Even if the crawl-the-IR is a heuristic, moving that into the compiler would mean exactly the cases that were reported are now fixed automatically. Non-deterministic compilers are broken. Arguably by definition, but also by heuristic as a non-deterministic compiler is enough of a nightmare to debug and fix that it can be expected to grow more errors over time. Heuristic compilers are very much a thing. Also sadly fragile optimisations are a thing, where minor changes to the application take it over some threshold and the change in output is larger than anticipated. However, the plan of attack of "change the program until the compiler did what you hoped for" is itself a heuristic that future changes to program and compiler may upset. An alternative is to patch the compiler to do the right thing, and add a regression test to the compiler so that it will continue to do the right thing. That has better engineering properties in the large.
  - loeg 3y ago
    Apologies for the confusion, but you've misread my comment and run with it. I don't think there's much here that's responsive to what I intended, so I'll elaborate a little: > Non-deterministic compilers are broken. I am not talking about non-determinism from run-to-run with identical sources and compiler version. I am talking about output variations between (1) different versions of the compiler, and (2) as the sources change slightly. Either of these things can easily nudge the heuristics a compiler uses in optimization into doing something different/worse.
    - JonChesterfield 3y ago
      It's all good, we're talking past each other slightly. Thanks for the clarification on non-determinism. I think you're giving sensible advice for the world as it is, I'm speculating as to what the world should be. Minor changes in compiler version and input source leading to widely varying output is a definite hazard. Especially when people change their code to make it run faster, or change the compiler to make some code run faster. That's one with no clear good solution.
matklad 3y ago
Depends on the context! For close-to-the-human high level languages, you generally want to solve these things through optimizations. For close-to-the-cpu low-level languages, you rather want tools to express the desired behavior in the source code. That is, you need an ability to write code in a way that _guarantees_ that optimization triggers. Eg, you want explicit SIMD types rather than just loop auto-vectorization. Zig is very much on the close-to-cpu side of the spectrum here. When it comes to copies, the current version of the language/compiler doesn't _yet_ provide the required guarantees with required ergonomics, but this is being actively worked on: - https://github.com/ziglang/zig/issues/2765 https://github.com/ziglang/zig/issues/2765 - https://github.com/ziglang/zig/issues/12251 https://github.com/ziglang/zig/issues/12251 - https://github.com/ziglang/zig/issues/5973 https://github.com/ziglang/zig/issues/5973 In the meantime, you need to look one level below the source language to ensure that the runtime behavior of the code is what you want it to be.
- JonChesterfield 3y ago
  > Depends on the context! Interesting distinction to draw. That's a pretty good definition for high level vs low level programming languages. Vector types and compiler intrinsics with an eye on the generated assembly are a good way to go for high performance subroutines. If you're willing to change the compiler back end to better serve the ISA, there's a decent chance you can emit exactly the machine code you wanted from somewhat convenient source code. edit: your post at https://matklad.github.io/2023/04/09/can-you-trust-a-compiler-to-optimize-your-code.html https://matklad.github.io/2023/04/09/can-you-trust-a-compile... is great on this topic, thanks for the blog. Where I disagree is the implication that higher level languages do not need a way to write code certain that an optimisation (transform?) will fire. All the constexpr clutter from C++ is an attempt to force optimisations to happen regardless of compiler (or compiler invocation, e.g. to say you want some map thing in .rodata in -O0 debug builds). Transforms guaranteed by the language - whether vectorisation, monomorphising generics, garbage collection, constant folding - are inherently more useful things than heuristic optimisations where you hope for the best. I'm happy to see Zig is moving in that direction for memory copies and wish them good hunting with the aliasing analysis. Hinted at in a sister comment, I think a missing lever in software dev is adding guaranteed-to-fire compiler transforms in application libraries. E.g. define some regex implementation along with optimisations that act on said regex implementation. Requires a sanely extensible compiler.
mhh__ 3y ago
I think there's a deeper problem in that the program can't easily tell the compiler what to do - we have pragmas and so on but they're crap. I remember a Jon Chesterfield pointing out to me that people mistake inlining to be solely about call-overhead and not specialization - you should be able to nudge the compiler which one you want if at all (the "don't do anything" case being more often desired than pass-authors might think, myself include)
- masklinn 3y ago
  > I remember a Jon Chesterfield pointing out to me that people mistake inlining to be solely about call-overhead and not specialization I recall multiple mentions over the years that inlining is the most important optimisation because of how many further optimisations it unlocks. I guess having some experience with Rust helps there, as the inlining itself is only a small fraction of all the work required to achieve "zero-cost abstraction". It’s very much load-bearing, but it’s the start if the work, not the end.
  - mhh__ 3y ago
    Not particularly unique to rust
    - chlorion 3y ago
      I don't think many languages other than rust and c++ use monomorphization extremely heavily. Pretty much everything in rust is going to be using traits and c++ uses templates for most things as well. Every time you use a trait or a template, the code is expanded for the specific instance, and this gives a lot of opportunity for inlining stuff. A lot of other languages would not have the same opportunity to inline code when doing polymorphism stuff because they rely on dynamic dispatch or something similar that can't be inlined very well or at all. So I'm not disagreeing with you at all but I think rust and c++ get to inline stuff a little more aggressively, but this is just my theory and may be incorrect!
- JonChesterfield 3y ago
  > I think there's a deeper problem in that the program can't easily tell the compiler what to do Yes. We have divisions - application, language library, language implementation - which are broadly convenient but drop information on the boundaries. The implementation probably pattern matches on the library and the library will probably depends on quirks of the implementation (e.g. compiler intrinsics written for it), but the application is unlikely to be be able to extend the implementation. In cases where it can, there's a tendency to call it monkey patching and disapprove. > people mistake inlining to be solely about call-overhead and not specialization Yeah, that sounds like me. Inlining is an approximation to per-callsite specialisation which is easier to implement, where you pay for ease of compiler development with (machine) code duplication. attributes noinline and alwaysinline are relatively likely to exist, but specialise with respect to the third parameter is still on a wishlist. This is in the ballpark of compilers could be better, and should be better, than the current state of the art.
tedunangst 3y ago
The year of the sufficiently smart compiler is always just over the horizon.
saagarjha 3y ago
Well, at some point you need to deal with a compiler that isn’t fixed yet.

loeg 3y ago

Disagree. Using reference (pointer) syntax explicitly to avoid relying on a non-deterministic compiler optimization doesn't decrease readability. It is extremely naive to assume that a heuristic-guided compiler optimization will always work or that you never need to write your code explicitly in a system that aims to be high-performance, like Tiger Beetle. Also, some of the optimizations they are hunting are implicit and surprising. Probably not something a new compiler optimization is going to automatically fix.

mhh__ 3y ago
Compilers in general aren't always perfect so if you really genuinely care about copying that much you should be inspecting and profiling the binary they output e.g. unless you have some truly massive value types (at which point my eyebrows would be raised anyway) being copied, the cost is probably going to be in the same ballpark as some other compiler-fuckups too
- loeg 3y ago
  > Compilers in general aren't always perfect Right. > so if you really genuinely care about copying that much you should be inspecting and profiling the binary they output Sure, but manual inspection is burdensome and unreliable. Are you advocating against just use the language-level constructs (pointers/references) that guarantee a copy isn't performed?
- OfferFun6595 3y ago
  True, not all companies have a perfect data to be relied upon. Some of them are old and redundant
anonymoushn 3y ago
Using the pointer syntax means "I want to mutate this" and not using it means "I don't need to mutate this." So it's unfortunate if readers cannot count on this meaning because the compiler is defective.
- loeg 3y ago
  Most languages have something like the ‘const’ keyword (or in Rust, eliding ‘mut’).
  - anonymoushn 3y ago
    The post is about a specific language. The way that you write the loop without being able to mutate the elements in this language is like this: for (items) |item| { } The way that you write the loop so that you can mutate the elements is like this: for (items) |*item| { } A way that you could try writing the loop so that it's clear that the elements are not mutated, even though you are using the pointer syntax, is maybe like this: for (items) |*_item| { const item: *const @typeInfo(@TypeOf(_item)).Pointer.child = _item; // there's no "drop _item" so unfortunately _item remains in scope but we promise not to use it. } I wasn't sure this would work, but it looks like you can also manage with this: const _items: []const @typeInfo(@TypeOf(items)).Pointer.child = items; for (_items) |*item| { // now item is of the correct type, even if items is a slice of mutable things. }
    - loeg 3y ago
      If Zig does not have a |*const item| syntax, maybe adding it would be a good idea (and also satisfy your concern)? (Zig already has a concept of mutable and const pointers.) For example, in C++ I would distinguish between mutating iteration and non-mutating iteration like this: for (const auto& item : collection) {} // non-mutating for (auto& item : collection) {} // mutating
    - jrockway 3y ago
      > there's no "drop _item" so unfortunately _item remains in scope but we promise not to use it This is actually a very intriguing programming language keyword. I feel like there are lots of cases where you don't want to split something into a separate function just to avoid being able to use a variable that was used previously, and these things often get refactored over time to the point where the not-to-be-used variable gets used unexpectedly.
JonChesterfield 3y ago
> ... avoid relying on a non-deterministic compiler optimization doesn't decrease readability. It is extremely naive to assume that a heuristic-guided ... This is a post about crawling the compiler IR to find things the compiler missed so that the programmer can change their code to work around the compiler. Even if the crawl-the-IR is a heuristic, moving that into the compiler would mean exactly the cases that were reported are now fixed automatically. Non-deterministic compilers are broken. Arguably by definition, but also by heuristic as a non-deterministic compiler is enough of a nightmare to debug and fix that it can be expected to grow more errors over time. Heuristic compilers are very much a thing. Also sadly fragile optimisations are a thing, where minor changes to the application take it over some threshold and the change in output is larger than anticipated. However, the plan of attack of "change the program until the compiler did what you hoped for" is itself a heuristic that future changes to program and compiler may upset. An alternative is to patch the compiler to do the right thing, and add a regression test to the compiler so that it will continue to do the right thing. That has better engineering properties in the large.
- loeg 3y ago
  Apologies for the confusion, but you've misread my comment and run with it. I don't think there's much here that's responsive to what I intended, so I'll elaborate a little: > Non-deterministic compilers are broken. I am not talking about non-determinism from run-to-run with identical sources and compiler version. I am talking about output variations between (1) different versions of the compiler, and (2) as the sources change slightly. Either of these things can easily nudge the heuristics a compiler uses in optimization into doing something different/worse.
  - JonChesterfield 3y ago
    It's all good, we're talking past each other slightly. Thanks for the clarification on non-determinism. I think you're giving sensible advice for the world as it is, I'm speculating as to what the world should be. Minor changes in compiler version and input source leading to widely varying output is a definite hazard. Especially when people change their code to make it run faster, or change the compiler to make some code run faster. That's one with no clear good solution.

matklad 3y ago

Depends on the context! For close-to-the-human high level languages, you generally want to solve these things through optimizations. For close-to-the-cpu low-level languages, you rather want tools to express the desired behavior in the source code. That is, you need an ability to write code in a way that _guarantees_ that optimization triggers. Eg, you want explicit SIMD types rather than just loop auto-vectorization. Zig is very much on the close-to-cpu side of the spectrum here. When it comes to copies, the current version of the language/compiler doesn't _yet_ provide the required guarantees with required ergonomics, but this is being actively worked on: - https://github.com/ziglang/zig/issues/2765 https://github.com/ziglang/zig/issues/2765 - https://github.com/ziglang/zig/issues/12251 https://github.com/ziglang/zig/issues/12251 - https://github.com/ziglang/zig/issues/5973 https://github.com/ziglang/zig/issues/5973 In the meantime, you need to look one level below the source language to ensure that the runtime behavior of the code is what you want it to be.

JonChesterfield 3y ago
> Depends on the context! Interesting distinction to draw. That's a pretty good definition for high level vs low level programming languages. Vector types and compiler intrinsics with an eye on the generated assembly are a good way to go for high performance subroutines. If you're willing to change the compiler back end to better serve the ISA, there's a decent chance you can emit exactly the machine code you wanted from somewhat convenient source code. edit: your post at https://matklad.github.io/2023/04/09/can-you-trust-a-compiler-to-optimize-your-code.html https://matklad.github.io/2023/04/09/can-you-trust-a-compile... is great on this topic, thanks for the blog. Where I disagree is the implication that higher level languages do not need a way to write code certain that an optimisation (transform?) will fire. All the constexpr clutter from C++ is an attempt to force optimisations to happen regardless of compiler (or compiler invocation, e.g. to say you want some map thing in .rodata in -O0 debug builds). Transforms guaranteed by the language - whether vectorisation, monomorphising generics, garbage collection, constant folding - are inherently more useful things than heuristic optimisations where you hope for the best. I'm happy to see Zig is moving in that direction for memory copies and wish them good hunting with the aliasing analysis. Hinted at in a sister comment, I think a missing lever in software dev is adding guaranteed-to-fire compiler transforms in application libraries. E.g. define some regex implementation along with optimisations that act on said regex implementation. Requires a sanely extensible compiler.

mhh__ 3y ago

I think there's a deeper problem in that the program can't easily tell the compiler what to do - we have pragmas and so on but they're crap. I remember a Jon Chesterfield pointing out to me that people mistake inlining to be solely about call-overhead and not specialization - you should be able to nudge the compiler which one you want if at all (the "don't do anything" case being more often desired than pass-authors might think, myself include)

masklinn 3y ago
> I remember a Jon Chesterfield pointing out to me that people mistake inlining to be solely about call-overhead and not specialization I recall multiple mentions over the years that inlining is the most important optimisation because of how many further optimisations it unlocks. I guess having some experience with Rust helps there, as the inlining itself is only a small fraction of all the work required to achieve "zero-cost abstraction". It’s very much load-bearing, but it’s the start if the work, not the end.
- mhh__ 3y ago
  Not particularly unique to rust
  - chlorion 3y ago
    I don't think many languages other than rust and c++ use monomorphization extremely heavily. Pretty much everything in rust is going to be using traits and c++ uses templates for most things as well. Every time you use a trait or a template, the code is expanded for the specific instance, and this gives a lot of opportunity for inlining stuff. A lot of other languages would not have the same opportunity to inline code when doing polymorphism stuff because they rely on dynamic dispatch or something similar that can't be inlined very well or at all. So I'm not disagreeing with you at all but I think rust and c++ get to inline stuff a little more aggressively, but this is just my theory and may be incorrect!
JonChesterfield 3y ago
> I think there's a deeper problem in that the program can't easily tell the compiler what to do Yes. We have divisions - application, language library, language implementation - which are broadly convenient but drop information on the boundaries. The implementation probably pattern matches on the library and the library will probably depends on quirks of the implementation (e.g. compiler intrinsics written for it), but the application is unlikely to be be able to extend the implementation. In cases where it can, there's a tendency to call it monkey patching and disapprove. > people mistake inlining to be solely about call-overhead and not specialization Yeah, that sounds like me. Inlining is an approximation to per-callsite specialisation which is easier to implement, where you pay for ease of compiler development with (machine) code duplication. attributes noinline and alwaysinline are relatively likely to exist, but specialise with respect to the third parameter is still on a wishlist. This is in the ballpark of compilers could be better, and should be better, than the current state of the art.

tedunangst 3y ago

The year of the sufficiently smart compiler is always just over the horizon.

saagarjha 3y ago

Well, at some point you need to deal with a compiler that isn’t fixed yet.