10 ms·

When zero cost abstractions aren't zero cost

166 points by harporoeder 5y ago

codetrotter 5y ago
There are many useful and interesting comments in the discussion about this on r/rust. https://old.reddit.com/r/rust/comments/p0ul6b/when_zero_cost_abstractions_arent_zero_cost/ https://old.reddit.com/r/rust/comments/p0ul6b/when_zero_cost...
CountHackulus 5y ago
There are no zero cost abstractions. (According to Chandler Carruth, and he makes a convincing point). https://www.youtube.com/watch?v=rHIkrotSwcc https://www.youtube.com/watch?v=rHIkrotSwcc
- HNisBaizuo 5y ago
  Even if an abstraction appears free, you pay opportunity cost by forming a reliance upon it.
- josephcsible 5y ago
  Most people consider "cost" in "zero-cost abstraction" to refer to runtime cost. That presentation is saying that if you let it refer to other kinds of cost too, then nothing is zero-cost, but to me that just means he picked a bad definition.
  - dkersten 5y ago
    And even then, I've often heard it stated as "zero cost if you don't use it", eg, exceptions have zero runtime cost unless an exception is actually thrown, so, given that they should only be thrown in exceptional cases, that really shouldn't impact your program. (Whether or not its truly zero cost depends, I guess, eg does memory count as cost? What if it makes something not fit in cache?)
    - bigcheesegs 5y ago
      I love that exceptions are the classic case for this, because it's not even true that "zero-cost exceptions" are zero _runtime_ cost on the non exceptional path. The most trivial example is they block vectorization.
      - TeMPOraL 5y ago
        Well, exceptions shouldn't be the classic case here, because - as you say - they're typically not zero cost. Zero-cost abstraction is what you get when your abstraction gets compiled away to nothing. Like properly written value wrappers in C++, or (presumably) newtype in Rust. These things disappear from final assembly.
        jcelerier 5y ago
        The definition of zero-cost abstractions, as introduced in C++, was that "it can't be slower that code than the equivalent hand-written code" (e.g. code not using the abstraction). In that regard, exceptions are interesting as if you're on the happy path (which should be 99.999% of the time - a normal program that uses exceptions as error handling method should not encounter any exception if you do a `catch throw` on an average run of the software), they can cost less than return-value-based error handling (https://nibblestew.blogspot.com/2017/01/measuring-execution-performance-of-c.html https://nibblestew.blogspot.com/2017/01/measuring-execution-...). If you're on a "sad path", though, they will cost more. What is pretty sure is that since compilers learned to put the "sad" path in .cold section, the code size issue has become a 100% non-issue, the "sad" path won't bloat the hot, exception-less path ; in my experience, exceptions are in cases that matter a negative-cost abstraction.
        anon1608 5y ago
        As always with microbenchmarks... struct Error* create_error(const char *msg) { struct Error *e = malloc(sizeof(struct Error)); e->msg = strdup(msg); return e; } Is it measuring exceptions vs return codes, or creation cost of std::runtime_error(const char *) (small string opt?) vs malloc() + strdup?
- PragmaticPulp 5y ago
  That’s a disingenuous misinterpretation of what “zero-cost” means. Zero-cost refers to runtime performance of compiled code. It’s also disingenuous to pretend there isn’t some downside to not using the abstraction. Obviously you need to evaluate the tradeoffs. High level code itself is a sort of abstraction. We could write raw assembly all the time to remove that abstraction and associated costs, but clearly that’s not very productive.
- quotemstr 5y ago
  Chandler is simply wrong here. Every single line of code has a cost. There's nevertheless massive benefit to that cost not occuring on millions of end user machines and instead only once in the mind of the developer.
hsn915 5y ago
This is .. amazing? Using what essentially should amount to a strong typedef .. causes code to be millions of times slower. This is worth bookmarking.
- readams 5y ago
  I think it's more accurate to say that there's an optimization that applies only to a very specific situation that's millions of times faster than without the optimization. It's a bit strange to say that before the u8 specialization this was not a problem and it's suddenly a problem now.
  - ixtli 5y ago
    Very well said. I actually came here to write a comment criticizing the post because the fact that this was “negative” has to do with the order in which the experiment took place. Flip it and you have a different sentiment. A similar point can be made but an explanation of “it’s very slow” presumes an assumption that every statement in rust will be maximally optimized via specialization which I consider to be an unreasonable expectation.
    - throwaway17_17 5y ago
      I’m not arguing against you, but why do you feel like expecting maximal optimization is unreasonable? Is it just when limited to specialization or do you think it applies more broadly in terms of general optimization?
  - hsn915 5y ago
    I expect the special case handling for u8 to also apply to types that are just a strong typedef around u8.
    - bilkow 5y ago
      Rust has "typedefs" and those are called type aliases. The intention of the newtype pattern is to specifically create a new type without the behaviour of the wrapped type, so if there's a special case just for u8, I would expect it not to be on the newtype.
erk__ 5y ago
Some months ago I ran into the same problem in slightly different clothing // 1st-way vec![0; len]; // 2nd-way std::iter::repeat(0).take(len).collect::<Vec<_>>(); As told about in the blog post the first one will be able to be optimized via specializations to a single calloc call. The other here cannot use the same specialization as it does not seem to be able to specialize on the type of iterator yet. This means it will be a malloc followed by a memset when compiled. godbolt: https://godbolt.org/z/8bM7zz4E7 https://godbolt.org/z/8bM7zz4E7 the specific specialization: https://github.com/rust-lang/rust/blob/master/library/alloc/src/vec/spec_from_elem.rs#L35..L48 https://github.com/rust-lang/rust/blob/master/library/alloc/...
- erk__ 5y ago
  Doing some further research you can make the optimized initialization of the WrappedByte vector by using it as a iterator. vec![0u8; len] .into_iter() .map(|x| WrappedByte(x)) .collect::<Vec<_>>() which will give you a list of WrappedBytes but initialized with calloc. I assume this is because llvm can see that the map is a identity function in this case and then can optimize it out. Godbolt: https://godbolt.org/z/hxnnPcjvG https://godbolt.org/z/hxnnPcjvG
  - topspin 5y ago
    That we might expect compilers to boil this statement down to calloc(n, s) is astonishing. The subtilty compilers is amazing.
    - tialaramex 5y ago
      If you write some of the "clever" pop count algorithms in C or C++, good compilers will go "Oh you're doing bit counting" and on a modern platform with a bit counting CPU instruction your whole algorithm turns into one CPU instruction. [In Rust the standard library provides count_ones() on integer types, and that's actually safely wrapping an intrinsic pop count feature]
      - 5y ago
stormbrew 5y ago
I don't really feel like this is an issue of "zero cost abstractions". The baseline expectation should be that it will do something reasonable, which in this case is probably `if Copy do bit-setting with memset/bzero if possible, otherwise clone() loop` which is what it appears to do according to a reddit comment linked to elsewhere. I'm not really sure it follows, though, that you should ever expect more than that. It's nice if you can get it but the presence of a very specific optimization shouldn't be taken as the new "zero cost" baseline, and I don't think it should be taken as given that a newtype will inherit all possible optimizations of its enclosed type.
- erk__ 5y ago
  I think a proper solution would be to expose the trait `IsZero` [0] as that is used to figure out when it is correct to use `calloc` instead of having to do `malloc` -> `memset` [1]. I don't know if there is any plans for exposing that trait, but it would be a nice thing to see. [0]: https://github.com/rust-lang/rust/blob/master/library/alloc/src/vec/is_zero.rs https://github.com/rust-lang/rust/blob/master/library/alloc/... [1]: https://github.com/rust-lang/rust/blob/master/library/alloc/src/vec/spec_from_elem.rs#L50..L60 https://github.com/rust-lang/rust/blob/master/library/alloc/...
  - conradludgate 5y ago
    I think you don't need an IsZero trait. It needs to know if the bytes are zero. A custom implementation of IsZero might not hold that guarantee. However, if the type is Copy and all the bytes are zeros, then you know it can be calloc
    - Measter 5y ago
      You mean by checking the bytes of the data directly kinda like in this godbolt example: https://godbolt.org/z/78cvfshva https://godbolt.org/z/78cvfshva
    - tialaramex 5y ago
      (As you likely know) Normally in Rust the trait is a gentleman's agreement. If my type does not have total Equivalence then I don't implement the Eq trait, and so if a type has Eq then it's promising to deliver total Equivalence. You can, in fact, just defy this and make types that do something "completely wrong" while implementing the syntax of the Trait and I provide a suite of them so that other people don't need to do that experiment: https://crates.io/crates/misfortunate https://crates.io/crates/misfortunate However, Rust does have two other tricks up its sleeve: 1. Unsafe traits. You can declare a trait to be "unsafe". Implementations of this trait likewise need that "unsafe" keyword and it alerts the programmer that, as with unsafe function calls, they are responsible for actually getting the implementation details correct. Not many standard library traits are unsafe because that's a considerable burden, but a few are, including Send and Sync. You wouldn't learn much by implementing unsafe traits wrongly in a library like misfortunate, it's the same lesson as a C++ library that scribbles on random memory addresses. 2. Rust knows whether you implemented a trait by hand or merely derived it. In a few cases it can be reasonable for the language to distinguish these cases for built-in traits, since in the former case it knows the derived trait does what is needed even though a manual implementation may not. It is possible that future improvements to Const Generics will rely on this latter idea. So long as you derive Eq rather than implementing it, Rust can reason that your type really is suitable as a constant type parameter whereas misfortunate::Always is not suitable (it would cause mayhem if permitted because of its idea of what "equality" is) despite claiming to be Eq.
yyyk 5y ago
The 'zero cost' attitude is one of C++ problems. Zero cost abstractions are never really entirely zero cost. You still pay, often in something that you didn't bother to measure, which (in C++) is usually compile time or programmer productivity or programmer cognitive cost or compiler complexity. There's a reason a lot of the complaints about C++ mention bloat or the difficulty of practically choosing a safe subset (you could use a subset, but you're very likely dependent on coworkers, existing codebases and library authors which may not share your subset). A lot of people added 'zero cost' stuff, and all of that has a cost. For example, lets say Rust's newtype pattern worked perfectly everywhere, and all of the author's examples run fine as u8 with no runtime cost. There would likely still be a small cognitive cost to learn this, a small cost to unwrap the types (for other programmers reading the code), and a tiny cost in compile time. Typedefs are worth it, and it's all very reasonable to pay this! But there's still a cost, and when people never believe there's a cost a language ends up like C++.
- 10000truths 5y ago
  'Zero cost' is generally meant to mean 'zero runtime cost'. For most workloads, an increase in compile times is okay for an optimized production build. In the rare cases where that's not enough, I find that using precompiled headers and splitting code into translation units helps significantly.
  - yyyk 5y ago
    Yet many people never look at other costs, so de facto they're assuming 'insignificant costs for everything' which isn't true. For example, C++'s template language as originally implemented was technically 'zero cost', but C++ programmers paid for it a lot for a long time, in inscrutable error messages and slow compile times (this was fixed to a large degree with modern implementations and standards).
    - jjoonathan 5y ago
      People didn't understand that they were paying in programmer / compile time / bad error messages? I don't believe that for a second. Those costs are extremely visible. As visible as it is possible to be. They wouldn't get more visible if you dressed them in reflective jackets, slapped a police light bar on top, and turned on a siren to make sure everyone was looking in the right direction. Who undergoes that kind of suffering and doesn't even notice? Nobody. In contrast, the benefits of Zero Cost Abstraction are quite subtle. "Why would I want that? Ever heard of Moore's law? Caring about perf is so 1990s!" goes the immediate thinking. If you never have to write high-perf code, that reasoning is even correct! Of course, there are still many places where performance does matter, and being able to use high level language features on the very innermost loops, the places that halve or quarter the throughput of your $6000 graphics card(s) if you carelessly toss in even a single call of overhead, is quite something to those in a position to take advantage. Since the caveats of ZCA are obvious and the benefits are subtle, I think it's perfectly fair to use the term as a way to draw attention to the latter.
hitekker 5y ago
> The point of this post is not to bash the Rust team, but rather to raise awareness. Language design is a difficult process full of contradictory tradeoffs. Without a clear vision of which properties you value above all else, it is easy to accidentally violate the guarantees you think you are making. This is especially true for complex systems languages like C++ and Rust which try to be all things to all people and leave no stone of potential optimization unturned. This paragraph is worth highlighting. Say yes to everything and vision becomes meaningless. Can't move in one direction when you're pulled by an infinite number of stakeholders on all sides.
- throwaway17_17 5y ago
  This gets a strong second form me. I don’t particularly care for Rust, but that is personal preference. This point stands strong for a wide swath of languages (both compiled and interpreted). There is a seemingly endless pull to be all the things, to all the people for a lot of languages. I would much prefer seeing a smaller set of highly focused languages with literally seamless interop between them.
  - deleted 5y ago
    [deleted]
worik 5y ago
Arguing against zero cost abstractions should use as a counter argument an abstraction that is advertised as zero cost. I do not think wrapped types are so advertised.
- quotemstr 5y ago
  Wrapper types in C++ really are zero cost. Rust is simply broken here, sorry. That happens when you throw away 30 years of work to reinvent the wheel because you like variable names before types or something.
  - adwn 5y ago
    > That happens when you throw away 30 years of work to reinvent the wheel because you like variable names before types or something. That's an extremely ignorant and incorrect take on Rust, and your phrasing makes me believe that you're doing this intentionally.
gridspy 5y ago
The article is not about what you think it's about. 1. Turns out that zeroing u8 is a special case so if you allocate 17 GB of your custom structure Rust is gonna init that for you. 2. Okay, so you don't understand how RefCell works? The caller to this function should be holding r, not passing it into the function. Of course that will just correctly Panic, so it's not wonderful. But that is the point of RefCell. Generally more stuff runs fast in Rust than in other languages, but you can always find bad counterexamples in any language.
- celeritascelery 5y ago
  > 2. Okay, so you don't understand how RefCell works? The caller to this function should be holding r, not passing it into the function. Of course that will just correctly Panic, so it's not wonderful. But that is the point of RefCell. That’s not the interesting part. The point is that rust won’t optimize a lifetime inside a struct the same way it will optimize an actual reference. It can’t assume that the lifetime will be valid for the whole function (as it can with ref). This limits the amount of reordering you can do in the optimizer. The example you were looking at was explaining why that is the case.
  - gridspy 5y ago
    Okay, thanks for clarifying that. Ultimately I'm happy for Rust to only be so smart, of course there are going to be limitations. But you are right, the author was making a clear example and my comment doesn't make sense in that context.
  - spoiler 5y ago
    Isn't this (at least in part) addressed by NLLs[1], or am I misunderstanding? [1]: https://github.com/rust-lang/rfcs/blob/master/text/2094-nll.md https://github.com/rust-lang/rfcs/blob/master/text/2094-nll....
deleted 5y ago
[deleted]
brundolf 5y ago
> the desire to optimize one specific case while neglecting the general case This feels like a recurring pattern in Rust's design whether we're talking about performance, what's accepted by the compiler, etc. I'm not sure it's a bad thing exactly, but it leads to a lot of unintuitive footguns. Of course the consequence of these footguns is de-optimization or a compiler error, instead of a runtime exception or a memory error. But nevertheless it can make for a confusing landscape of behavior to navigate. Should they not try and spot-optimize common cases where they can, just for the sake of predictability? No, I don't think so. And so I don't know what the answer is. I just often (and even more so when I was new to it) find myself surprised by Rust's general-rules-that-actually-have-notable-exceptions. Though who knows, maybe it's just not possible to make a language this powerful that doesn't run into this problem.
- brundolf 5y ago
  Clarification: "Should they not try and spot-optimize common cases where they can, just for the sake of predictability? No, I don't think so." I meant they "shouldn't not", as in I don't think it would make sense to do otherwise from what they're doing
- PoignardAzur 5y ago
  > Should they not try and spot-optimize common cases where they can, just for the sake of predictability? No, I don't think so. And so I don't know what the answer is. I just often (and even more so when I was new to it) find myself surprised by Rust's general-rules-that-actually-have-notable-exceptions. Though who knows, maybe it's just not possible to make a language this powerful that doesn't run into this problem. I think the industry as a whole has already accepted "try to optimize common cases on a best-effort basis, at the cost of unpredictable performance cliffs in unusual cases" as the default paradigm. From multi-tiers JITs, auto-vectorizing, all the way down to branch prediction, source code is a leaky abstraction. If you don't care about performance, you still get the benefits most of the time. If you do care, then you need to peel back the abstraction and have a good understanding of what the compiler/JIT/CPU does behind the scenes.
WiSaGaN 5y ago
According to this comment: https://old.reddit.com/r/rust/comments/p0ul6b/when_zero_cost_abstractions_arent_zero_cost/h8b7b2w/ https://old.reddit.com/r/rust/comments/p0ul6b/when_zero_cost... , `rustc`/`llvm` is indeed able to optimize the wrapped example to a big `memset`. Sure in this specific case, it is different from the first one which delayed the zeroing, but that is not unintelligently `clone` many times as claimed in the article.
nathias 5y ago
There is no zero cost abstractions.
pdimitar 5y ago
What does that post prove except that "allocating memory that must be zeroed on use but is never used" is million times faster than "allocate memory and zero it right away"? Sure, I can see how the code doesn't immediately tell you the difference but after you understood it, isn't it completely logical? The title seems clickbaity and not accurate when you dig into the article.
- Hackbraten 5y ago
  Users who write Rust code don’t necessarily know the underlying optimization (nor should anyone expect that they do). Do you think the title is still inaccurate even from that point of view?
  - pdimitar 5y ago
    The entire article could basically be boiled down to your first sentence + the two code examples. And IMO the title is still inaccurate because it attacks a general promised/desired property of Rust (and other languages) and I honestly don't see the connection. Looks like the observed behaviour is an unfortunate side effect of the `newtype` thing and is very prone to fixing and optimization in future versions. It's not a failure of the "zero-cost abstraction" ideal which is being chased after every day and is still in flux.
  - YawningAngel 5y ago
    > nor should anyone expect that they do I'm not a systems programmer, so I might be wildly off base in what I'm about to say. However, isn't it in fact expected of systems programmers that they do understand in detail what the compiler will do with their code?
  - markenqualitaet 5y ago
    As a Rust beginner, I would never assume code I write to not be executed, or cut out by the compiler. I know this sometimes happens, but I wouldn't count on it for lack of understanding. I don't think that's anyone's expectation who doesn't know the inner workings well, really.
- 5y ago
the8472 5y ago
> However, when v has an arbitrary type, it will instead just clone() it for every single element of the array, which is many orders of magnitude slower. So much for zero cost abstraction. Arguably the WrapperType, i.e. any arbitrary type is the general case and thus the baseline performance. `u8` is the special case (hence specialization) that performs the alloc_zeroed optimization. So it's not really that the abstraction adds a cost. It's just that the special case removes a cost paid by everything else. In the future the vec initialization might be fixable, but this requires turning potentially undefined values into some valid if arbitrary bit patterns (i.e. llvm's freeze) to compare it against zero.
jasonhansel 5y ago
Personally, this is why I think specialization is a bad idea. It means that minor refactors can have very strange impacts on performance, and that every API now has an (often undocumented) set of types for which it is unexpectedly much more performant.
- the8472 5y ago
  You already get this with optimizing compilers. They do not guarantee to optimize all equivalent patterns equally.
- still_grokking 5y ago
  What would be the alternative? Not specializing at all would make your code awful slow (as you would need some quite complicated and costly treatment of generics at runtime). "Specializing by hand" is not realistically doable besides toy programs. And not having generics in the language at all will result in "solutions" like in Go… I see no way around (semi-)automatic specialization done by the compiler.

166 points by harporoeder 5y ago

There are many useful and interesting comments in the discussion about this on r/rust. https://old.reddit.com/r/rust/comments/p0ul6b/when_zero_cost_abstractions_arent_zero_cost/ https://old.reddit.com/r/rust/comments/p0ul6b/when_zero_cost...

CountHackulus 5y ago

There are no zero cost abstractions. (According to Chandler Carruth, and he makes a convincing point). https://www.youtube.com/watch?v=rHIkrotSwcc https://www.youtube.com/watch?v=rHIkrotSwcc

HNisBaizuo 5y ago
Even if an abstraction appears free, you pay opportunity cost by forming a reliance upon it.
josephcsible 5y ago
Most people consider "cost" in "zero-cost abstraction" to refer to runtime cost. That presentation is saying that if you let it refer to other kinds of cost too, then nothing is zero-cost, but to me that just means he picked a bad definition.
- dkersten 5y ago
  And even then, I've often heard it stated as "zero cost if you don't use it", eg, exceptions have zero runtime cost unless an exception is actually thrown, so, given that they should only be thrown in exceptional cases, that really shouldn't impact your program. (Whether or not its truly zero cost depends, I guess, eg does memory count as cost? What if it makes something not fit in cache?)
  - bigcheesegs 5y ago
    I love that exceptions are the classic case for this, because it's not even true that "zero-cost exceptions" are zero _runtime_ cost on the non exceptional path. The most trivial example is they block vectorization.
    - TeMPOraL 5y ago
      Well, exceptions shouldn't be the classic case here, because - as you say - they're typically not zero cost. Zero-cost abstraction is what you get when your abstraction gets compiled away to nothing. Like properly written value wrappers in C++, or (presumably) newtype in Rust. These things disappear from final assembly.
      - jcelerier 5y ago
        The definition of zero-cost abstractions, as introduced in C++, was that "it can't be slower that code than the equivalent hand-written code" (e.g. code not using the abstraction). In that regard, exceptions are interesting as if you're on the happy path (which should be 99.999% of the time - a normal program that uses exceptions as error handling method should not encounter any exception if you do a `catch throw` on an average run of the software), they can cost less than return-value-based error handling (https://nibblestew.blogspot.com/2017/01/measuring-execution-performance-of-c.html https://nibblestew.blogspot.com/2017/01/measuring-execution-...). If you're on a "sad path", though, they will cost more. What is pretty sure is that since compilers learned to put the "sad" path in .cold section, the code size issue has become a 100% non-issue, the "sad" path won't bloat the hot, exception-less path ; in my experience, exceptions are in cases that matter a negative-cost abstraction.
        anon1608 5y ago
        As always with microbenchmarks... struct Error* create_error(const char *msg) { struct Error *e = malloc(sizeof(struct Error)); e->msg = strdup(msg); return e; } Is it measuring exceptions vs return codes, or creation cost of std::runtime_error(const char *) (small string opt?) vs malloc() + strdup?
PragmaticPulp 5y ago
That’s a disingenuous misinterpretation of what “zero-cost” means. Zero-cost refers to runtime performance of compiled code. It’s also disingenuous to pretend there isn’t some downside to not using the abstraction. Obviously you need to evaluate the tradeoffs. High level code itself is a sort of abstraction. We could write raw assembly all the time to remove that abstraction and associated costs, but clearly that’s not very productive.
quotemstr 5y ago
Chandler is simply wrong here. Every single line of code has a cost. There's nevertheless massive benefit to that cost not occuring on millions of end user machines and instead only once in the mind of the developer.

hsn915 5y ago

This is .. amazing? Using what essentially should amount to a strong typedef .. causes code to be millions of times slower. This is worth bookmarking.

readams 5y ago
I think it's more accurate to say that there's an optimization that applies only to a very specific situation that's millions of times faster than without the optimization. It's a bit strange to say that before the u8 specialization this was not a problem and it's suddenly a problem now.
- ixtli 5y ago
  Very well said. I actually came here to write a comment criticizing the post because the fact that this was “negative” has to do with the order in which the experiment took place. Flip it and you have a different sentiment. A similar point can be made but an explanation of “it’s very slow” presumes an assumption that every statement in rust will be maximally optimized via specialization which I consider to be an unreasonable expectation.
  - throwaway17_17 5y ago
    I’m not arguing against you, but why do you feel like expecting maximal optimization is unreasonable? Is it just when limited to specialization or do you think it applies more broadly in terms of general optimization?
- hsn915 5y ago
  I expect the special case handling for u8 to also apply to types that are just a strong typedef around u8.
  - bilkow 5y ago
    Rust has "typedefs" and those are called type aliases. The intention of the newtype pattern is to specifically create a new type without the behaviour of the wrapped type, so if there's a special case just for u8, I would expect it not to be on the newtype.

erk__ 5y ago

Some months ago I ran into the same problem in slightly different clothing // 1st-way vec![0; len]; // 2nd-way std::iter::repeat(0).take(len).collect::<Vec<_>>(); As told about in the blog post the first one will be able to be optimized via specializations to a single calloc call. The other here cannot use the same specialization as it does not seem to be able to specialize on the type of iterator yet. This means it will be a malloc followed by a memset when compiled. godbolt: https://godbolt.org/z/8bM7zz4E7 https://godbolt.org/z/8bM7zz4E7 the specific specialization: https://github.com/rust-lang/rust/blob/master/library/alloc/src/vec/spec_from_elem.rs#L35..L48 https://github.com/rust-lang/rust/blob/master/library/alloc/...

erk__ 5y ago
Doing some further research you can make the optimized initialization of the WrappedByte vector by using it as a iterator. vec![0u8; len] .into_iter() .map(|x| WrappedByte(x)) .collect::<Vec<_>>() which will give you a list of WrappedBytes but initialized with calloc. I assume this is because llvm can see that the map is a identity function in this case and then can optimize it out. Godbolt: https://godbolt.org/z/hxnnPcjvG https://godbolt.org/z/hxnnPcjvG
- topspin 5y ago
  That we might expect compilers to boil this statement down to calloc(n, s) is astonishing. The subtilty compilers is amazing.
  - tialaramex 5y ago
    If you write some of the "clever" pop count algorithms in C or C++, good compilers will go "Oh you're doing bit counting" and on a modern platform with a bit counting CPU instruction your whole algorithm turns into one CPU instruction. [In Rust the standard library provides count_ones() on integer types, and that's actually safely wrapping an intrinsic pop count feature]
    - 5y ago

stormbrew 5y ago

I don't really feel like this is an issue of "zero cost abstractions". The baseline expectation should be that it will do something reasonable, which in this case is probably `if Copy do bit-setting with memset/bzero if possible, otherwise clone() loop` which is what it appears to do according to a reddit comment linked to elsewhere. I'm not really sure it follows, though, that you should ever expect more than that. It's nice if you can get it but the presence of a very specific optimization shouldn't be taken as the new "zero cost" baseline, and I don't think it should be taken as given that a newtype will inherit all possible optimizations of its enclosed type.

erk__ 5y ago
I think a proper solution would be to expose the trait `IsZero` [0] as that is used to figure out when it is correct to use `calloc` instead of having to do `malloc` -> `memset` [1]. I don't know if there is any plans for exposing that trait, but it would be a nice thing to see. [0]: https://github.com/rust-lang/rust/blob/master/library/alloc/src/vec/is_zero.rs https://github.com/rust-lang/rust/blob/master/library/alloc/... [1]: https://github.com/rust-lang/rust/blob/master/library/alloc/src/vec/spec_from_elem.rs#L50..L60 https://github.com/rust-lang/rust/blob/master/library/alloc/...
- conradludgate 5y ago
  I think you don't need an IsZero trait. It needs to know if the bytes are zero. A custom implementation of IsZero might not hold that guarantee. However, if the type is Copy and all the bytes are zeros, then you know it can be calloc
  - Measter 5y ago
    You mean by checking the bytes of the data directly kinda like in this godbolt example: https://godbolt.org/z/78cvfshva https://godbolt.org/z/78cvfshva
  - tialaramex 5y ago
    (As you likely know) Normally in Rust the trait is a gentleman's agreement. If my type does not have total Equivalence then I don't implement the Eq trait, and so if a type has Eq then it's promising to deliver total Equivalence. You can, in fact, just defy this and make types that do something "completely wrong" while implementing the syntax of the Trait and I provide a suite of them so that other people don't need to do that experiment: https://crates.io/crates/misfortunate https://crates.io/crates/misfortunate However, Rust does have two other tricks up its sleeve: 1. Unsafe traits. You can declare a trait to be "unsafe". Implementations of this trait likewise need that "unsafe" keyword and it alerts the programmer that, as with unsafe function calls, they are responsible for actually getting the implementation details correct. Not many standard library traits are unsafe because that's a considerable burden, but a few are, including Send and Sync. You wouldn't learn much by implementing unsafe traits wrongly in a library like misfortunate, it's the same lesson as a C++ library that scribbles on random memory addresses. 2. Rust knows whether you implemented a trait by hand or merely derived it. In a few cases it can be reasonable for the language to distinguish these cases for built-in traits, since in the former case it knows the derived trait does what is needed even though a manual implementation may not. It is possible that future improvements to Const Generics will rely on this latter idea. So long as you derive Eq rather than implementing it, Rust can reason that your type really is suitable as a constant type parameter whereas misfortunate::Always is not suitable (it would cause mayhem if permitted because of its idea of what "equality" is) despite claiming to be Eq.

yyyk 5y ago

The 'zero cost' attitude is one of C++ problems. Zero cost abstractions are never really entirely zero cost. You still pay, often in something that you didn't bother to measure, which (in C++) is usually compile time or programmer productivity or programmer cognitive cost or compiler complexity. There's a reason a lot of the complaints about C++ mention bloat or the difficulty of practically choosing a safe subset (you could use a subset, but you're very likely dependent on coworkers, existing codebases and library authors which may not share your subset). A lot of people added 'zero cost' stuff, and all of that has a cost. For example, lets say Rust's newtype pattern worked perfectly everywhere, and all of the author's examples run fine as u8 with no runtime cost. There would likely still be a small cognitive cost to learn this, a small cost to unwrap the types (for other programmers reading the code), and a tiny cost in compile time. Typedefs are worth it, and it's all very reasonable to pay this! But there's still a cost, and when people never believe there's a cost a language ends up like C++.

10000truths 5y ago
'Zero cost' is generally meant to mean 'zero runtime cost'. For most workloads, an increase in compile times is okay for an optimized production build. In the rare cases where that's not enough, I find that using precompiled headers and splitting code into translation units helps significantly.
- yyyk 5y ago
  Yet many people never look at other costs, so de facto they're assuming 'insignificant costs for everything' which isn't true. For example, C++'s template language as originally implemented was technically 'zero cost', but C++ programmers paid for it a lot for a long time, in inscrutable error messages and slow compile times (this was fixed to a large degree with modern implementations and standards).
  - jjoonathan 5y ago
    People didn't understand that they were paying in programmer / compile time / bad error messages? I don't believe that for a second. Those costs are extremely visible. As visible as it is possible to be. They wouldn't get more visible if you dressed them in reflective jackets, slapped a police light bar on top, and turned on a siren to make sure everyone was looking in the right direction. Who undergoes that kind of suffering and doesn't even notice? Nobody. In contrast, the benefits of Zero Cost Abstraction are quite subtle. "Why would I want that? Ever heard of Moore's law? Caring about perf is so 1990s!" goes the immediate thinking. If you never have to write high-perf code, that reasoning is even correct! Of course, there are still many places where performance does matter, and being able to use high level language features on the very innermost loops, the places that halve or quarter the throughput of your $6000 graphics card(s) if you carelessly toss in even a single call of overhead, is quite something to those in a position to take advantage. Since the caveats of ZCA are obvious and the benefits are subtle, I think it's perfectly fair to use the term as a way to draw attention to the latter.

hitekker 5y ago

> The point of this post is not to bash the Rust team, but rather to raise awareness. Language design is a difficult process full of contradictory tradeoffs. Without a clear vision of which properties you value above all else, it is easy to accidentally violate the guarantees you think you are making. This is especially true for complex systems languages like C++ and Rust which try to be all things to all people and leave no stone of potential optimization unturned. This paragraph is worth highlighting. Say yes to everything and vision becomes meaningless. Can't move in one direction when you're pulled by an infinite number of stakeholders on all sides.

throwaway17_17 5y ago
This gets a strong second form me. I don’t particularly care for Rust, but that is personal preference. This point stands strong for a wide swath of languages (both compiled and interpreted). There is a seemingly endless pull to be all the things, to all the people for a lot of languages. I would much prefer seeing a smaller set of highly focused languages with literally seamless interop between them.
- deleted 5y ago
  [deleted]

worik 5y ago

Arguing against zero cost abstractions should use as a counter argument an abstraction that is advertised as zero cost. I do not think wrapped types are so advertised.

quotemstr 5y ago
Wrapper types in C++ really are zero cost. Rust is simply broken here, sorry. That happens when you throw away 30 years of work to reinvent the wheel because you like variable names before types or something.
- adwn 5y ago
  > That happens when you throw away 30 years of work to reinvent the wheel because you like variable names before types or something. That's an extremely ignorant and incorrect take on Rust, and your phrasing makes me believe that you're doing this intentionally.

gridspy 5y ago

The article is not about what you think it's about. 1. Turns out that zeroing u8 is a special case so if you allocate 17 GB of your custom structure Rust is gonna init that for you. 2. Okay, so you don't understand how RefCell works? The caller to this function should be holding r, not passing it into the function. Of course that will just correctly Panic, so it's not wonderful. But that is the point of RefCell. Generally more stuff runs fast in Rust than in other languages, but you can always find bad counterexamples in any language.

celeritascelery 5y ago
> 2. Okay, so you don't understand how RefCell works? The caller to this function should be holding r, not passing it into the function. Of course that will just correctly Panic, so it's not wonderful. But that is the point of RefCell. That’s not the interesting part. The point is that rust won’t optimize a lifetime inside a struct the same way it will optimize an actual reference. It can’t assume that the lifetime will be valid for the whole function (as it can with ref). This limits the amount of reordering you can do in the optimizer. The example you were looking at was explaining why that is the case.
- gridspy 5y ago
  Okay, thanks for clarifying that. Ultimately I'm happy for Rust to only be so smart, of course there are going to be limitations. But you are right, the author was making a clear example and my comment doesn't make sense in that context.
- spoiler 5y ago
  Isn't this (at least in part) addressed by NLLs[1], or am I misunderstanding? [1]: https://github.com/rust-lang/rfcs/blob/master/text/2094-nll.md https://github.com/rust-lang/rfcs/blob/master/text/2094-nll....

deleted 5y ago

[deleted]

brundolf 5y ago

> the desire to optimize one specific case while neglecting the general case This feels like a recurring pattern in Rust's design whether we're talking about performance, what's accepted by the compiler, etc. I'm not sure it's a bad thing exactly, but it leads to a lot of unintuitive footguns. Of course the consequence of these footguns is de-optimization or a compiler error, instead of a runtime exception or a memory error. But nevertheless it can make for a confusing landscape of behavior to navigate. Should they not try and spot-optimize common cases where they can, just for the sake of predictability? No, I don't think so. And so I don't know what the answer is. I just often (and even more so when I was new to it) find myself surprised by Rust's general-rules-that-actually-have-notable-exceptions. Though who knows, maybe it's just not possible to make a language this powerful that doesn't run into this problem.

brundolf 5y ago
Clarification: "Should they not try and spot-optimize common cases where they can, just for the sake of predictability? No, I don't think so." I meant they "shouldn't not", as in I don't think it would make sense to do otherwise from what they're doing
PoignardAzur 5y ago
> Should they not try and spot-optimize common cases where they can, just for the sake of predictability? No, I don't think so. And so I don't know what the answer is. I just often (and even more so when I was new to it) find myself surprised by Rust's general-rules-that-actually-have-notable-exceptions. Though who knows, maybe it's just not possible to make a language this powerful that doesn't run into this problem. I think the industry as a whole has already accepted "try to optimize common cases on a best-effort basis, at the cost of unpredictable performance cliffs in unusual cases" as the default paradigm. From multi-tiers JITs, auto-vectorizing, all the way down to branch prediction, source code is a leaky abstraction. If you don't care about performance, you still get the benefits most of the time. If you do care, then you need to peel back the abstraction and have a good understanding of what the compiler/JIT/CPU does behind the scenes.

WiSaGaN 5y ago

According to this comment: https://old.reddit.com/r/rust/comments/p0ul6b/when_zero_cost_abstractions_arent_zero_cost/h8b7b2w/ https://old.reddit.com/r/rust/comments/p0ul6b/when_zero_cost... , `rustc`/`llvm` is indeed able to optimize the wrapped example to a big `memset`. Sure in this specific case, it is different from the first one which delayed the zeroing, but that is not unintelligently `clone` many times as claimed in the article.

nathias 5y ago

There is no zero cost abstractions.

pdimitar 5y ago

What does that post prove except that "allocating memory that must be zeroed on use but is never used" is million times faster than "allocate memory and zero it right away"? Sure, I can see how the code doesn't immediately tell you the difference but after you understood it, isn't it completely logical? The title seems clickbaity and not accurate when you dig into the article.

Hackbraten 5y ago
Users who write Rust code don’t necessarily know the underlying optimization (nor should anyone expect that they do). Do you think the title is still inaccurate even from that point of view?
- pdimitar 5y ago
  The entire article could basically be boiled down to your first sentence + the two code examples. And IMO the title is still inaccurate because it attacks a general promised/desired property of Rust (and other languages) and I honestly don't see the connection. Looks like the observed behaviour is an unfortunate side effect of the `newtype` thing and is very prone to fixing and optimization in future versions. It's not a failure of the "zero-cost abstraction" ideal which is being chased after every day and is still in flux.
- YawningAngel 5y ago
  > nor should anyone expect that they do I'm not a systems programmer, so I might be wildly off base in what I'm about to say. However, isn't it in fact expected of systems programmers that they do understand in detail what the compiler will do with their code?
- markenqualitaet 5y ago
  As a Rust beginner, I would never assume code I write to not be executed, or cut out by the compiler. I know this sometimes happens, but I wouldn't count on it for lack of understanding. I don't think that's anyone's expectation who doesn't know the inner workings well, really.
5y ago

the8472 5y ago

> However, when v has an arbitrary type, it will instead just clone() it for every single element of the array, which is many orders of magnitude slower. So much for zero cost abstraction. Arguably the WrapperType, i.e. any arbitrary type is the general case and thus the baseline performance. `u8` is the special case (hence specialization) that performs the alloc_zeroed optimization. So it's not really that the abstraction adds a cost. It's just that the special case removes a cost paid by everything else. In the future the vec initialization might be fixable, but this requires turning potentially undefined values into some valid if arbitrary bit patterns (i.e. llvm's freeze) to compare it against zero.

jasonhansel 5y ago

Personally, this is why I think specialization is a bad idea. It means that minor refactors can have very strange impacts on performance, and that every API now has an (often undocumented) set of types for which it is unexpectedly much more performant.

the8472 5y ago
You already get this with optimizing compilers. They do not guarantee to optimize all equivalent patterns equally.
still_grokking 5y ago
What would be the alternative? Not specializing at all would make your code awful slow (as you would need some quite complicated and costly treatment of generics at runtime). "Specializing by hand" is not realistically doable besides toy programs. And not having generics in the language at all will result in "solutions" like in Go… I see no way around (semi-)automatic specialization done by the compiler.