8 ms·
I don't do much C++, but I have definitely found that engineers will just assert that something is "faster" without any evidence to back that up. Quick example
by tombert 3y ago
I don't do much C++, but I have definitely found that engineers will just assert that something is "faster" without any evidence to back that up.
Quick example, I got in an argument with someone a few years ago that claimed in C# that a `switch` was better than an `if(x==1) elseif(x==2)...` because switch was "faster" and rejected my PR. I mentioned that that doesn't appear to be true, we went back and forth until I did a compile-then-decompile of a minimal test with equality-based-ifs, and showed that the compiler actually converts equality-based-ifs to `switch` behind the scenes. The guy accepted my PR after that.
But there's tons of this stuff like this in CS, and I kind of blame professors for a lot of it [1]. A large part of becoming a decent engineer [2] for me was learning to stop trusting what professors taught me in college. Most of what they said was fine, but you can't assume that; what they tell you could be out of date, or simply never correct to begin with, and as far as I can tell you have to always test these things.
It doesn't help that a lot of these "it's faster" arguments are often reductive because they only are faster in extremely minimal tests. Sometimes a microbenchmark will show that something is faster, and there's value in that, but I think it's important that that can also be a small percentage of the total program; compilers are obscenely good at optimizing nowadays, it can be difficult to determine when something will be optimized, and your assertion that something is "faster" might not actually be true in a non-trivial program.
This is why I don't really like doing any kind of major optimizations before the program actually works. I try to keep the program in a reasonable Big-O and I try and minimize network calls cuz of latency, but I don't bother with any kind of micro-optimizations in the first draft. I don't mess with bitwise, I don't concern myself on which version of a particular data structure is a millisecond faster, I don't focus too much on whether I can get away with a smaller sized float, etc. Once I know that the program is correct, then I benchmark to see if any kind of micro-optimizations will actually matter, and often they really don't.
[1] That includes me up to about a year ago.
[2] At least I like to pretend I am.
- BurningFrog 3y agoEven if one of these constructs is faster it doesn't matter 99% of the time. Writing well structured readable code is typically far more important than making it twice as fast. And those times can rarely be predicted beforehand, so you should mostly not worry about it until you see real performance problems.
- tombert 3y agoI mostly focus on "using stuff that won't break", and yeah "if it actually matters". For example, much to the annoyance of a lot of people, I don't typically use floating point numbers when I start out. I will use the "decimal" or "money" types of the language, or GMP if I'm using C. When I do that, I can be sure that I won't have to worry about any kind of funky overflow issues or bizarre rounding problems. There might be a performance overhead associated with it, but then I have to ask myself "how often is this actually called?" If the answer is "a billion times" or "once in every iteration of the event loop" or something, then I will probably eventually go back and figure out if I can use a float or convert it to an integer-based thing, but in a lot of cases the answer is "like ten or twenty times", and at that point I'm not even 100% sure it would be even measurable to change to the "faster" implementations. What annoys me is that people will act like they really care about speed, do all these annoying micro-optimizations, and then forget that pretty much all of them get wiped out immediately upon hitting the network, since the latency associated with that is obscene.
- apantel 3y agoThe counter-argument to this is if you are building something that is in the critical path of an application (for example, parsing HTTP in a web server), you need to be performance-minded from the beginning because design decisions lead to design decisions. If you are building something in the critical path of the application, the best thing to do is build it from the ground up measuring the performance of what you have as you go. This way, each time you add something you will see the performance impact and usually there’s a more performant way of doing something that isn’t more obscure. If you do this as you build, early choices become constraints, but because you chose the most performant thing at every stage, the whole process takes you in the direction of a highly-performant implementation. Why should you care about performance? I can give you my personal experience: I’ve been working on a Java web/application server for the past 15 years and a typical request (only reading, not writing to the db) would take maybe 4-5 ms to execute. That includes HTTP request parsing, JSON parsing, session validation, method execution, JSON serialization, and HTTP response dispatch. Over the past 9 months I have refactored the entire application for performance and a typical request now takes about 0.25 ms or 250 microseconds. The computer is doing so much less work to accomplish the same tasks, it’s almost silly how much work it was doing before. And the result is the machine can handle 20x more requests in the same amount of time. If it could handle 200 requests per second per core before, now it can handle 4000. That means the need to scale is felt 20x less intensely, which means less complexity around scaling. High performance means reduced scaling requirements.
- leetcrew 3y agoagreed, especially in cases like this. final is primarily a way to prohibit overriding methods and extending classes, and it indicates to the reader that they should not be doing this. use it when it makes conceptual sense. that said, c++ is usually a language you use when you care about performance, at least to an extent. it's worth understanding features like nrvo and rewriting functions to allow the compiler to pick the optimization if it doesn't hurt readability too much.
- wvenable 3y agoIn my opinion, the only things that really matter are algorithmic complexity and readability. And even algorithmic complexity is usually only an issue a certain scales. Whether or not an 'if' is faster than a 'switch' is the micro of micro optimizations -- you better have a good reason to care. The question I would have for you is was your bunch of ifs more readable than a switch would be.
- doctor_phil 3y agoBut a switch and an if-else *is* a matter of algorithmic complexity. (Well, at least could be for a naive compiler). A switch could be converted to a constant time jump, but the if-else would be trying each case linearly.
- cogman10 3y agoYup. That said, the linear test is often faster due to CPU caches, which is why JITs will often convert switches to if/elses. IMO, switch is clearer in general and potentially faster (at very least the same speed) so it should be preferred when dealing with 3+ if/elseif statements.
- tombert 3y agoHard disagree that it's "clearer". I have had to deal with a ton of bugs with people trying to be clever with the `break` logic, or forgetting to put `break` in there at all. if statements are dumber, and maybe arguably uglier, but I feel like they're also more clear, and people don't try and be clever with them.
- cogman10 3y agoUpdates to languages (don't know where C# is on this) have different types of switch statements that eliminate the `break` problem. For example, with java there's enhanced switch that looks like this var val = switch(foo) { case 1, 2, 3 -> bar; case 4 -> baz; default -> { yield bat(); } } The C style switch break stuff is definitely a language mistake.
- saghm 3y ago> But there's tons of this stuff like this in CS Reminds me of the classic https://stackoverflow.com/questions/24848359/which-is-faster-while1-or-while2 https://stackoverflow.com/questions/24848359/which-is-faster...
- sgerenser 3y agoNever saw that before, that is indeed a classic.
- jollyllama 3y agoI've encountered similar situations before. It's insane to me when people hold up PRs over that kind of thing.
- dosshell 3y ago> I can get away with a smaller sized float When talking about not assuming optimizations... 32bit float is slower than 64bit float on reasonable modern x86-64. The reason is that 32bit float is emulated by using 64bit. Of course if you have several floats you need to optimize against cache.
- tombert 3y agoSure, I clarified this in a sibling comment, but I kind of meant that I will use the slower "money" or "decimal" types by default. Usually those are more accurate and less error-prone, and then if it actually matters I might go back to a floating point or integer-based solution.
- sgerenser 3y agoI think this is only true if using x87 floating point, which anything computationally intensive is generally avoiding these days in favor of SSE/AVX floats. In the latter case, for a given vector width, the cpu can process twice as many 32 bit floats as 64 bit floats per clock cycle.
- dosshell 3y agoYes, as I wrote, it is only true for one float value. SIMD/MIMD will benefit of working on smaller width. This is not only true because they do more work per clock but because memory is slow. Super slow compared to the cpu. Optimization is alot about cache misses optimization. (But remember that the cache line is 64 bytes, so reading a single value smaller than that will take the same time. So it does not matter in theory when comparing one f32 against one f64)
- jcranmer 3y agoUm... no. This is 100% completely and totally wrong. x86-64 requires the hardware to support SSE2, which has native single-precision and double-precision instructions for floating-point (e.g., scalar multiply is MULSS and MULSD, respectively). Both the single precision and the double precision instructions will take the same time, except for DIVSS/DIVSD, where the 32-bit float version is slightly faster (about 2 cycles latency faster, and reciprocal throughput of 3 versus 5 per Agner's tables). You might be thinking of x87 floating-point units, where all arithmetic is done internally using 80-bit floating-point types. But all x86 chips in like the last 20 years have had SSE units--which are faster anyways. Even in the days when it was the major floating-point units, it wasn't any slower, since all floating-point operations took the same time independent of format. It might be slower if you insisted that code compilation strictly follow IEEE 754 rules, but the solution everybody did was to not do that and that's why things like Java's strictfp or C's FLT_EVAL_METHOD were born. Even in that case, however, 32-bit floats would likely be faster than 64-bit for the simple fact that 32-bit floats can safely be emulated in 80-bit without fear of double rounding but 64-bit floats cannot.
- jandrewrogers 3y agoA significant part of it is that what engineers believe was effectively true at one time. They simply haven't revisited those beliefs or verified their relevance in a long time. It isn't a terrible heuristic for life in general to assume that what worked ten years ago will work today. The rate at which the equilibriums shift due to changes in hardware and software environments when designing for system performance is so rapid that you need to make a continuous habit of checking that your understanding of how the world works maps to reality. I've solved a lot of arguments with godbolt and simple performance tests. Some topics are recurring themes among software engineers e.g.: - compilers are almost always better at micro-optimizations than you are - disk I/O is almost never a bottleneck in competent designs - brute-force sequential scans are often optimal algorithms - memory is best treated as a block device - vectorization can offer large performance gains - etc... No one is immune to this. I am sometimes surprised at the extent to which assumptions are no longer true when I revisit optimization work I did 10+ years ago. Most performance these days is architectural, so getting the initial design right often has a bigger impact than micro-optimizations and localized Big-O tweaks. You can always go back and tweak algorithms or codegen later but architecture is permanent.
- neonsunset 3y ago.NET is a particularly bad case for this because it was a decade of few performance improvements, which caused a certain intuition to develop within the industry, then 6-8 years of significant changes each year (with most wins compressed to the last 4 years or so). Companies moving from .NET Framework 4.6/7/8 to .NET 8 experience a 10x average performance improvement, which naturally comes with rendering a lot of performance know-how obsolete overnight. (the techniques that used to work were similar to earlier Java versions and overall very dynamic languages with some exceptions, the techniques that still work and now are required today are the same as in C++ or Rust)
- throwaway2037 3y ago.NET 4.6 to .NET 8 is a 10x "average" performance improvement. I find this hard to believe. In what scenarios? I tried to Google for it and found very little hard evidence.
- klyrs 3y ago> A large part of becoming a decent engineer [2] for me was learning to stop trusting what professors taught me in college When I was taught about performance, it was all about benchmarking and profiling. I never needed to trust what my professors taught, because they taught me to dig in and find the truth for myself. This was taught alongside the big-O stuff, with several examples where "fast" algorithms are slower on small inputs.
- TylerE 3y agoHow do you even get meaningful profiling out of most modern langs? It seems the vast majority of time and calls gets spent inside tiny anonymous functions, GC allocations, and stuff like that.
- klyrs 3y agoI don't use most modern langs! And especially if I'm doing work where performance is critical, I won't kneecap myself by using a language that I can't reasonably profile.
- neonsunset 3y agoThis is easy in most modern programming languages. JVM ecosystem has IntelliJ Idea profiler and similar advanced tools (AFAIK). .NET has VS/Rider/dotnet-trace profilers (they are very detailed) to produce flamegraphs. Then there are native profilers which can work with any AOT compiled language that produces canonically symbolicated binaries: Rust, C#/F#(AOT mode), Go, Swift, C++, etc. For example, you can do `samply record ./some_binary`[0] and then explore multi-threaded flamegraph once completed (I use it to profile C#, it's more convenient than dotTrace for preliminary perf work and is usually more than sufficient). [0] https://github.com/mstange/samply https://github.com/mstange/samply
- TylerE 3y agoI mean sure, but I've never seen much in a flamegraph besides noise.
- trueismywork 3y agoThere's not yet a culture of writing reproducible benchmarks to gage these effects.
- zmj 3y ago.NET is a little smarter about switch code generation these days: https://github.com/dotnet/roslyn/pull/66081 https://github.com/dotnet/roslyn/pull/66081
- KerrAvon 3y ago> `if(x==1) elseif(x==2)...` because switch was "faster" and rejected my PR Yeah, that's never been true. Old compilers would often compile a switch to __slower__ code because they'd tend to always go to a jump table implementation. A better reason to use the switch is because it's better style in C-like languages. Using an if statement for that sort of thing looks like Python; it makes the code harder to maintain.
- wzdd 3y agoAnd it's better style because it better conveys intent. An if-else chain in C/C++ implies there's something important about the ordering of cases. Though I'd say that for a very small number of cases it's fine. (Also, Python has a switch-like construct now.)
- mynameisnoone 3y agoYep. "Profiling or it didn't happen." The issue is that it's essentially impossible for even the most neckbeard of us to predict with a high degree of accuracy and precision the performance on modern systems impact of change A vs. change B due to the unpredictable nature of the many variables that are difficult to control including compiler optimization passes, architecture gotchas (caches, branch misses), and interplay of quirks on various platforms. Therefore, irreducible and necessary work to profile the differences become the primary viable path to resolving engineering decision points. Hopefully, LLMs now and in the future will be able to help build out boilerplate roughly in the direct of creating such profiling benchmarks and fixtures. PS: I'm presently revisiting C++14 because it's the most universal statically-compiled language to quickly answer interview problems. It would be unfair to impose Rust, Go, Elixir, or Haskell on an interviewer software engineer.
- pjmlp 3y agoI would say it would be safer to go up to C++17, and there are some goodies there, specially for better compile time stuff.
- ot1138 3y ago>I don't do much C++, but I have definitely found that engineers will just assert that something is "faster" without any evidence to back that up. Very true, though there is one case where one can be highly confident that this is the case: code elimination. You can't get any faster than not doing something in the first place.
- konstantinua00 3y agoinb4 instruction (cache) alignment screws everythin up