7 ms·
This is one of those things that is completely lost on someone who has never written in a low level language. I automatically assume JavaScript developers to b
by naikrovek 7y ago
This is one of those things that is completely lost on someone who has never written in a low level language. I automatically assume JavaScript developers to be completely oblivious to this entire class of software development knowledge.
It is important to understand your platform all the way down to the CPU, including things like branch prediction and caches if you want to have performant software.
Software has been getting slower more rapidly than hardware has been getting faster for nearly a decade, and the free performance gains that software people have taken advantage of when better hardware is made available are just about exhausted.
It's time to learn your platforms, software people.
[later addition] This kind of thing is also one of the reasons that teaching OOP principles as they are taught today is so bad for software performance. Modeling object relationships to match the real world will, in every non-toy program, produce object structures that are actively unfriendly to cache efficiency, and will therefore produce software which performs very poorly when compared to software that was written with the hardware platform in mind.
- gildas 7y ago> I automatically assume JavaScript developers to be completely oblivious to this entire class of software development knowledge. Surprisingly, In JavaScript the technique described in the article is quite efficient on Firefox whereas the gain is almost negligible on Chrome. https://jsperf.com/mispredicted-branches https://jsperf.com/mispredicted-branches Edit: Jsperf seeems to be down. Here are the 2 snippets of code I tested: // Unoptimized let howmany = 5000000; let index = 0; const out = []; while (howmany) { const val = Math.floor(Number.MAX_SAFE_INTEGER * Math.random()); if (val % 2) { out[index] = val; index += 1; } howmany--; } // Optimized for branch prediction let howmany = 5000000; let index = 0; const out = []; while (howmany) { const val = Math.floor(Number.MAX_SAFE_INTEGER * Math.random()); out[index] = val; index += (val & 1); howmany--; }
- naikrovek 7y agoYeah, of course performance gains can be had in JavaScript, and I will add to that fact with the experience that I've had when working with JavaScript developers: only one that I've ever known has ever considered performance and worked to produce performant JS before multiple users complained and an issue was raised. All others simply do not care about Javascript performance until there is an issue created to address it. This single person is also the only JS developer I know who has any clue how to actually improve performance in existing code.
- littlestymaar 7y agoThis matches my experience as well (as a former full-time JavaScript dev), but in most cases, where JavaScript devs don't care about performance, you don't even need to understand how the machine works to speed things up. 90% of the time, the algorithmic complexity isn't thought at all and there are plenty of quadratic (or worse) behaviors everywhere.
- djokkataja 7y agoI tried both of those out in the console in firefox and chrome and found that in chrome the optimized version was about 10% faster, but in firefox the unoptimized version was about 5% faster. Also it ran 30x faster in chrome... (~200ms vs ~6000ms)
- meheleventyone 7y ago> This is one of those things that is completely lost on someone who has never written in a low level language. I automatically assume JavaScript developers to be completely oblivious to this entire class of software development knowledge. This is a shame because not only are there lots of developers who write JS that have low-level backgrounds there are also a lot who haven't and are still interested. It seems rather snobby to jump in and write off a bunch of people because of the language they use on the assumption that they use the language without being aware of the implications. On OOP versus more cache friendly approaches and branch prediction this talk from CPPCon 2019 is great: https://www.youtube.com/watch?v=HG6c4Kwbv4I https://www.youtube.com/watch?v=HG6c4Kwbv4I The interesting result is that although the DoD approach is eventually faster a branch misprediction causes havoc until uncovered.
- naikrovek 7y agoI realize that it is unfair to categorize ALL JS devs in this way, and it certainly is a tight fit for the JS developers that I have worked with in the past.
- ScottFree 7y agoOT: Why did you feel the need to add this disclaimer? Isn't it assumed that there are always exceptions anytime somebody makes a statement on the macro level? I don't think anybody would mistake "JS devs" for "Every single last individual JS dev".
- meheleventyone 7y agoI don’t think that is a general assumption. Particularly when the generality is made not as a statement of fact but of opinion and in a disparaging manner.
- ScottFree 7y agoAre you telling me when somebody says "Programmers make the web", you can't infer from context that they're not talking about all programmers?
- pjmlp 7y agoAnd when you outsmart the compiler, the CPU gets a microcode update and those clever micro-optimizations are thrown out of the window.
- minipci1321 7y agoYour logic would also apply to the compiler back-end to the same measure. Should it emit optimized code then?
- pjmlp 7y agoYep, it is easier to update the compiler than doing manual clever tricks, specially if the compiler happens to be an AOT/JIT with PGO feedback loop. Most people aren't able to outsmart their compiler optimizers.
- minipci1321 7y ago(This missed my question. Once the compiler is updated to the uArch change, we don't rebuild all binaries out there that were built by the old versions, and their speed is possibly compromised by the uArch change. Should we then even care about emitting binary code optimized for a given uArch, if we can guarantee it is stable in time?)
- pjmlp 7y agoDepends on the use case. One reason mainframes still use bytecode as executable format is that it allows for AOT compiling the world after and OS or hardware upgrade. And there are solutions for doing that to plain native opcodes as well. Finally there is the whole issue that many micro-opmizations, while fun to implement, seldom contribute for visible optimizations of the business case at hand.
- planteen 7y agoThe article is C++, so not sure why you are bringing up AOT/JIT since it doesn't apply here. And clearly there is a large gain possible by doing a calculation rather than a conditional with a high entropy result, otherwise, the article's mentioned speedup wouldn't have happened. A PGO feedback loop isn't going to do anything on a high entropy conditional. I saw a great talk at CppCon 2019 by Andrei Alexandrescu about speeding up std::sort. He does this same trick among others to speed up the C++ std library sort routine. https://www.youtube.com/watch?v=FJJTYQYB1JQ https://www.youtube.com/watch?v=FJJTYQYB1JQ
- vsareto 7y ago>It is important to understand your platform all the way down to the CPU, including things like branch prediction and caches if you want to have performant software. Let's not drastically increase job requirements for no good reason. >It's time to learn your platforms, software people. Many of these platforms have undocumented CPU instructions, so until you get a full accounting of that, what's the point? You can't learn the platform fully if they keep that a secret. Secondly, we've had CPU-level issues like spectre and meltdown introduced that affected performance in some cases. We can't even trust the platform makers to get it right!
- naikrovek 7y ago> Let's not drastically increase job requirements for no good reason. Well, I would say that it's a very good reason, and that learning about branch prediction and caches is not a "drastic" step by any means. Is there any software that you write whose users would not be made happier if the software performed better? Any at all? > Many of these platforms have undocumented CPU instructions You don't need to know the secrets of a platform to understand branch prediction and caches, or to use that knowledge to produce software that performs far better than software written without that knowledge. You don't need to know the platform on a logic gate level, you need to understand how the platform executes your code, so that you can take advantage of the strengths of the platform. You don't need to know any hidden instructions or secrets to take advantage of the platform. > we've had CPU-level issues like spectre and meltdown introduced that affected performance in some cases those things didn't affect performance, the fixes for those things did. The fixes also required no code changes outside of the firmware and the operating system, and knowing the platform is still the best way to write performant software, no matter what is going on to avoid hardware vulnerabilities.
- vsareto 7y ago>Is there any software that you write whose users would not be made happier if the software performed better? Any at all? I'd say security is a bigger issue than performance most of the time. And most gains are going to happen within the code itself by, e.g., not writing n^2 when there's a log(n) solution or something similar. Plus we're talking about javascript, and that's likely to be software with network concerns, so your optimizations might be a rounding error compared to performance degradation from slow network connections. >You don't need to know any hidden instructions or secrets to take advantage of the platform. You don't know what they do though. Some of those ops could be more advantageous to performance to use in some cases. You can't fully know the platform if there are secret ops. You can still make optimizations with partial knowledge, but don't pretend to know the platform when the platform manufacturer doesn't tell you everything about it. >The fixes also required no code changes outside of the firmware and the operating system, and knowing the platform is still the best way to write performant software, no matter what is going on to avoid hardware vulnerabilities. Good algorithm knowledge and practice is the most cost-effective way of writing performant code and is more than likely going to be the lion's share of issues.
- hamilyon2 7y agoWhat good software modeling techniques could one learn instead of OOP?
- pjmlp 7y agoLike Wirth puts it, algorithms + data structures. Programming paradigms are orthogonal to that.
- naikrovek 7y agoData Driven Development is the single most impactful way to write performant code, if I had to single out some single thing. CPUs work on data, not a developer's abstractions about the data. Everything from your structs and objects all the way up should be written with the data in mind. Think about what pieces of data are needed at the same time, or in sequence, and not what classes you need to perform a certain action. Structuring your data types to keep pieces of data that will be operated on together, or read together alone can dramatically improve performance. It's difficult for me to explain, and I'm confident once one sees examples that demonstrates the concept well, it will become clear pretty quickly.
- hamilyon2 7y agoI am quite sure that writing performant code for current architectures is great idea. I am sure one can learn that skill. My question is quite orthogonal to that. OOP lets you model your application in a way that somewhat natural. It's principles are well-researched. Resulting code is widely understood. It can be applied across many different domains. I know no competing way to model software that comes even close to that qualities. Maybe I am wrong and data driven approach provides all that.
- DrScientist 7y agoAnd if your platform is a virtual machine or interpreter running across a number of chip archs? Perhaps it would be better to simply have appropriate tests and benchmarks to see what's slow on what platform? Otherwise it's guess work based on incomplete understanding of the many layers below.
- ip26 7y agoGenerically cache and branch predict friendly code is worth most of the benefit. Micro-tuning is small potatoes compared to accidentally wiping the cache on each instruction (e.g. with badly designed objects) or mispredicting a thousand times in a row (e.g. with a poorly designed loop)
- ThrowawayR2 7y ago> "And if your platform is a virtual machine or interpreter running across a number of chip archs?" Outside of microcontrollers, I can't think of a processor in common use that doesn't have branch prediction or caches.
- DrScientist 7y agoTrue though I suspect that that the length of the pipeline and the cost of a cache miss will be quite different between ARM versus Intel. While, I'm a big fan not not writing poorly performing code, as you get death by a 1000 cuts. Is it not better to test rather than educated guess? ie The reason there is alot of poor performing code, is not a lack of understanding at a low level about caches and branch prediction, but a lack of care about performance?
- naikrovek 7y agoEven then, you're not going to be running on a wildly different set of architectures, I'm betting; it will be a finite set, of course, so you can attend to all the common optimizations and give the virtual machine what it needs to optimize the best it can. No one is going to write a single piece of code that runs on both a z80 and a 56-core Xeon, for example.
- amelius 7y agoInstead of peephole optimizing your code, I think more can be gained by ensuring that your code doesn't perform useless computations. For example, a game that recomputes the entire scene every frame, or a UI that recomputes an entire virtual-dom tree after every user action.
- inetknght 7y agoOr a machine learning algorithm which recomputes everything on every iteration
- dspillett 7y ago> It is important to understand your platform all the way down to the CPU, including things like branch prediction and caches if you want to have performant software. I would agree it can be useful to understand the lower levels, for the majority of developers macro optimisations are far far more important. Things like not forcing unnecessary redraws client-side, or excess DOM manipulations in general, caching values and references instead of rederiving them on each iteration of a loop, avoiding excess database hits, not using a naive sorting method, being aware of network latency issues, not properly indexing the database, and so on, will dwarf the effect of micro-optimisations for branch prediction and CPU caching if you get them wrong. > understand your platform all the way down to the CPU Which platform though? Not everyone is a back-end or embedded dev with constrained hardware to support. Are you targetting amd64 or ARM or something else? Any particular generation of CPU? In many cases an optimisation for one will have no effect elsewhere or worse will make others slower. Unless you are doing much tight-loop number crunching, is faffing around at this level really worth your time? Some understanding of cache concepts will help with overall algorithm design but most of what that knowledge will help you with generally (rather than for specific CPUs) is good practise for other reasons too. > JavaScript developers JS and other JIT compiled languages make tweaks for specific platforms even less important. For the most part let the optimising compiler worry about that for the platform it is compiling for at the time or consider a lower level language. > never written in a low level language I've written in assembly in the distant past (6502 & related, Z80, early x86 and a little of later x86). I did once used knowledge of cache sizes and population behaviour to optimise a little image processing (having the routine work on blocks of the pixel data that were small enough to remain in cache for longer than they would if not dividing into smaller chunks or if jumping around more randomly) done in inline assembly because Delphi's compiler produced a massively less optimal result if it wasn't. I have enough low-level knowledge to feel safe saying I know that most developers (especially junior devs away from embedded (or other low-resource) systems) don't really need it, at least not in as much detail, in the current multi-platform world. They aren't working on code where the benefit of optimisations enabled by that knowledge aren't dwarfed by many other factors.
- rhinoceraptor 7y agoHow is it not a good thing that you don't need to worry about CPU branch prediction when you're writing Javascript to POST a form to your server? In most cases you don't need to go that deep to fix performance issues. Your SPA is probably slow because your app bundle is 10mb, not because you have an inner loop that is tripping the branch predictor.