3 ms·
Is it true that optimization follows the 80/20 rule? What are some common performance issues that new languages and their implementations face? Are there common
by matheusmoreira 7y ago
Is it true that optimization follows the 80/20 rule? What are some common performance issues that new languages and their implementations face? Are there common optimization techniques that can be applied in order to make the new language competitive with existing ones?
For example, I know that it's generally better to compile programs into a linear code structure such as bytecode instead of interpreting a tree structure directly.
- barrkel 7y agoLinear encoding is better than interpreting a tree because accessing adjacent elements in arrays is faster than chasing pointers. Poor performance depends on what your language does. What's poor performance for SQL will be different than for JS. Features which don't have efficient implementations will be slow. If the language encourages use of inefficient features, then it will generally be slower. The easiest optimization technique is to leverage an existing back end; for example, target LLVM or JVM or .NET.
- matheusmoreira 7y ago> Linear encoding is better than interpreting a tree because accessing adjacent elements in arrays is faster than chasing pointers. Yes. From this we can derive some general optimization principles: reduce indirection, increase data locality. Are there any others? > Poor performance depends on what your language does. What's poor performance for SQL will be different than for JS. Let's consider modern dynamic languages like Javascript and Python. What performance problems did these languages face? Which optimizations had the biggest impact on code execution performance? Which optimizations were the easiest to understand and implement? For example, I know Javascript virtual machines will automatically create hidden classes for objects. This allows the data to be accessed by constant offsets rather than lookups.
- barrkel 7y agoYou're sticking your toe in a very big lake, and this isn't the right forum to meaningfully talk about it. (I don't mean this in any way negatively, but your two posts have a distinct whiff of Help Vampire - https://meta.stackexchange.com/questions/19665/the-help-vampire-problem https://meta.stackexchange.com/questions/19665/the-help-vamp... - just a tiny chunk of information, and a naive question where you'd need years to properly survey the knowledge. It's a big topic and you need to be goal directed to get anywhere IMO.)
- matheusmoreira 7y agoThe author of the article said we could ask anything: https://news.ycombinator.com/item?id=23054664 https://news.ycombinator.com/item?id=23054664 I thought it was a good opportunity to ask the questions that have been in my mind for a long time. They were open-ended questions but I don't think they were incoherent. I've read the source code of certain programming language virtual machines and have seen optimizations both simple and complex. For a long time I've been wondering if there was a set of basic optimizations every developer could apply to their language and get significant improvements with little difficulty. I don't have a project I need help with, I was just curious. I certainly don't think I'm entitled to answers or attention. If I offended anybody, I apologize. My intent was to spawn a discussion. > this isn't the right forum to meaningfully talk about it What is the correct place?
- WalterBright 7y agoOptimizing code is a book-length topic just for an introduction. It's also true that knowing how optimizers work can feed back into improving the language design. For example, `const` in C++ doesn't mean the data is immutable - it can change with any assignment through a pointer. No optimizations assuming immutability will work. That's why D has an `immutable` qualifier, giving the optimizer to do optimizations assuming it does not change. For a famous example, Fortran assumes two arrays never overlap. In C/C++ they can. This is the source of a persistent gap in performance between Fortran and C/C++. C attempted to fix it by adding the `restrict` qualification, but this failed because it is just too arcane and brittle for most users. In D, I'm working on adding an Ownership/Borrowing system, which will enable the compiler to figure out the pointer aliasing optimization opportunities.
- muldvarp 7y agoAn Ownership/Borrowing system like the one Rust uses? Rust is sadly not (yet) able to use its stricter rules about ownership and aliasing for optimization, because the LLVM backend has been shown to have a number of bugs that prevent such optimizations. Sadly, I'm not that knowledgeable on optimization. Do you expect that such optimizations would make most programs significantly faster or is it more of a special case that would not improve the performance of most programs? Keep up the good work with D. I especially love its template/metaprogramming capabilities!
- WalterBright 7y ago> Do you expect that such optimizations would make most programs significantly faster or is it more of a special case that would not improve the performance of most programs? I don't know because I have no personal experience with such. What I have to go on is the Fortran vs C/C++ experience I mentioned.
- matheusmoreira 7y ago> That's why D has an `immutable` qualifier, giving the optimizer to do optimizations assuming it does not change. Which optimizations are enabled by immutable data? I can think of constant folding. Is the data statically allocated in a read-only page? > For a famous example, Fortran assumes two arrays never overlap. In C/C++ they can. This is the source of a persistent gap in performance between Fortran and C/C++. What sort of optimizations does this assumption enable? Does it allow the compiler to freely reorder or parallelize the code? C assumes that pointers to different types are never equal. This is incompatible with common systems programming concepts such as type punning. Even something simple like reinterpreting some data structure as an array of uint8_t can make the optimizer introduce bugs into the code. Notably, the Linux kernel is compiled with strict aliasing disabled: https://lkml.org/lkml/2003/2/26/158 https://lkml.org/lkml/2003/2/26/158 https://lkml.org/lkml/2009/1/12/369 https://lkml.org/lkml/2009/1/12/369
- robocat 7y agoAside: this is the most amazing optimisation information I have seen: https://www.youtube.com/watch?v=r-TLSBdHe1A https://www.youtube.com/watch?v=r-TLSBdHe1A Unfortunate that it is a video... Summary: Modern CPUs have so many hidden causes of performance variation, that you require special tools to actually measure small performance gains. He finds an amazing but unobvious speed up in SQLite using the causal analysis tool they developed. And a surprising difference between -O2 and -O3 implying overfitting.