6 ms·

I'm increasingly coming to realize that in programming, flexibility is something you pay for in performance, whether you use it or not. Data being mutable even

by m12k 5y ago

I'm increasingly coming to realize that in programming, flexibility is something you pay for in performance, whether you use it or not. Data being mutable even if you never mutate it. Automatic bounds checks for data that you've already made sure can't go out of bounds. vtable lookups even when you're not using inheritance, because who knows, someone else might. Compilers needing to do extra work in case two pointers are aliases, even though that never makes sense for them to be from the programmer's perspective. Objects allocated one at a time, scattered around memory, even if you don't need them to be, and wouldn't mind their innards being kept together in cache aligned arrays. They say you can solve any problem in computer science with another level of indirection - except performance, it seems, because we're paying the cost of all this indirection all the time, even if we don't need it most of the time. There's a crazy amount of unnecessary "housekeeping" done by compilers and runtimes to make sure everything always works all the time, which requires them to always do extra work to protect against pathological cases that almost never arise. Modern compilers and runtimes are getting better and better at recognizing the common "happy case" and optimizing for that, and that's good. But it still feels backwards - both that we can't explicitly opt out of unneeded flexibility, so instead the compiler needs to deduce that we aren't using it. But also that a lot of this isn't opt-in to begin with, rather than opt out - i.e. make things immutable by default. Or let the programmer tell the compiler a little bit more about what they need and don't need the code to do, so the compiler can do a much better job if it.

bob1029 5y ago
> Or let the programmer tell the compiler a little bit more about what they need and don't need the code to do, so the compiler can do a much better job if it. You can do this today in languages like C#: https://docs.microsoft.com/en-us/dotnet/csharp/write-safe-efficient-code#declare-readonly-members-for-mutable-structs https://docs.microsoft.com/en-us/dotnet/csharp/write-safe-ef... > Adding the readonly modifier to members that don't mutate state provides two related benefits. First, the compiler enforces your intent. That member can't mutate the struct's state. Second, the compiler won't create defensive copies of in parameters when accessing a readonly member. The compiler can make this optimization safely because it guarantees that the struct is not modified by a readonly member.
- XorNot 5y ago
  This feels like some seriously missed static analysis optimization though - why do I need to specify this? There's vanishingly few cases the answer to "looks like you don't modify this ever, make it readonly?" where the answer is not yes.
  - bob1029 5y ago
    Probably because reflection & compilation+activation of types existing outside the original compilation context is possible at runtime. I much prefer to having a language/runtime that can handle extremely complex problem scenarios out of the box (with the need for occasional tuning), rather than the other way around. It is not very often that I find myself screwing around with performance-intensive code. Usually, a few very important things are addressed at the lowest levels and then you put all your nice abstractions back on top for daily driving.
jiggawatts 5y ago
> flexibility is something you pay for in performance I lightbulb moment I had recently was that the classic interface-based OO programming paradigm utilising virtual functions was essentially the following declaration by the programmer to the compiler: The function that is invoked can change at any time, even call-to-call; assume nothing. it is a black box with unknown, unknowable contents. Do not inline it, do not inspect it, expect the worst. If you look at a typical web framework, with call stacks hundreds of methods deep, you'll come to the same horrifying realisation that these are all inherently un-optimisable because of that statement. The compiler can do nearly nothing, and the processor will also struggle to paper over the inefficiency. Meanwhile, in reality, the specific implementations of those interfaces never change during actual production execution! Dynamic dispatch is utilised only during development time, and even then only at compile time to inject test harnesses, mocks, or whatever. At runtime, 99.999999% of all function calls are essentially static, and dynamic dispatch is a pure waste.
- dzaima 5y ago
  Except that a smart runtime can actually work with the assumption that only one implementation of a class will be used, and inline what would otherwise be virtual methods. And, as far as I know, the JVM definitely does this, and at multiple levels too - if there's only one implementation of an interface, it can just assume that everything's static and inlineable (at the cost of deoptimizations if that turns out to be false later). If not, it can still decide to inline, and just check the type beforehand.
- MaxBarraclough 5y ago
  > The compiler can do nearly nothing, Optimizing JIT-compiling JVMs are often able to optimize virtual calls that only ever resolve to one implementation. They may even inline such calls. (edit dzaima got there first.)
  - duped 5y ago
    It's old too, Self was doing this in the late 80s
mtVessel 5y ago
As much as people like to talk about "footguns", that's the value proposition of C++, zero cost for anything you don't use. Managed/interpreted langs have a different proposition: pay up front for all the features, but your complexity never increases whether you use none, one or all of them.
- MauranKilom 5y ago
  Well, pointer aliasing for example is a very tricky thing in C++ too. Even with strict aliasing, it is very unintuitive from the outside when you're going to trip over unexpected performance hurdles. (Does this data structure have a char* inside? Who knows, but if it does, there goes vectorization!).
  - username90 5y ago
    If you don't dereference the char* it will get treated an an integer and be vectorizable. In C++ you don't pay for what you don't use. If you need the char* in the loop I'm not sure why you'd be thinking about vectorization. Maybe you meant parallelism?
de_keyboard 5y ago
Great talk: Constraints Liberate, Liberties Constrain — Runar Bjarnason https://www.youtube.com/watch?v=GqmsQeSzMdw https://www.youtube.com/watch?v=GqmsQeSzMdw
- adverbly 5y ago
  Wow thanks for linking that. Great talk. I'm always on the lookout for talks about interesting and generally useful concepts, but where I can follow only a small part of them. Makes me feel like I can learn and grow into a wiser person. This certainly fits that criteria.
  - whoisburbansky 5y ago
    This talk fits a similar bucket in my head; do you keep a playlist of other such talks you’ve run across in the past?
Svetlitski 5y ago
Definitely agree. Another example of lost performance that comes to mind is (ignorance of) C struct padding. In terms of modern tools, Rust has definitely made some headway here: • Variables are immutable by default • The layout of a struct’s fields in memory is opaque to the programmer, allowing the compiler to reorder them to minimize padding • As a natural consequence of the borrow checker, Rust has much more information about whether or not two references could alias than most other languages • The programmer is very much made to feel the cost of dynamic-dispatch (via dyn), and static-dispatch is the norm • The compiler can omit bounds checks when it can prove they’re unnecessary • It’s easy to safely have structs that live entirely on the stack (i.e. no heap allocation) Overall this is an unsolved problem though. It seems to me that the trend is that the more details about the internals that the language exposes to the programmer, the fewer optimizations can be made automatically. Essentially the ideal is declarative performance, where the programmer expresses the idea and the compiler figures out the optimal implementation (which is in stark contrast to most high-performance code being hand-tuned today). Take a look at something like Halide for a successful application of this philosophy: https://halide-lang.org/ https://halide-lang.org/
specialist 5y ago
> ...with another level of indirection Ya. I've spent the last two decades removing levels of indirection, reducing call stack depth. This has put me at odds with most everyone. I've been insulted many times for not understanding "software architecture". Heh. I started a design pattern study group shortly after GoF's book was published. I think it's a lot like jazz. 1st time thru, mind blown. 2nd time, ah, I think I'm starting to get it. 3rd time, well, duh. Eventually students get full circle, a la "there is no spoon", and you reject the veneer of "design patterns". Said another way, it's just another tool, learn to use it wisely. Learn to reject temptation. Aspects, annotations, runtime code reflection, metaprogramming, frameworks, schema mappings, factories, singletons, and whatnot are just monkey work. (Most of the time.) Exquisite puzzles (finger traps) to suck smart people people in. To distract from the real work of solving customer's problems, business needs. I think the kids these days call it "yak shaving." Yes, I am very guilty of this. Past, present, and surely future. I'm no better than anyone else. I recognize my failings and am trying to improve.

by m12k 5y ago

bob1029 5y ago

> Or let the programmer tell the compiler a little bit more about what they need and don't need the code to do, so the compiler can do a much better job if it. You can do this today in languages like C#: https://docs.microsoft.com/en-us/dotnet/csharp/write-safe-efficient-code#declare-readonly-members-for-mutable-structs https://docs.microsoft.com/en-us/dotnet/csharp/write-safe-ef... > Adding the readonly modifier to members that don't mutate state provides two related benefits. First, the compiler enforces your intent. That member can't mutate the struct's state. Second, the compiler won't create defensive copies of in parameters when accessing a readonly member. The compiler can make this optimization safely because it guarantees that the struct is not modified by a readonly member.

XorNot 5y ago
This feels like some seriously missed static analysis optimization though - why do I need to specify this? There's vanishingly few cases the answer to "looks like you don't modify this ever, make it readonly?" where the answer is not yes.
- bob1029 5y ago
  Probably because reflection & compilation+activation of types existing outside the original compilation context is possible at runtime. I much prefer to having a language/runtime that can handle extremely complex problem scenarios out of the box (with the need for occasional tuning), rather than the other way around. It is not very often that I find myself screwing around with performance-intensive code. Usually, a few very important things are addressed at the lowest levels and then you put all your nice abstractions back on top for daily driving.

jiggawatts 5y ago

> flexibility is something you pay for in performance I lightbulb moment I had recently was that the classic interface-based OO programming paradigm utilising virtual functions was essentially the following declaration by the programmer to the compiler: The function that is invoked can change at any time, even call-to-call; assume nothing. it is a black box with unknown, unknowable contents. Do not inline it, do not inspect it, expect the worst. If you look at a typical web framework, with call stacks hundreds of methods deep, you'll come to the same horrifying realisation that these are all inherently un-optimisable because of that statement. The compiler can do nearly nothing, and the processor will also struggle to paper over the inefficiency. Meanwhile, in reality, the specific implementations of those interfaces never change during actual production execution! Dynamic dispatch is utilised only during development time, and even then only at compile time to inject test harnesses, mocks, or whatever. At runtime, 99.999999% of all function calls are essentially static, and dynamic dispatch is a pure waste.

dzaima 5y ago
Except that a smart runtime can actually work with the assumption that only one implementation of a class will be used, and inline what would otherwise be virtual methods. And, as far as I know, the JVM definitely does this, and at multiple levels too - if there's only one implementation of an interface, it can just assume that everything's static and inlineable (at the cost of deoptimizations if that turns out to be false later). If not, it can still decide to inline, and just check the type beforehand.
MaxBarraclough 5y ago
> The compiler can do nearly nothing, Optimizing JIT-compiling JVMs are often able to optimize virtual calls that only ever resolve to one implementation. They may even inline such calls. (edit dzaima got there first.)
- duped 5y ago
  It's old too, Self was doing this in the late 80s

mtVessel 5y ago

As much as people like to talk about "footguns", that's the value proposition of C++, zero cost for anything you don't use. Managed/interpreted langs have a different proposition: pay up front for all the features, but your complexity never increases whether you use none, one or all of them.

MauranKilom 5y ago
Well, pointer aliasing for example is a very tricky thing in C++ too. Even with strict aliasing, it is very unintuitive from the outside when you're going to trip over unexpected performance hurdles. (Does this data structure have a char* inside? Who knows, but if it does, there goes vectorization!).
- username90 5y ago
  If you don't dereference the char* it will get treated an an integer and be vectorizable. In C++ you don't pay for what you don't use. If you need the char* in the loop I'm not sure why you'd be thinking about vectorization. Maybe you meant parallelism?

de_keyboard 5y ago

Great talk: Constraints Liberate, Liberties Constrain — Runar Bjarnason https://www.youtube.com/watch?v=GqmsQeSzMdw https://www.youtube.com/watch?v=GqmsQeSzMdw

adverbly 5y ago
Wow thanks for linking that. Great talk. I'm always on the lookout for talks about interesting and generally useful concepts, but where I can follow only a small part of them. Makes me feel like I can learn and grow into a wiser person. This certainly fits that criteria.
- whoisburbansky 5y ago
  This talk fits a similar bucket in my head; do you keep a playlist of other such talks you’ve run across in the past?

Svetlitski 5y ago

Definitely agree. Another example of lost performance that comes to mind is (ignorance of) C struct padding. In terms of modern tools, Rust has definitely made some headway here: • Variables are immutable by default • The layout of a struct’s fields in memory is opaque to the programmer, allowing the compiler to reorder them to minimize padding • As a natural consequence of the borrow checker, Rust has much more information about whether or not two references could alias than most other languages • The programmer is very much made to feel the cost of dynamic-dispatch (via dyn), and static-dispatch is the norm • The compiler can omit bounds checks when it can prove they’re unnecessary • It’s easy to safely have structs that live entirely on the stack (i.e. no heap allocation) Overall this is an unsolved problem though. It seems to me that the trend is that the more details about the internals that the language exposes to the programmer, the fewer optimizations can be made automatically. Essentially the ideal is declarative performance, where the programmer expresses the idea and the compiler figures out the optimal implementation (which is in stark contrast to most high-performance code being hand-tuned today). Take a look at something like Halide for a successful application of this philosophy: https://halide-lang.org/ https://halide-lang.org/

specialist 5y ago

> ...with another level of indirection Ya. I've spent the last two decades removing levels of indirection, reducing call stack depth. This has put me at odds with most everyone. I've been insulted many times for not understanding "software architecture". Heh. I started a design pattern study group shortly after GoF's book was published. I think it's a lot like jazz. 1st time thru, mind blown. 2nd time, ah, I think I'm starting to get it. 3rd time, well, duh. Eventually students get full circle, a la "there is no spoon", and you reject the veneer of "design patterns". Said another way, it's just another tool, learn to use it wisely. Learn to reject temptation. Aspects, annotations, runtime code reflection, metaprogramming, frameworks, schema mappings, factories, singletons, and whatnot are just monkey work. (Most of the time.) Exquisite puzzles (finger traps) to suck smart people people in. To distract from the real work of solving customer's problems, business needs. I think the kids these days call it "yak shaving." Yes, I am very guilty of this. Past, present, and surely future. I'm no better than anyone else. I recognize my failings and am trying to improve.