7 ms·

I would expect to be able to find another 1% slowdown somewhere in an implementation of another "system programming language" - would that also disqualify it fr

I would expect to be able to find another 1% slowdown somewhere in an implementation of another "system programming language" - would that also disqualify it from being such a language?

WalterBright 4y ago
It's a competitive business. If a compiler generates 1% faster code, you need 1% fewer servers in your server farm, which translates to enormous amounts of money. Or if your hedgefund trading software runs 1% faster, you can get your trades in faster than the other guy, eating his lunch. Or would you like to reduce your costs of rented servers in the cloud by 1%?
- hayley-patton 4y ago
  Right, I don't mean to downplay the significance of 1%, but aren't there many ways to win or lose 1%? More specifically perhaps, are there so few GCed pointers that a faster collector cannot recuperate that 1%? moonchild's suggestion to monomorphise against GC/explicitly managed pointers might reduce that figure lower; evidently you know D programs better than I, but it doesn't seem unreasonable that the 1% can be won by a faster GC.
  - WalterBright 4y ago
    > More specifically perhaps, are there so few GCed pointers that a faster collector cannot recuperate that 1%? Of course, it depends on your coding style, but yes, that is generally true.
  - kohlerm 4y ago
    A 1% slowdown is a ridiculous low number IMHO. I would argue that almost no program is within 1% of the optimal speed (however you define it). Therefore practically it does not matter. If you think it matters then the consequence would be that you write everything in Assembler.
    - WalterBright 4y ago
      > Therefore practically it does not matter. You won't notice it on your desktop. But you will notice it in cases I mentioned. Top shelf games also are not going to give up that 1%, or so the game devs tell me.
      - pjmlp 4y ago
        And yet even the likes of Sony and Nintendo are working in major titles with Unity. Unity is also the official developer SDK for Microsoft's HoloLens Virtual Reality Toolkit.
        jcelerier 4y ago
        And 70 million people go eat Mickey D's every day, doesn't make it good in any way. Whenever I play unity games (rarely to be honest as I play maybe 5-20 hours a year nowadays, but this has been consistent) I brace myself for a laggy experience. Most people don't care about this it seems but it really makes me want to close the game when there's a laggy scene (and definitely put the tech used in my blacklist).
        pjmlp 4y ago
        I have the same happening to me with C and C++ games, so it isn't a given that the language along is the cause, rather the ability of the programmers.
        jcelerier 4y ago
        The very vast majority of native games I played don't have the kind of lag that unity games have (and seem to do much more things). Only very bad one I could remember was Neverwinter nights 2.. that one was really unplayable even on god-tier computers
        pjmlp 4y ago
        I would still blame the programmers. Doing the game in C++ certainly hasn't helped Cyberpunk 2077 performance and being bug free.
- pca006132 4y ago
  I guess the problem is usually not about 1% throughput, but about where the 1% is and how it affects the latency. You can have the 1% overhead evenly distributed which is normally fine, but you can also have a latency spike for some operation (and I guess this is the case for barriers), which IMO really hurts in some cases.
  - hayley-patton 4y ago
    Barriers are probably too small to cause latency spikes. The object logging barriers are a bit complicated, as they write an object into a thread-local log and infrequently have to get more memory for the log. But a card barrier is rather simple, and could look like SHR R_pointer, 9 AND R_pointer, <cards> MOV BYTE [R_cards + R_pointer], 0 where <cards> is one below an immediate power of two number of cards, each card covering 2^9 = 512 bytes of heap. This is roughly the write barrier in SBCL on x86-64; there isn't much of a reason to think it will cause a latency spike. On the other hand, "Barriers revisited" does mention a pathology about how cards can introduce false sharing, but that's still more of a smear than a spike.
- mrighele 4y ago
  1% Less servers in the farm means 1% less expenses on the server farms. That is a lot only if the server farm is your main cost and you run on thin margins. My guess is that it is not that big of a niche, although the players are probably big
  - WalterBright 4y ago
    If you look at the revenue cloud service providers get, and that revenue is based on CPU time used, 1% is a huge amount of money. It's like weight on an airplane. Boeing told me back in 1980 that saving one pound was worth a quarter million dollars of investment. That'd be like a million bucks today.
    - darksaints 4y ago
      There are massive python and bash codebases used at all of the major cloud service providers. And none of the cloud service providers are using custom-compiled PGO-optimized OSes specifically optimized for the exact architecture and instruction set and resource set that they will be deployed on, despite >1% performance advantages for doing so. Because sometimes the complexity of doing things the fastest way possible is not worth the performance improvement. The boeing example is illuminating. One could conceive of infinite possibilities where it would be better to spend the extra million dollars than to get something that saved a pound but didn't meet other project requirements. What good would it be to buy a $1M cheaper 747 if the landing gear broke on each landing? Everything is a tradeoff, even things that are extremely important like weight on airplanes or runtime performance in computers.
- nequo 4y ago
  > if your hedgefund trading software runs 1% faster Jane Street has used OCaml for about 20 years. Last year, OCaml merged an improvement to its GC that reduced latency by 75% (by one measure), and reduced execution time by 23% when compiling OCaml and around 6 to 7% when compiling Coq libraries: https://github.com/ocaml/ocaml/pull/10195#issuecomment-896150306 https://github.com/ocaml/ocaml/pull/10195#issuecomment-89615... Performance is definitely left on the table even in mature and mission-critical software.
  - WalterBright 4y ago
    Hedge fund traders are using D, for performance. I've even had a hedge fund trader come to me to teach them how to implement their own compiler, as they wanted to make it faster. Remember the stories about traders trying to shave milliseconds off of their trading speed?
    - deleted 4y ago
      [deleted]
    - tptacek 4y ago
      Finance shops use everything. The exchanges themselves are, or were last time I worked on anything there, built largely out of Java.
    - kasey_junk 4y ago
      Milliseconds is at least 20 years out of date. Last I worked in the industry we measured our latency in micros and that was 10 years ago. Even then anyone that actually cared about latency never left the switch, all fpga & asic. I even met a hedge fund that ran their back test platform mostly on hardware because it allowed them to run more tests, which was a strategic advantage. That said, 1% for free makes the ghost of Admiral Hopper happy and that’s reason enough for all of us.
- throwaway894345 4y ago
  I mean, 1% is fine if it comes with no other tradeoffs, but that's probably not the case. I would trade a 1% improvement in performance for a 1% improvement in developer productivity almost every single time (very rarely are businesses performance constrained such that they can afford to slow development for marginal performance deltas), and when you're operating near the margins of performance, it's usually more likely that you'll have to halve productivity to eek out a small amount of performance improvement.

I would expect to be able to find another 1% slowdown somewhere in an implementation of another "system programming language" - would that also disqualify it from being such a language?

WalterBright 4y ago

It's a competitive business. If a compiler generates 1% faster code, you need 1% fewer servers in your server farm, which translates to enormous amounts of money. Or if your hedgefund trading software runs 1% faster, you can get your trades in faster than the other guy, eating his lunch. Or would you like to reduce your costs of rented servers in the cloud by 1%?

hayley-patton 4y ago
Right, I don't mean to downplay the significance of 1%, but aren't there many ways to win or lose 1%? More specifically perhaps, are there so few GCed pointers that a faster collector cannot recuperate that 1%? moonchild's suggestion to monomorphise against GC/explicitly managed pointers might reduce that figure lower; evidently you know D programs better than I, but it doesn't seem unreasonable that the 1% can be won by a faster GC.
- WalterBright 4y ago
  > More specifically perhaps, are there so few GCed pointers that a faster collector cannot recuperate that 1%? Of course, it depends on your coding style, but yes, that is generally true.
- kohlerm 4y ago
  A 1% slowdown is a ridiculous low number IMHO. I would argue that almost no program is within 1% of the optimal speed (however you define it). Therefore practically it does not matter. If you think it matters then the consequence would be that you write everything in Assembler.
  - WalterBright 4y ago
    > Therefore practically it does not matter. You won't notice it on your desktop. But you will notice it in cases I mentioned. Top shelf games also are not going to give up that 1%, or so the game devs tell me.
    - pjmlp 4y ago
      And yet even the likes of Sony and Nintendo are working in major titles with Unity. Unity is also the official developer SDK for Microsoft's HoloLens Virtual Reality Toolkit.
      - jcelerier 4y ago
        And 70 million people go eat Mickey D's every day, doesn't make it good in any way. Whenever I play unity games (rarely to be honest as I play maybe 5-20 hours a year nowadays, but this has been consistent) I brace myself for a laggy experience. Most people don't care about this it seems but it really makes me want to close the game when there's a laggy scene (and definitely put the tech used in my blacklist).
        pjmlp 4y ago
        I have the same happening to me with C and C++ games, so it isn't a given that the language along is the cause, rather the ability of the programmers.
        jcelerier 4y ago
        The very vast majority of native games I played don't have the kind of lag that unity games have (and seem to do much more things). Only very bad one I could remember was Neverwinter nights 2.. that one was really unplayable even on god-tier computers
        pjmlp 4y ago
        I would still blame the programmers. Doing the game in C++ certainly hasn't helped Cyberpunk 2077 performance and being bug free.
pca006132 4y ago
I guess the problem is usually not about 1% throughput, but about where the 1% is and how it affects the latency. You can have the 1% overhead evenly distributed which is normally fine, but you can also have a latency spike for some operation (and I guess this is the case for barriers), which IMO really hurts in some cases.
- hayley-patton 4y ago
  Barriers are probably too small to cause latency spikes. The object logging barriers are a bit complicated, as they write an object into a thread-local log and infrequently have to get more memory for the log. But a card barrier is rather simple, and could look like SHR R_pointer, 9 AND R_pointer, <cards> MOV BYTE [R_cards + R_pointer], 0 where <cards> is one below an immediate power of two number of cards, each card covering 2^9 = 512 bytes of heap. This is roughly the write barrier in SBCL on x86-64; there isn't much of a reason to think it will cause a latency spike. On the other hand, "Barriers revisited" does mention a pathology about how cards can introduce false sharing, but that's still more of a smear than a spike.
mrighele 4y ago
1% Less servers in the farm means 1% less expenses on the server farms. That is a lot only if the server farm is your main cost and you run on thin margins. My guess is that it is not that big of a niche, although the players are probably big
- WalterBright 4y ago
  If you look at the revenue cloud service providers get, and that revenue is based on CPU time used, 1% is a huge amount of money. It's like weight on an airplane. Boeing told me back in 1980 that saving one pound was worth a quarter million dollars of investment. That'd be like a million bucks today.
  - darksaints 4y ago
    There are massive python and bash codebases used at all of the major cloud service providers. And none of the cloud service providers are using custom-compiled PGO-optimized OSes specifically optimized for the exact architecture and instruction set and resource set that they will be deployed on, despite >1% performance advantages for doing so. Because sometimes the complexity of doing things the fastest way possible is not worth the performance improvement. The boeing example is illuminating. One could conceive of infinite possibilities where it would be better to spend the extra million dollars than to get something that saved a pound but didn't meet other project requirements. What good would it be to buy a $1M cheaper 747 if the landing gear broke on each landing? Everything is a tradeoff, even things that are extremely important like weight on airplanes or runtime performance in computers.
nequo 4y ago
> if your hedgefund trading software runs 1% faster Jane Street has used OCaml for about 20 years. Last year, OCaml merged an improvement to its GC that reduced latency by 75% (by one measure), and reduced execution time by 23% when compiling OCaml and around 6 to 7% when compiling Coq libraries: https://github.com/ocaml/ocaml/pull/10195#issuecomment-896150306 https://github.com/ocaml/ocaml/pull/10195#issuecomment-89615... Performance is definitely left on the table even in mature and mission-critical software.
- WalterBright 4y ago
  Hedge fund traders are using D, for performance. I've even had a hedge fund trader come to me to teach them how to implement their own compiler, as they wanted to make it faster. Remember the stories about traders trying to shave milliseconds off of their trading speed?
  - deleted 4y ago
    [deleted]
  - tptacek 4y ago
    Finance shops use everything. The exchanges themselves are, or were last time I worked on anything there, built largely out of Java.
  - kasey_junk 4y ago
    Milliseconds is at least 20 years out of date. Last I worked in the industry we measured our latency in micros and that was 10 years ago. Even then anyone that actually cared about latency never left the switch, all fpga & asic. I even met a hedge fund that ran their back test platform mostly on hardware because it allowed them to run more tests, which was a strategic advantage. That said, 1% for free makes the ghost of Admiral Hopper happy and that’s reason enough for all of us.
throwaway894345 4y ago
I mean, 1% is fine if it comes with no other tradeoffs, but that's probably not the case. I would trade a 1% improvement in performance for a 1% improvement in developer productivity almost every single time (very rarely are businesses performance constrained such that they can afford to slow development for marginal performance deltas), and when you're operating near the margins of performance, it's usually more likely that you'll have to halve productivity to eek out a small amount of performance improvement.