8 ms·

All the more reason to go distributed and parallel, and embrace simpler architectures that can support this. We've gotten complacent. Software keeps getting fa

by arnon 8y ago

All the more reason to go distributed and parallel, and embrace simpler architectures that can support this. We've gotten complacent. Software keeps getting faster, despite bad coding - just because of the improvements in hardware.

chooseaname 8y ago
We've been complacent. Thus the old saying, "Intel giveth, Microsoft taketh away."[0] Engineers have known for a long time that we aren't as efficient as we could be. [0] Not attempting to beat up on either of those companies.
- overcast 8y ago
  I don't know, Windows 10 seems to be faster on even older hardware. Upgrading from 7 to 10 makes the machine feel MUCH more responsive. Seems to bring life back to the old machines around here.
  - gameswithgo 8y ago
    Maybe you are just feeling a machine without bloat vs one with lots of bloat that has built up for 3rd party software? At least one aspect of windows10 design will add input latency, which is the forced vsync in the windowing system.
    - overcast 8y ago
      Nope, fresh installs of Windows 7 definitely not nearly as responsive. Thousands of machines a year, Windows 10 is by far the fastest.
      - monocasa 8y ago
        Not sure why you're b eing downvoted. Windows 10 is the first release that Microsoft had really fully internalized the lessons learned from the whole Longhorn debacle. That is consumers aren't ever going to have 10Ghz processors.
- ghaff 8y ago
  We've also just optimized for features, time to market, and programming approaches that don't require the most highly-skilled and trained developers. You can change all that of course (and we do for some applications/situations) but you don't get significantly more efficient code for free. It probably takes longer to write and a lot of current developers probably can't do it.
  - gameswithgo 8y ago
    >We've also just optimized for features, time to market, and programming approaches that don't require the most highly-skilled and trained developers We think we have been optimizing for those things, but I don't know that we actually have. Does Ruby/Python really help you get to market faster than would similar languages with static typing and compilers or JITs? (Kotlin, C#, F#, Nim, etc) There is research that suggests not, certainly I don't think the effect is large if it even exists. Many people use TypeScript because they feel it is a productivity boon over Javascript. We might one day leverage that and use the static types to compile TypeScript more efficient WASM or something, win win situation.
    - mrguyorama 8y ago
      If you're looking for efficiency, wouldn't the actual win be to just use less javascript?
      - gameswithgo 8y ago
        Yes, but typescript is a great way to ease people into that, since it is a superset of javascript which could be leveraged to AOT compile more efficient code. Also it is a very nice language!
    - jkabrg 8y ago
      It's not clear what he's referring to. It could be garbage collection, libraries, frameworks, optimizing compilers... - Garbage collection make performance unpredictable because of collection spikes. - If you use a big library, you now have to load the library, which is an extra time-sink. - Likewise for frameworks. - Optimizing compilers optimize the 80% of our programs that don't need optimizing, and under-optimize the 20% that do. (This is DJB's argument.) That might be what he's saying. He didn't say anything about dynamic vs. static typing. Also, there's a counter-argument to this, that writing non-garbage-collected code might result in security bugs (to do with memory management). And security is a non-functional requirement that some might say is more important than speed. It's hard to judge though; there are loads of requiremenets in software dev, and it's hard to balance them.
- pacaro 8y ago
  One irony of this expression is the role Intel played in making sure that as much of USB as possible had to go through the CPU
- agumonkey 8y ago
  do you have a list of things that are really in need of redesign ?
- ryanmarsh 8y ago
  I'm not a low level developer (whatever that means today). I've written a very minimal amount of production C. Everything else has been higher level languages. So I apologize in advance for my ignorance. Isn't the problem really memory bandwidth? Aren't most CPU cycles "wasted" on speculative execution while waiting on fetches from memory slower than CPU caches? Wouldn't a hardware model where all of the RAM (SRAM?) was on the CPU die (fabrication and DRAM clock speed aside) solve this problem and make everything an order of magnitude faster? What am I missing? Is this really the programmers fault? The CPU literally spends most of its time waiting on my instructions, not choking on them, so why is this my fault for being lazy?
  - baybal2 8y ago
    >What am I missing? You are missing money... SRAM and wider data paths are expensive. You are totally right about that taking more logic into synchronous domain domain is an almost universal solution to getting more performance. The faster the chips become, the more IO gap becomes. Experienced MCU programmers know quite a number of algorithms that may run faster on "dumb" 100mhz MCUs than on a 20-cored Xeon, and are proud to show them off. Even when some of MCU cores may take like 6 cycles to do an arithmetic op, their strong side is that it will be taking them these 6 cycles to do that all and every time consistently, and that their memory runs synchronously. Say, in a 1000000 cycle loop, where every operation results in cache miss, Xeon will spend more time on a loop than a wimpy 100Mhz MCU that simply don't take any penalty on unpredictable, heavy on synchronous ops cope. Apple A11 is so fast for an ARM just because they choose the simplest, the surest, most direct, but also the most expensive way to scale performance: tons of SRAM and prefetch logic everywhere to reduce IO gap as much as possible.
  - marcosdumay 8y ago
    Memory bandwidth is a problem. A large one. But saying that "most" CPU cycles are wasted to it is a huge overrate. I have read estimates that go as high as 1/3 of the CPU lost to it (sorry, no reference here), but even those were not usual.
  - paulmd 8y ago
    20 MB of RAM should be enough for anybody! /s I was reading a thing a while back on here or Reddit where someone asked about an OS that ran entirely in cache, after all in principle a modern processor has more L3 than an original Unix minicomputer. Since many of the memory subsystems operate on the assumption that changes should be propagated to memory that you would still need to have RAM there even if you weren't really "using" it. There was a suggestion that since the bringup code begins entirely on-processor (before initializing RAM) that you could hack the Management Engine and keep running in that mode forever.
- cm2187 8y ago
  The modern version being, "Intel giveth, javascript and python taketh away".
wlesieutre 8y ago
Apple doing an optimization focused update with iOS 12 is a breath of fresh air. My iPad Pro 9.7" (one version old, came before the 10.5") drops frames in iOS 11's full screen blur transition animations. Before iOS 11 it was fine, but either the iOS team didn't test animations on anything but their brand new hardware, or they tested and didn't care if it ran smoothly. Having one of the OS's most frequent animations choke on anything but the brand newest GPU in the $650+ model? Not a great look, especially with Apple's reputation for making devices that last a while. Fingers crossed that iOS 12 fixes it.
- overcast 8y ago
  iOS 12 is indeed faster, but it seems to be at a cost to increased battery usage. From my understanding, CPU is ramped up to maximum performance mode when needed, like launching applications and other demanding tasks. Then dialed back down when not needed. It could be just the normal beta battery usage issues, but mine gets hit hard.
  - gameswithgo 8y ago
    getting the work done as fast as possible then shutting down is usually the way to get the best battery life.
    - overcast 8y ago
      I would agree, but doesn't seem to be the case here! This is also with a new(6 month) battery. It's either ramping up for much too long, or it's just normal beta battery usage.
    - baybal2 8y ago
      No, Power consumption scales roughly quadratically in relation to frequency. Modern cores all have features like power and frequency gating, performance counters - the stuff needed to allocate just enough of computing power to complete computations in time. This is very, very evident in things like music playback - a lot of intermittent io and computational spikes.
jcranmer 8y ago
> All the more reason to go distributed and parallel, and embrace simpler architectures that can support this. It's very easy to stamp out more ALUs on a chip than we can reasonably fill with data (and, arguably, we've already hit that point with consumer hardware). Improving parallelization nearly always comes at a cost to serial performance, and if you have code that requires a latency-sensitive, unparallelizable critical loop... well, you're basically saying "screw you."
- ansible 8y ago
  You might be interested in the tech talks from Mill Computing: https://millcomputing.com/ https://millcomputing.com/ They're working on a new processor architecture, which is very interesting in many, many ways. And they're working on ways to parallelize existing code, in ways that can't really be done with conventional superscalar architectures.
  - jcranmer 8y ago
    People keep bringing up the Mill processor in response to parallelization; I've been hearing about it for ~5 years now. And yet, where is it? I see lots of talk about its architecture, but there's no hard comparisons on things like "we get DGEMM at 2x the FLOPS/W of NVidia's latest GPU" or "we get 2x SPECCPU 2017 as Intel's latest processor."
    - mechagodzilla 8y ago
      That's because they don't have a real product, and they likely never will. For the most part, the ISA just needs to be 'good enough' to allow things like branch prediction, speculative execution, out-of-order execution, etc. You can do that with x86, ARM, POWER, whatever - the instruction set doesn't matter a whole lot, it's just how many/what type of transistors you can throw at it, and how hard you want to work on timing and power optimization. That's the biggest problem with Moore's law fizzling out - computer architecture isn't actually that hard, most of the work is just figuring out how to apply the extra transistors you're given. So no matter how specialized you get, you aren't going to get more than 1-2 generations of a chip out of a single semiconductor process, as there is only so much that clever architecture can accomplish.
rootlocus 8y ago
Herb Sutter wrote an article on this called "The Free Lunch is Over" in 2005, 13 years ago. http://www.gotw.ca/publications/concurrency-ddj.htm http://www.gotw.ca/publications/concurrency-ddj.htm
- ghaff 8y ago
  When it became clear around that time that processor frequency was tapped out and that multi-core was the future there was a lot of hand-wringing around parallel programing. There weren't the tools. There weren't the skills. Etc. For a variety of reasons, the issues turned out to be less than feared overall. But you were still getting more transistors at the time. You just had to use them in different ways.
gameswithgo 8y ago
>. Software keeps getting faster, despite bad coding Disagree. Much of it keeps getting slower.
- castlecrasher2 8y ago
  Websites for sure. Load up HN compared to most shop or news sites, enormous difference.
- jcriddle4 8y ago
  Have you tried the latest fad? Serverless functions, so you take maybe a 128 to 512 MB of memory virtual machine and have it run a function that is maybe a few hundred lines of code? I double dare you to find something less efficient. Even better is the speed between machines isn't going to get that much faster as some jerk has declared the speed of light to be a constant so say hello to some fun latency issues. To top it off after you invest massive amounts in serverless functions, with all there lockin, you have no control over what Amazon, Microsoft or xxx are going to be charging you in 5 or 10 years. Particularly if the fad falls out of favor and these companies decide to wind the operations down.
  - noir_lord 8y ago
    I agree with everything but your tone. I'm sure it solves problems they just aren't my problems. Which frankly applies to nearly everything AWS offers. I've seen people design for massive scaleout before they even have a final idea of the product. Frankly for the kind of systems I work on I have collosal vertical headroom before I even need to think about going lateral with scaling. I mean the largest database I deal with would fit into the RAM of a single mid-range 2018 server (192GB would do it). The most important database on our main work system will fit into the RAM on my Thinkpad with enough space to run an IDE and Vagrant (32GB of RAM in the laptop so high but insane).
  - alayne 8y ago
    Have you seen the previous fad? Data centers full of idle servers.

by arnon 8y ago

chooseaname 8y ago

We've been complacent. Thus the old saying, "Intel giveth, Microsoft taketh away."[0] Engineers have known for a long time that we aren't as efficient as we could be. [0] Not attempting to beat up on either of those companies.

overcast 8y ago
I don't know, Windows 10 seems to be faster on even older hardware. Upgrading from 7 to 10 makes the machine feel MUCH more responsive. Seems to bring life back to the old machines around here.
- gameswithgo 8y ago
  Maybe you are just feeling a machine without bloat vs one with lots of bloat that has built up for 3rd party software? At least one aspect of windows10 design will add input latency, which is the forced vsync in the windowing system.
  - overcast 8y ago
    Nope, fresh installs of Windows 7 definitely not nearly as responsive. Thousands of machines a year, Windows 10 is by far the fastest.
    - monocasa 8y ago
      Not sure why you're b eing downvoted. Windows 10 is the first release that Microsoft had really fully internalized the lessons learned from the whole Longhorn debacle. That is consumers aren't ever going to have 10Ghz processors.
ghaff 8y ago
We've also just optimized for features, time to market, and programming approaches that don't require the most highly-skilled and trained developers. You can change all that of course (and we do for some applications/situations) but you don't get significantly more efficient code for free. It probably takes longer to write and a lot of current developers probably can't do it.
- gameswithgo 8y ago
  >We've also just optimized for features, time to market, and programming approaches that don't require the most highly-skilled and trained developers We think we have been optimizing for those things, but I don't know that we actually have. Does Ruby/Python really help you get to market faster than would similar languages with static typing and compilers or JITs? (Kotlin, C#, F#, Nim, etc) There is research that suggests not, certainly I don't think the effect is large if it even exists. Many people use TypeScript because they feel it is a productivity boon over Javascript. We might one day leverage that and use the static types to compile TypeScript more efficient WASM or something, win win situation.
  - mrguyorama 8y ago
    If you're looking for efficiency, wouldn't the actual win be to just use less javascript?
    - gameswithgo 8y ago
      Yes, but typescript is a great way to ease people into that, since it is a superset of javascript which could be leveraged to AOT compile more efficient code. Also it is a very nice language!
  - jkabrg 8y ago
    It's not clear what he's referring to. It could be garbage collection, libraries, frameworks, optimizing compilers... - Garbage collection make performance unpredictable because of collection spikes. - If you use a big library, you now have to load the library, which is an extra time-sink. - Likewise for frameworks. - Optimizing compilers optimize the 80% of our programs that don't need optimizing, and under-optimize the 20% that do. (This is DJB's argument.) That might be what he's saying. He didn't say anything about dynamic vs. static typing. Also, there's a counter-argument to this, that writing non-garbage-collected code might result in security bugs (to do with memory management). And security is a non-functional requirement that some might say is more important than speed. It's hard to judge though; there are loads of requiremenets in software dev, and it's hard to balance them.
pacaro 8y ago
One irony of this expression is the role Intel played in making sure that as much of USB as possible had to go through the CPU
agumonkey 8y ago
do you have a list of things that are really in need of redesign ?
ryanmarsh 8y ago
I'm not a low level developer (whatever that means today). I've written a very minimal amount of production C. Everything else has been higher level languages. So I apologize in advance for my ignorance. Isn't the problem really memory bandwidth? Aren't most CPU cycles "wasted" on speculative execution while waiting on fetches from memory slower than CPU caches? Wouldn't a hardware model where all of the RAM (SRAM?) was on the CPU die (fabrication and DRAM clock speed aside) solve this problem and make everything an order of magnitude faster? What am I missing? Is this really the programmers fault? The CPU literally spends most of its time waiting on my instructions, not choking on them, so why is this my fault for being lazy?
- baybal2 8y ago
  >What am I missing? You are missing money... SRAM and wider data paths are expensive. You are totally right about that taking more logic into synchronous domain domain is an almost universal solution to getting more performance. The faster the chips become, the more IO gap becomes. Experienced MCU programmers know quite a number of algorithms that may run faster on "dumb" 100mhz MCUs than on a 20-cored Xeon, and are proud to show them off. Even when some of MCU cores may take like 6 cycles to do an arithmetic op, their strong side is that it will be taking them these 6 cycles to do that all and every time consistently, and that their memory runs synchronously. Say, in a 1000000 cycle loop, where every operation results in cache miss, Xeon will spend more time on a loop than a wimpy 100Mhz MCU that simply don't take any penalty on unpredictable, heavy on synchronous ops cope. Apple A11 is so fast for an ARM just because they choose the simplest, the surest, most direct, but also the most expensive way to scale performance: tons of SRAM and prefetch logic everywhere to reduce IO gap as much as possible.
- marcosdumay 8y ago
  Memory bandwidth is a problem. A large one. But saying that "most" CPU cycles are wasted to it is a huge overrate. I have read estimates that go as high as 1/3 of the CPU lost to it (sorry, no reference here), but even those were not usual.
- paulmd 8y ago
  20 MB of RAM should be enough for anybody! /s I was reading a thing a while back on here or Reddit where someone asked about an OS that ran entirely in cache, after all in principle a modern processor has more L3 than an original Unix minicomputer. Since many of the memory subsystems operate on the assumption that changes should be propagated to memory that you would still need to have RAM there even if you weren't really "using" it. There was a suggestion that since the bringup code begins entirely on-processor (before initializing RAM) that you could hack the Management Engine and keep running in that mode forever.
cm2187 8y ago
The modern version being, "Intel giveth, javascript and python taketh away".

wlesieutre 8y ago

Apple doing an optimization focused update with iOS 12 is a breath of fresh air. My iPad Pro 9.7" (one version old, came before the 10.5") drops frames in iOS 11's full screen blur transition animations. Before iOS 11 it was fine, but either the iOS team didn't test animations on anything but their brand new hardware, or they tested and didn't care if it ran smoothly. Having one of the OS's most frequent animations choke on anything but the brand newest GPU in the $650+ model? Not a great look, especially with Apple's reputation for making devices that last a while. Fingers crossed that iOS 12 fixes it.

overcast 8y ago
iOS 12 is indeed faster, but it seems to be at a cost to increased battery usage. From my understanding, CPU is ramped up to maximum performance mode when needed, like launching applications and other demanding tasks. Then dialed back down when not needed. It could be just the normal beta battery usage issues, but mine gets hit hard.
- gameswithgo 8y ago
  getting the work done as fast as possible then shutting down is usually the way to get the best battery life.
  - overcast 8y ago
    I would agree, but doesn't seem to be the case here! This is also with a new(6 month) battery. It's either ramping up for much too long, or it's just normal beta battery usage.
  - baybal2 8y ago
    No, Power consumption scales roughly quadratically in relation to frequency. Modern cores all have features like power and frequency gating, performance counters - the stuff needed to allocate just enough of computing power to complete computations in time. This is very, very evident in things like music playback - a lot of intermittent io and computational spikes.

jcranmer 8y ago

> All the more reason to go distributed and parallel, and embrace simpler architectures that can support this. It's very easy to stamp out more ALUs on a chip than we can reasonably fill with data (and, arguably, we've already hit that point with consumer hardware). Improving parallelization nearly always comes at a cost to serial performance, and if you have code that requires a latency-sensitive, unparallelizable critical loop... well, you're basically saying "screw you."

ansible 8y ago
You might be interested in the tech talks from Mill Computing: https://millcomputing.com/ https://millcomputing.com/ They're working on a new processor architecture, which is very interesting in many, many ways. And they're working on ways to parallelize existing code, in ways that can't really be done with conventional superscalar architectures.
- jcranmer 8y ago
  People keep bringing up the Mill processor in response to parallelization; I've been hearing about it for ~5 years now. And yet, where is it? I see lots of talk about its architecture, but there's no hard comparisons on things like "we get DGEMM at 2x the FLOPS/W of NVidia's latest GPU" or "we get 2x SPECCPU 2017 as Intel's latest processor."
  - mechagodzilla 8y ago
    That's because they don't have a real product, and they likely never will. For the most part, the ISA just needs to be 'good enough' to allow things like branch prediction, speculative execution, out-of-order execution, etc. You can do that with x86, ARM, POWER, whatever - the instruction set doesn't matter a whole lot, it's just how many/what type of transistors you can throw at it, and how hard you want to work on timing and power optimization. That's the biggest problem with Moore's law fizzling out - computer architecture isn't actually that hard, most of the work is just figuring out how to apply the extra transistors you're given. So no matter how specialized you get, you aren't going to get more than 1-2 generations of a chip out of a single semiconductor process, as there is only so much that clever architecture can accomplish.

rootlocus 8y ago

Herb Sutter wrote an article on this called "The Free Lunch is Over" in 2005, 13 years ago. http://www.gotw.ca/publications/concurrency-ddj.htm http://www.gotw.ca/publications/concurrency-ddj.htm

ghaff 8y ago
When it became clear around that time that processor frequency was tapped out and that multi-core was the future there was a lot of hand-wringing around parallel programing. There weren't the tools. There weren't the skills. Etc. For a variety of reasons, the issues turned out to be less than feared overall. But you were still getting more transistors at the time. You just had to use them in different ways.

gameswithgo 8y ago

>. Software keeps getting faster, despite bad coding Disagree. Much of it keeps getting slower.

castlecrasher2 8y ago
Websites for sure. Load up HN compared to most shop or news sites, enormous difference.
jcriddle4 8y ago
Have you tried the latest fad? Serverless functions, so you take maybe a 128 to 512 MB of memory virtual machine and have it run a function that is maybe a few hundred lines of code? I double dare you to find something less efficient. Even better is the speed between machines isn't going to get that much faster as some jerk has declared the speed of light to be a constant so say hello to some fun latency issues. To top it off after you invest massive amounts in serverless functions, with all there lockin, you have no control over what Amazon, Microsoft or xxx are going to be charging you in 5 or 10 years. Particularly if the fad falls out of favor and these companies decide to wind the operations down.
- noir_lord 8y ago
  I agree with everything but your tone. I'm sure it solves problems they just aren't my problems. Which frankly applies to nearly everything AWS offers. I've seen people design for massive scaleout before they even have a final idea of the product. Frankly for the kind of systems I work on I have collosal vertical headroom before I even need to think about going lateral with scaling. I mean the largest database I deal with would fit into the RAM of a single mid-range 2018 server (192GB would do it). The most important database on our main work system will fit into the RAM on my Thinkpad with enough space to run an IDE and Vagrant (32GB of RAM in the laptop so high but insane).
- alayne 8y ago
  Have you seen the previous fad? Data centers full of idle servers.