14 ms·

Moore to the rescue. But can they keep up with the bloatware curve? If we cook all the horsepower down to glue, and glue ever more horrible libraries together,

Moore to the rescue. But can they keep up with the bloatware curve? If we cook all the horsepower down to glue, and glue ever more horrible libraries together, when will we reach peak horse-pisa-stack? I stopped eyeing for cpu advances a decade ago. If we could have a NN-driven prefetcher that is able to anticipate cache misses from instructions and data, 300 cycles ahead of time, that would be some speedup we all could benefit from, if it found its way into hardware. https://drive.google.com/file/d/17THn_qNQJTH0ewRvDukllRAKSFgEI9mm/view https://drive.google.com/file/d/17THn_qNQJTH0ewRvDukllRAKSFg...

simias 5y ago
I think it's the glue that keeps up with the CPU power curve, so to speak. You give devs more RAM and more cycles and they'll find a way to use them with inefficient languages, suboptimal code and shiny UI. I think it's important to remember that for instance Zelda: Link's Awakening was less than 512KiB in size and ran on a primitive 1MHz CPU. But at the same time we have to acknowledge how better it can be to develop for a modern target. We can decide to waste this potential with bloated electron runtimes, but we can also leverage it to make things we thought impossible before.
- enkid 5y ago
  I don't think this is a good example. Zelda: Link's Awakening, while ground breaking, can be, and has been improved upon. Using those resources to create a better user experience is not inefficiency. That's the entire point of a video game - a good user experience.
  - dijit 5y ago
    I think you're not disagreeing with the parent. You can have an exceptional experience with a game in less than the storage and processing power it takes to run Hello World today; To that end you can make things better and nicer than Links Awakening with modern resources. But can you make something 500-3000x better? (that's the order of magnitude we're talking about, with Slack vs Zelda)
    - dale_glass 5y ago
      No, because improvements don't scale that way. 320x200 was more or less where graphics started. 640x480 was much better. 720p was a lot better than broadcast TV. 1080p was a very nice improvement for a lot of uses. 4K is visibly better, but mostly eye candy. 8K is pointless for a desktop monitor. The steps give you 4X the number of pixels. At 320x200 you're severely constrained in detail. You'd have a hard time for instance accurately simulating an aircraft's cockpit -- you don't have enough pixels to work with. 1080p to 4K is in most cases not the same kind of difference. There's little for which 1080p is actually constrained in some way and require sacrificing something or working around the lack of pixels. This isn't because modern software design is bloated and sucks, but simply because improvements diminish. Our needs, abilities and physical senses are finite.
      - dangus 5y ago
        While I agree that returns diminish, I’ll be the first to disagree that those enhancements are unnecessary. If I do text work on a 1440p 27” monitor I can see the pixels. Anything less than 4K is unacceptable to me. I could see an 8K monitor being a worthy upgrade if I could afford it and my computer could drive it, especially if it meant that I could make the screen larger or wider and still avoid seeing pixels. Also, we might consider that digital tech is arguably still catching up in some aspects to our best analog technology like 70mm film.
        zozbot234 5y ago
        What's wrong with "seeing pixels" when all you're doing is basic text work? Bitmap fonts - or properly-hinted TrueType/OpenType fonts, now that the relevant patents have expired - can be pixel-perfect and are far from unreadable.
        dangus 5y ago
        Because I have to look at it all day and I’d like it to look smooth. Also, I can read smaller text when the resolution is higher. When it comes to technology and innovation I believe that asking “why do I need this?” can often be the wrong question. It feels like a bias toward complacency. Why drive a car when you’ve already got a horse?
- bogwog 5y ago
  > but we can also leverage it to make things we thought impossible before. Or realistically, to lower hiring costs.
  - simias 5y ago
    Or shorten dev time, sure. I don't think those are necessarily bad things. The industry seems to have enough trouble as it is to recruit software devs. Can you imagine if you needed deep assembler mastery in order to develop a website? It would have hampered the development of the industry tremendously.
    - PicassoCTs 5y ago
      Joke aside, imagine a language that is optimized, to allow for ease of later optimization. As in object orientated, fancy beginner level code goes in, and the hot-loop could be rewritten if needs be by a professional, without doing a complete data-orientated refactoring of the original code.
      - hvidgaard 5y ago
        I don't think that is possible to be honest. I'd love it, but most sub optimal choices require extensive refactorings to work around. I'd love to be proven wrong though.
        Siira 5y ago
        The current frontiers are nowhere near what is possible though; E.g., python, bash, ruby could be a lot faster if their development was subsidized by governments. Cpp doesn’t have a good package manager (and likely will not have one in the coming decade either). Go doesn’t have obvious features such as string interpolation and generics. Rust’s tooling still isn’t pleasant. ...
        elzbardico 5y ago
        ADA and in a minor extent COBOL, were heavily subsidized by the government. Probably even PL/1
- hvidgaard 5y ago
  > You give devs more RAM and more cycles and they'll find a way to use them with inefficient languages, suboptimal code and shiny UI. I'd go as far as saying that most devs do not want to use inefficient languages, write suboptimal code or program that new shiny UI. Quite to the contrary, but users and management demand most of those things just with other names.
  - gmadsen 5y ago
    really depends on the context. Embedded software has its own fun challenges, but for a simple web app, the last thing I want to worry about is controlling allocations
- DerekL 5y ago
  > I think it's important to remember that for instance Zelda: Link's Awakening was less than 512KiB in size and ran on a primitive 1MHz CPU. The Game Boy CPU ran at 4.19 MHz.
  - monocasa 5y ago
    Eh, sort of. It's memory bus runs at 1.024 MHz, and operations take 4 CPU cycle multiples to execute. It's really more like a 1.024Mhz processor with the internals running QDR.
- biglost 5y ago
  My boss would never allow me to optimize anything, the only thing that matters to him is write the fastest you can and just make it work and make the client suffer until the client crys and threat to leave the service
ComodoHacker 5y ago
>I stopped eyeing for cpu advances a decade ago So have you completely missed the huge advance of ARM that you have in your phone now?
- pjmlp 5y ago
  ART and V8 take care of it for me, so yeah.
- deleted 5y ago
  [deleted]
maccard 5y ago
> I stopped eyeing for cpu advances a decade ago Even in the x64 space massive advances have been made [0]. For 2.5x the power, you can have 8x the parallelism, with 50% faster single core speeds, (plus all the micro improvements with avx, or the other improvements that make 3.5GHz today faster than a 3.5GHz cpu from 10 years ago) [0] https://cpu.userbenchmark.com/Compare/AMD-Ryzen-TR-3970X-vs-Intel-Core-i7-2600/m970030vs620 https://cpu.userbenchmark.com/Compare/AMD-Ryzen-TR-3970X-vs-...
- wmanley 5y ago
  I think the point is that while CPUs have gotten (significantly) better over time using a computer doesn't feel any faster than 10 years ago because software has more than made up for it. On my 3 year old Ubuntu machine here it takes ~2-3s to launch gnome-calculator. This is no faster than launching gnome-calculator in 2004 on my 2004 Athlon 64 with Gnome 2.6. It's not like gnome-calculator is suddenly doing a lot more now. It feels like I spend more time now waiting for the computer than I did then. It's the same with the phone - click, wait, click, wait, click, wait. For a taste of what it could be like try running Windows 2000 in a VM. Zoooooooom. Why? Developers will only optimise their software when it becomes unbearably slow - and in the absence of automated performance tracking and eternal vigilance software gets slower over time. The result is that there is no reason* to get excited about CPU advances. It doesn't make your life better, it's just a cost you have to pay to stop your life getting worse. * Sure there are some reasons - if you're already using/dependant upon heavily optimised software like games or machine learning, but for most people that's a very small part of their computing use.
  - 2pEXgD0fZ5cF 5y ago
    Fully agree. It also feels like some kind of disdain for performance and optimization has taken root in certain parts of our industry and communities. The "client does not care about this" school of thought that aggressively sees optimizations as foolish to even think about. While this kind of thinking obviously has its place especially when it comes to proper management, it has been way overblown to the point of not being able to mention performance advantages without someone appearing out of nowhere to smugly tell you how the evil performance and optimization (the enemies of creating revenue, apparently) should be ignored and looked down upon in the face of the allmighty "faster development times".
Const-me 5y ago
> If we could have a NN-driven prefetcher that is able to anticipate cache misses It’s not that critical for memory prefetcher because cache hierarchy is helping a lot. Most software doesn’t read random addresses. And prefetchers in modern CPUs are pretty good already. Another thing, prefetcher is not a great application of AI because the address space is huge. Branch prediction is another story. CPUs only need to predict a single Boolean value, taken/not taken. Modern processors are actually using neural networks for that: https://www.amd.com/en/technologies/sense-mi https://www.amd.com/en/technologies/sense-mi https://www.youtube.com/watch?v=uZRih6APtiQ https://www.youtube.com/watch?v=uZRih6APtiQ
- thrwaeasddsaf 5y ago
  > Most software doesn’t read random addresses. Is there any literature or studies to back this? I thought one of the downfalls of modern software is a ridiculous number of tiny dynamic allocations, often deliberately randomized in space. Lots of pointer chasing and hard-to-predict access patterns. And some people work really hard to make their access patterns cache friendly, which is far from trivial and for most software, not a cost they can justify. Sometimes changing the access patterns means reorganizing all vital data structures and going SoA to AoS (or vice versa) and due to limited language support, that can mean sweeping changes across the entire code base. It doesn't help that as new features are added, requirements on these data structures and access patterns can change a lot.
  - Const-me 5y ago
    > Lots of pointer chasing and hard-to-predict access patterns. They’re hard to predict individually, but statistically I think they’re rather predictable. Many programs indeed use tons of small allocations and chase quite a few of these pointers. Still, after the program runs for a while, the data is distributed appropriately across caches (e.g. my CPU has 32kb L1D, 512kb L2 and 32MB L3) and it sometimes becomes not terribly bad. > work really hard to make their access patterns cache friendly, which is far from trivial and for most software Depends on the goal. If the goal is approaching the numbers listed in CPU specs, FLOPS or RAM bandwidth, indeed, it can be prohibitively expensive to do. If the goal is more modest, to make software that’s reasonably fast, for many statically typed languages it’s not that bad. Prefer arrays/vector over the rest of the containers, prefer B+ trees over red-black ones, prefer value types over classes, reduce strings to integer IDs when possible, and it might be good enough without going full-HPC with these structs-of-arrays and the complexity overhead.
fallingknife 5y ago
Wouldn't that level of speculation massively degrade security?
titzer 5y ago
It doesn't matter if you fetch 300 cycles in the future, you'll eventually saturate cache bandwidth. It's worth noting that the M1 has a reorder buffer of more 600 micro-ops, which essentially means it can be 600 instructions in the "future" (it's only the future from the point of view of instruction commit).
jdashg 5y ago
Worth mentioning that Ryzen's branch predictor does in fact have an NN component.
joe_the_user 5y ago
Regardless of bloatware, I don't see how more transistors can indefinitely simulate an increase in processor speed. It seems like it would violate P!=NP or the halting problem or something - as well as being a massive security hole. This is why gpgpu processing seems like the wave of the future. Alternatively, human level AI won't come from OpenAI but the final desperate Intel team trying to create a system that anticipates all serial routines the processor can run in parallel.

simias 5y ago

I think it's the glue that keeps up with the CPU power curve, so to speak. You give devs more RAM and more cycles and they'll find a way to use them with inefficient languages, suboptimal code and shiny UI. I think it's important to remember that for instance Zelda: Link's Awakening was less than 512KiB in size and ran on a primitive 1MHz CPU. But at the same time we have to acknowledge how better it can be to develop for a modern target. We can decide to waste this potential with bloated electron runtimes, but we can also leverage it to make things we thought impossible before.

enkid 5y ago
I don't think this is a good example. Zelda: Link's Awakening, while ground breaking, can be, and has been improved upon. Using those resources to create a better user experience is not inefficiency. That's the entire point of a video game - a good user experience.
- dijit 5y ago
  I think you're not disagreeing with the parent. You can have an exceptional experience with a game in less than the storage and processing power it takes to run Hello World today; To that end you can make things better and nicer than Links Awakening with modern resources. But can you make something 500-3000x better? (that's the order of magnitude we're talking about, with Slack vs Zelda)
  - dale_glass 5y ago
    No, because improvements don't scale that way. 320x200 was more or less where graphics started. 640x480 was much better. 720p was a lot better than broadcast TV. 1080p was a very nice improvement for a lot of uses. 4K is visibly better, but mostly eye candy. 8K is pointless for a desktop monitor. The steps give you 4X the number of pixels. At 320x200 you're severely constrained in detail. You'd have a hard time for instance accurately simulating an aircraft's cockpit -- you don't have enough pixels to work with. 1080p to 4K is in most cases not the same kind of difference. There's little for which 1080p is actually constrained in some way and require sacrificing something or working around the lack of pixels. This isn't because modern software design is bloated and sucks, but simply because improvements diminish. Our needs, abilities and physical senses are finite.
    - dangus 5y ago
      While I agree that returns diminish, I’ll be the first to disagree that those enhancements are unnecessary. If I do text work on a 1440p 27” monitor I can see the pixels. Anything less than 4K is unacceptable to me. I could see an 8K monitor being a worthy upgrade if I could afford it and my computer could drive it, especially if it meant that I could make the screen larger or wider and still avoid seeing pixels. Also, we might consider that digital tech is arguably still catching up in some aspects to our best analog technology like 70mm film.
      - zozbot234 5y ago
        What's wrong with "seeing pixels" when all you're doing is basic text work? Bitmap fonts - or properly-hinted TrueType/OpenType fonts, now that the relevant patents have expired - can be pixel-perfect and are far from unreadable.
        dangus 5y ago
        Because I have to look at it all day and I’d like it to look smooth. Also, I can read smaller text when the resolution is higher. When it comes to technology and innovation I believe that asking “why do I need this?” can often be the wrong question. It feels like a bias toward complacency. Why drive a car when you’ve already got a horse?
bogwog 5y ago
> but we can also leverage it to make things we thought impossible before. Or realistically, to lower hiring costs.
- simias 5y ago
  Or shorten dev time, sure. I don't think those are necessarily bad things. The industry seems to have enough trouble as it is to recruit software devs. Can you imagine if you needed deep assembler mastery in order to develop a website? It would have hampered the development of the industry tremendously.
  - PicassoCTs 5y ago
    Joke aside, imagine a language that is optimized, to allow for ease of later optimization. As in object orientated, fancy beginner level code goes in, and the hot-loop could be rewritten if needs be by a professional, without doing a complete data-orientated refactoring of the original code.
    - hvidgaard 5y ago
      I don't think that is possible to be honest. I'd love it, but most sub optimal choices require extensive refactorings to work around. I'd love to be proven wrong though.
      - Siira 5y ago
        The current frontiers are nowhere near what is possible though; E.g., python, bash, ruby could be a lot faster if their development was subsidized by governments. Cpp doesn’t have a good package manager (and likely will not have one in the coming decade either). Go doesn’t have obvious features such as string interpolation and generics. Rust’s tooling still isn’t pleasant. ...
        elzbardico 5y ago
        ADA and in a minor extent COBOL, were heavily subsidized by the government. Probably even PL/1
hvidgaard 5y ago
> You give devs more RAM and more cycles and they'll find a way to use them with inefficient languages, suboptimal code and shiny UI. I'd go as far as saying that most devs do not want to use inefficient languages, write suboptimal code or program that new shiny UI. Quite to the contrary, but users and management demand most of those things just with other names.
- gmadsen 5y ago
  really depends on the context. Embedded software has its own fun challenges, but for a simple web app, the last thing I want to worry about is controlling allocations
DerekL 5y ago
> I think it's important to remember that for instance Zelda: Link's Awakening was less than 512KiB in size and ran on a primitive 1MHz CPU. The Game Boy CPU ran at 4.19 MHz.
- monocasa 5y ago
  Eh, sort of. It's memory bus runs at 1.024 MHz, and operations take 4 CPU cycle multiples to execute. It's really more like a 1.024Mhz processor with the internals running QDR.
biglost 5y ago
My boss would never allow me to optimize anything, the only thing that matters to him is write the fastest you can and just make it work and make the client suffer until the client crys and threat to leave the service

ComodoHacker 5y ago

>I stopped eyeing for cpu advances a decade ago So have you completely missed the huge advance of ARM that you have in your phone now?

pjmlp 5y ago
ART and V8 take care of it for me, so yeah.
deleted 5y ago
[deleted]

maccard 5y ago

> I stopped eyeing for cpu advances a decade ago Even in the x64 space massive advances have been made [0]. For 2.5x the power, you can have 8x the parallelism, with 50% faster single core speeds, (plus all the micro improvements with avx, or the other improvements that make 3.5GHz today faster than a 3.5GHz cpu from 10 years ago) [0] https://cpu.userbenchmark.com/Compare/AMD-Ryzen-TR-3970X-vs-Intel-Core-i7-2600/m970030vs620 https://cpu.userbenchmark.com/Compare/AMD-Ryzen-TR-3970X-vs-...

wmanley 5y ago
I think the point is that while CPUs have gotten (significantly) better over time using a computer doesn't feel any faster than 10 years ago because software has more than made up for it. On my 3 year old Ubuntu machine here it takes ~2-3s to launch gnome-calculator. This is no faster than launching gnome-calculator in 2004 on my 2004 Athlon 64 with Gnome 2.6. It's not like gnome-calculator is suddenly doing a lot more now. It feels like I spend more time now waiting for the computer than I did then. It's the same with the phone - click, wait, click, wait, click, wait. For a taste of what it could be like try running Windows 2000 in a VM. Zoooooooom. Why? Developers will only optimise their software when it becomes unbearably slow - and in the absence of automated performance tracking and eternal vigilance software gets slower over time. The result is that there is no reason* to get excited about CPU advances. It doesn't make your life better, it's just a cost you have to pay to stop your life getting worse. * Sure there are some reasons - if you're already using/dependant upon heavily optimised software like games or machine learning, but for most people that's a very small part of their computing use.
- 2pEXgD0fZ5cF 5y ago
  Fully agree. It also feels like some kind of disdain for performance and optimization has taken root in certain parts of our industry and communities. The "client does not care about this" school of thought that aggressively sees optimizations as foolish to even think about. While this kind of thinking obviously has its place especially when it comes to proper management, it has been way overblown to the point of not being able to mention performance advantages without someone appearing out of nowhere to smugly tell you how the evil performance and optimization (the enemies of creating revenue, apparently) should be ignored and looked down upon in the face of the allmighty "faster development times".

Const-me 5y ago

> If we could have a NN-driven prefetcher that is able to anticipate cache misses It’s not that critical for memory prefetcher because cache hierarchy is helping a lot. Most software doesn’t read random addresses. And prefetchers in modern CPUs are pretty good already. Another thing, prefetcher is not a great application of AI because the address space is huge. Branch prediction is another story. CPUs only need to predict a single Boolean value, taken/not taken. Modern processors are actually using neural networks for that: https://www.amd.com/en/technologies/sense-mi https://www.amd.com/en/technologies/sense-mi https://www.youtube.com/watch?v=uZRih6APtiQ https://www.youtube.com/watch?v=uZRih6APtiQ

thrwaeasddsaf 5y ago
> Most software doesn’t read random addresses. Is there any literature or studies to back this? I thought one of the downfalls of modern software is a ridiculous number of tiny dynamic allocations, often deliberately randomized in space. Lots of pointer chasing and hard-to-predict access patterns. And some people work really hard to make their access patterns cache friendly, which is far from trivial and for most software, not a cost they can justify. Sometimes changing the access patterns means reorganizing all vital data structures and going SoA to AoS (or vice versa) and due to limited language support, that can mean sweeping changes across the entire code base. It doesn't help that as new features are added, requirements on these data structures and access patterns can change a lot.
- Const-me 5y ago
  > Lots of pointer chasing and hard-to-predict access patterns. They’re hard to predict individually, but statistically I think they’re rather predictable. Many programs indeed use tons of small allocations and chase quite a few of these pointers. Still, after the program runs for a while, the data is distributed appropriately across caches (e.g. my CPU has 32kb L1D, 512kb L2 and 32MB L3) and it sometimes becomes not terribly bad. > work really hard to make their access patterns cache friendly, which is far from trivial and for most software Depends on the goal. If the goal is approaching the numbers listed in CPU specs, FLOPS or RAM bandwidth, indeed, it can be prohibitively expensive to do. If the goal is more modest, to make software that’s reasonably fast, for many statically typed languages it’s not that bad. Prefer arrays/vector over the rest of the containers, prefer B+ trees over red-black ones, prefer value types over classes, reduce strings to integer IDs when possible, and it might be good enough without going full-HPC with these structs-of-arrays and the complexity overhead.

fallingknife 5y ago

Wouldn't that level of speculation massively degrade security?

titzer 5y ago

It doesn't matter if you fetch 300 cycles in the future, you'll eventually saturate cache bandwidth. It's worth noting that the M1 has a reorder buffer of more 600 micro-ops, which essentially means it can be 600 instructions in the "future" (it's only the future from the point of view of instruction commit).

jdashg 5y ago

Worth mentioning that Ryzen's branch predictor does in fact have an NN component.

joe_the_user 5y ago

Regardless of bloatware, I don't see how more transistors can indefinitely simulate an increase in processor speed. It seems like it would violate P!=NP or the halting problem or something - as well as being a massive security hole. This is why gpgpu processing seems like the wave of the future. Alternatively, human level AI won't come from OpenAI but the final desperate Intel team trying to create a system that anticipates all serial routines the processor can run in parallel.