3 ms·

I'm also seeing this trend as well.. "Software needs to be made more energy efficient." Usually this comment is coming from disgruntled data center designers w

I'm also seeing this trend as well.. "Software needs to be made more energy efficient." Usually this comment is coming from disgruntled data center designers who feel like they are unfairly being forced to eek out more efficiency when the onus should be the IT manufacturers and SW developers. This attitude is somewhat justified as it is possible to run servers at much warmer temps, but the DC industry (namely their customers) still continue to insist on SLAs that have unreasonably low air temperature set points. IMO, the first wave of optimizing software for energy consumption has already happened via virtualization. Ideally, I can run the same workload with less metal/watts. I feel like the next round of optimization will be more problematic. You could re-write software to take advantage of hardware acceleration features that are baked into the silicon to ideally speed things up/reduce power consumption, however this comes with its own issues. You need to be very familiar with your hardware. Integrating much more tightly means your married to your metal... kinda like CUDA lock-in in the GPU ecosystem.

josephg 4y ago
> You need to be very familiar with your hardware. Integrating much more tightly means your married to your metal I don’t know how true it is in the HPC space, but the more optimization work I do the more I realise how horribly inefficient 99% of programs are. Most devs these days just don’t care about making software fast, and when they do they have no idea where to start or what will help. Last year I spent some time optimising CRDTs (used in collaborative text editing). Automerge (a well known library) took 5 minutes to run a certain benchmark. Yjs (considered a mature, well performing library) took 1 second to run the same test. After spending some time optimizing, I have code now which can do the same work in 4 milliseconds. I’m not using any special hardware - this is just in straight forward rust. The remarkable part is that automerge isn’t written in an uncharacteristically bad way. Sure, it uses immutablejs - but plenty of modern javascript programs do something very similar. Even yjs, despite being the target of some careful optimization work, is still 200x slower than it could be. - and thus, wasting 99.5% of its energy budget. If even yjs can be sped up by 200x, how much do you think most enterprise software could be sped up, if people tried? 500x? 10000x? I don’t think we need deeper integration with hardware. We just need more software developers who care at all about performance. Virtualisation wipes out 20%+ of the performance of the machine, depending on what you’re doing. But we pay that cost gladly because it’s marginally easier than sharing a Linux kernel between multiple processes. I think if our CPUs had capped out 50x slower than they are today, the average user would hardly notice any difference. The only difference would be that software would be more expensive to make, because software engineers would need to actually understand how the computer works to make software perform. Things like the virtual DOM wouldn’t be fast enough to use at scale. Video games and zoom wouldn’t be quite as pretty. But we’d live.
- jstimpfle 4y ago
  Yes. Stuff only needs to be fast enough to be barely usable. It's a simple economic principle. Making programs faster need not be hard. When looking at programs I often think how easier it would have been to organize them right, so they would be both easier to make and faster to run. However, this requires more experienced engineers, which are a scarce resource. So we're back to economics.
- kaba0 4y ago
  > If even yjs can be sped up by 200x, how much do you think most enterprise software could be sped up, if people tried? 500x? 10000x? Like 10x most? You compare something that is basically one hot loop with something that at most spends 1% of its CPU time in hot loops, the other being just random code paths and calls to other, often optimized libs (e.g. for making network connections, db communication, etc)
  - anonymoushn 4y ago
    It sounds like the latter thing probably has some design issues that lead to a flat performance profile where every single thing takes orders of magnitude more work than necessary?
  - josephg 4y ago
    10x? Look, we can’t know for sure. But yjs was already an optimized library and another 200x speed up was available there. Why do you assume any of the network code used to talk to your database is well optimized? A flat performance profile isn’t a sign everything is fast. It’s usually a sign everything is uniformly sluggish. And yeah, that makes optimization harder. But it’s far from impossible. Take rendering html for example. I’ve seen rendering (just the rendering part, not database lookups) take over 200ms in nodejs. Generating a string with the equivalent rust compile-time templating engine can spit out the same html in microseconds. There is an insane amount of performance on the table with modern CPUs. Just about everything short of network round trips and stable diffusion and should complete instantly.
alrlroipsp 4y ago
> I feel like the next round of optimization will be more problematic. You could re-write software to take advantage of hardware acceleration features that are baked into the silicon to ideally speed things up/reduce power consumption, however this comes with its own issues. I'm sorry but this is far from the truth. The absolute majority of software written today is not optimized. There has been a huge shift away from efficient close-to-the-metal programming that was popular in the 90s and earlier, to what we have today. There is enormous energy and efficiency benefits we could reap today, without "problematic" optimizations. We just need to start optimizing at all.
- c_o_n_v_e_x 4y ago
  >The absolute majority of software written today is not optimized. If it's not problematic, then why is it not being done already? Obviously, it's the economics... which are problematic.
  - anonymoushn 4y ago
    This presupposes a lot of market efficiency that may not exist. Writing faster software often allows you to avoid a lot of architectural complexity.
- kaba0 4y ago
  Is there? Most applications are not running in a hot loop over terabytes of data. There are a few where indeed insane speed-ups are possible (e.g. it would be dumb to compete with video codecs in anything other than assembly with plenty of vector instructions), but most programs barely make a CPU sweat, they just idle on IO. Virtual threads like Go, Java would bring much much more benefit over rewriting said programs in a lower level language.

josephg 4y ago

> You need to be very familiar with your hardware. Integrating much more tightly means your married to your metal I don’t know how true it is in the HPC space, but the more optimization work I do the more I realise how horribly inefficient 99% of programs are. Most devs these days just don’t care about making software fast, and when they do they have no idea where to start or what will help. Last year I spent some time optimising CRDTs (used in collaborative text editing). Automerge (a well known library) took 5 minutes to run a certain benchmark. Yjs (considered a mature, well performing library) took 1 second to run the same test. After spending some time optimizing, I have code now which can do the same work in 4 milliseconds. I’m not using any special hardware - this is just in straight forward rust. The remarkable part is that automerge isn’t written in an uncharacteristically bad way. Sure, it uses immutablejs - but plenty of modern javascript programs do something very similar. Even yjs, despite being the target of some careful optimization work, is still 200x slower than it could be. - and thus, wasting 99.5% of its energy budget. If even yjs can be sped up by 200x, how much do you think most enterprise software could be sped up, if people tried? 500x? 10000x? I don’t think we need deeper integration with hardware. We just need more software developers who care at all about performance. Virtualisation wipes out 20%+ of the performance of the machine, depending on what you’re doing. But we pay that cost gladly because it’s marginally easier than sharing a Linux kernel between multiple processes. I think if our CPUs had capped out 50x slower than they are today, the average user would hardly notice any difference. The only difference would be that software would be more expensive to make, because software engineers would need to actually understand how the computer works to make software perform. Things like the virtual DOM wouldn’t be fast enough to use at scale. Video games and zoom wouldn’t be quite as pretty. But we’d live.

jstimpfle 4y ago
Yes. Stuff only needs to be fast enough to be barely usable. It's a simple economic principle. Making programs faster need not be hard. When looking at programs I often think how easier it would have been to organize them right, so they would be both easier to make and faster to run. However, this requires more experienced engineers, which are a scarce resource. So we're back to economics.
kaba0 4y ago
> If even yjs can be sped up by 200x, how much do you think most enterprise software could be sped up, if people tried? 500x? 10000x? Like 10x most? You compare something that is basically one hot loop with something that at most spends 1% of its CPU time in hot loops, the other being just random code paths and calls to other, often optimized libs (e.g. for making network connections, db communication, etc)
- anonymoushn 4y ago
  It sounds like the latter thing probably has some design issues that lead to a flat performance profile where every single thing takes orders of magnitude more work than necessary?
- josephg 4y ago
  10x? Look, we can’t know for sure. But yjs was already an optimized library and another 200x speed up was available there. Why do you assume any of the network code used to talk to your database is well optimized? A flat performance profile isn’t a sign everything is fast. It’s usually a sign everything is uniformly sluggish. And yeah, that makes optimization harder. But it’s far from impossible. Take rendering html for example. I’ve seen rendering (just the rendering part, not database lookups) take over 200ms in nodejs. Generating a string with the equivalent rust compile-time templating engine can spit out the same html in microseconds. There is an insane amount of performance on the table with modern CPUs. Just about everything short of network round trips and stable diffusion and should complete instantly.

alrlroipsp 4y ago

> I feel like the next round of optimization will be more problematic. You could re-write software to take advantage of hardware acceleration features that are baked into the silicon to ideally speed things up/reduce power consumption, however this comes with its own issues. I'm sorry but this is far from the truth. The absolute majority of software written today is not optimized. There has been a huge shift away from efficient close-to-the-metal programming that was popular in the 90s and earlier, to what we have today. There is enormous energy and efficiency benefits we could reap today, without "problematic" optimizations. We just need to start optimizing at all.

c_o_n_v_e_x 4y ago
>The absolute majority of software written today is not optimized. If it's not problematic, then why is it not being done already? Obviously, it's the economics... which are problematic.
- anonymoushn 4y ago
  This presupposes a lot of market efficiency that may not exist. Writing faster software often allows you to avoid a lot of architectural complexity.
kaba0 4y ago
Is there? Most applications are not running in a hot loop over terabytes of data. There are a few where indeed insane speed-ups are possible (e.g. it would be dumb to compete with video codecs in anything other than assembly with plenty of vector instructions), but most programs barely make a CPU sweat, they just idle on IO. Virtual threads like Go, Java would bring much much more benefit over rewriting said programs in a lower level language.