3 ms·

Seems to me that most software is horribly inefficient anyway. If push came to shove we might just consider making better software?

rat9988 4y ago
Software engineers are expensive too.
sneak 4y ago
The cutting edge of software that mostly uses these advanced process node sizes is not horribly inefficient. The new applications enabled by, for example, GPU-accelerated ML are not the same as running 400 chrome tabs on an M1. The hot paths in most important high-end computing are mostly quite efficient.
- colinsane 4y ago
  > The hot paths in most important high-end computing are mostly quite efficient. it’s hard to determine the lower bound on a lot of these things. in scientific computing (e.g. physics sims) i still run across a lot of runtime branching that could easily be compiled out if we were using languages that had stronger type systems/templates. we use linear approximations for a lot of things, but it’s been shown that smarter interpolation can let you decrease your resolution without losing accuracy/stability. there’s an easy 2x or more perf gain every time you do that successfully. there’s also the boundary conditions: we always simulate more volume of space than we’re actually interested in so as to avoid certain types of distortion (reflections from the boundary of the simulation). over time we’ve learned tricks for reducing those reflections: we’ve applied a lot of general-purpose optimizations, but there’s evidence that we can go further if we encode more simulation-specific information at these boundaries. this whole area has just been slow, steady, compounded optimizations. i don’t work in ML, but i find it very unlikely that (a) optimizing neural networks is a completely solved problem or (b) that you all know how to train them using the fewest iterations. you make sweeping statements that just aren't obvious to me.
  - lrem 4y ago
    You seem to be arguing against "optimal", while quoting parent's "mostly quite efficient". The reality is that while everything is converging to a locally optimal point on the tradeoff curve, a _lot_ of software out there runs in the "hardware is my driving cost" locale. This makes companies squeeze epsilons out of their servers. There's also the "selling hardware is my profit centre", which end-users get to pay for.
    - colinsane 4y ago
      the comment which spawned this chain is: > If push came to shove we might just consider making better software? as you say, "a _lot_ of software out there runs in the 'hardware is my driving cost' locale." if something changed -- say, we lost access to high-end fabs -- then my argument is that we have room to focus higher on the stack to make this far less impactful than it would otherwise be. in most places, software is not "optimal", as you seem to agree. in very few places have we even calculated a bound of what optimal _is_. the HPC example was to highlight the latter part: we shift from something like a linear gradient descent to some different descent algorithm and it works better in practice. ML training techniques are always changing to get faster convergence. very, very few of these HPC areas have actual, well-defined performance bounds. it's just a tradeoff of "how much do we invest in algorithms" v.s. "how much do we invest in hardware". i read GP as suggesting otherwise, based on the content of the comment they were replying to. i'm not even sure there's a disagreement between you and i. maybe not even between me and sneak. i kinda wonder if all of us here just have a different number in mind for what qualifies as "mostly quite efficient". i'm completely confident we have at least 2x perf gains sitting around in just about any computational area that's presently dominated by hardware costs (e.g. HPC, but not e.g. packet-switching/networking). 10x perf gains in many of them: not by micro-optimizing things like memory allocations, but by being inventive with algorithms. that kind of gain is meaningful to me in this area: enough that i wouldn't dismiss it as "mostly quite efficient" as a rebuttal against that original "if push came to show we might just consider making better software".
baja_blast 4y ago
It's depressing how bloated and slow some software is despite running on way faster machines. The software engineering accomplished in the 90's was nothing short of astonishing.
- mike_hock 4y ago
  No, it wasn't. It was passable. The software "engineering" "accomplished" today is just nothing short of pathetic.
onlyrealcuzzo 4y ago
Software is bloated and slow to make it faster to develop mostly. Look at your average website with 85 megabytes of JavaScript and Bootstrap for the UI. TreeShaking in theory can get rid of all that unneeded JS - and there should be something that can do it for CSS, too. But most people aren't using it. And even if they did, there would still be a lot of bloat left over from using the libraries to make life a little bit easier. Go to average midsize startup and the amount of this going on is multiplied exponentially by the number of developers. There's not really a cure here. There's not enough engineers that know how to do things absent these frameworks - that even if you do know - it's still not worth it because the people you'll be working with probably don't.

rat9988 4y ago

Software engineers are expensive too.

sneak 4y ago

The cutting edge of software that mostly uses these advanced process node sizes is not horribly inefficient. The new applications enabled by, for example, GPU-accelerated ML are not the same as running 400 chrome tabs on an M1. The hot paths in most important high-end computing are mostly quite efficient.

colinsane 4y ago
> The hot paths in most important high-end computing are mostly quite efficient. it’s hard to determine the lower bound on a lot of these things. in scientific computing (e.g. physics sims) i still run across a lot of runtime branching that could easily be compiled out if we were using languages that had stronger type systems/templates. we use linear approximations for a lot of things, but it’s been shown that smarter interpolation can let you decrease your resolution without losing accuracy/stability. there’s an easy 2x or more perf gain every time you do that successfully. there’s also the boundary conditions: we always simulate more volume of space than we’re actually interested in so as to avoid certain types of distortion (reflections from the boundary of the simulation). over time we’ve learned tricks for reducing those reflections: we’ve applied a lot of general-purpose optimizations, but there’s evidence that we can go further if we encode more simulation-specific information at these boundaries. this whole area has just been slow, steady, compounded optimizations. i don’t work in ML, but i find it very unlikely that (a) optimizing neural networks is a completely solved problem or (b) that you all know how to train them using the fewest iterations. you make sweeping statements that just aren't obvious to me.
- lrem 4y ago
  You seem to be arguing against "optimal", while quoting parent's "mostly quite efficient". The reality is that while everything is converging to a locally optimal point on the tradeoff curve, a _lot_ of software out there runs in the "hardware is my driving cost" locale. This makes companies squeeze epsilons out of their servers. There's also the "selling hardware is my profit centre", which end-users get to pay for.
  - colinsane 4y ago
    the comment which spawned this chain is: > If push came to shove we might just consider making better software? as you say, "a _lot_ of software out there runs in the 'hardware is my driving cost' locale." if something changed -- say, we lost access to high-end fabs -- then my argument is that we have room to focus higher on the stack to make this far less impactful than it would otherwise be. in most places, software is not "optimal", as you seem to agree. in very few places have we even calculated a bound of what optimal _is_. the HPC example was to highlight the latter part: we shift from something like a linear gradient descent to some different descent algorithm and it works better in practice. ML training techniques are always changing to get faster convergence. very, very few of these HPC areas have actual, well-defined performance bounds. it's just a tradeoff of "how much do we invest in algorithms" v.s. "how much do we invest in hardware". i read GP as suggesting otherwise, based on the content of the comment they were replying to. i'm not even sure there's a disagreement between you and i. maybe not even between me and sneak. i kinda wonder if all of us here just have a different number in mind for what qualifies as "mostly quite efficient". i'm completely confident we have at least 2x perf gains sitting around in just about any computational area that's presently dominated by hardware costs (e.g. HPC, but not e.g. packet-switching/networking). 10x perf gains in many of them: not by micro-optimizing things like memory allocations, but by being inventive with algorithms. that kind of gain is meaningful to me in this area: enough that i wouldn't dismiss it as "mostly quite efficient" as a rebuttal against that original "if push came to show we might just consider making better software".

baja_blast 4y ago

It's depressing how bloated and slow some software is despite running on way faster machines. The software engineering accomplished in the 90's was nothing short of astonishing.

mike_hock 4y ago
No, it wasn't. It was passable. The software "engineering" "accomplished" today is just nothing short of pathetic.

onlyrealcuzzo 4y ago

Software is bloated and slow to make it faster to develop mostly. Look at your average website with 85 megabytes of JavaScript and Bootstrap for the UI. TreeShaking in theory can get rid of all that unneeded JS - and there should be something that can do it for CSS, too. But most people aren't using it. And even if they did, there would still be a lot of bloat left over from using the libraries to make life a little bit easier. Go to average midsize startup and the amount of this going on is multiplied exponentially by the number of developers. There's not really a cure here. There's not enough engineers that know how to do things absent these frameworks - that even if you do know - it's still not worth it because the people you'll be working with probably don't.