4 ms·

It is mostly a matter of priorities. For most applications it simply does not make any sense to spend this much time on relatively small optimizations. If you

by crote 3y ago

It is mostly a matter of priorities. For most applications it simply does not make any sense to spend this much time on relatively small optimizations. If you can choose to either buy 32GiB of RAM for your server for less than $50 or spend probably over 40 hours of developer time at at least $20 / hour, it is quite obvious which one makes more sense from a business perspective. Not to mention that the website was offline for an entire week - that alone would've killed most businesses! A lot of tech people really like doing such deep dives and would happily spend years micro-optimizing even the most trivial code, but endless "yak shaving" isn't going to pay any bills. When the code runs on a trivial number of machines, it probably just isn't worth it. Not to mention that such optimizations often end up in code which is more difficult to maintain. In my opinion, a lot of "software bloat" we see these days for apps running on user machines comes from a mismatch between the developer machine and the user machine. The developer is often equipped with a high-end workstation as they simply need those resources to do their job, but they end up using the same machine to do basic testing. On the other hand, the user is running it on a five-year-old machine which was at best mid-range when they bought it. You can't really sell "we can save 150MB of memory" to your manager, but you can sell "saving 150MB of memory will make our app's performance go from terrible to borderline for 10% of users".

vincnetas 3y ago
So i guess that is the point GP is making. From practical perspective one would just spend 50$ on RAM and forget about it. But you miss opportunity to make something great in terms of algorithm improvements for example. Even if it costs you more. SO here artificial constraint is that "you cant have more RAM" and so you need to find other more creative solutions.
- uoaei 3y ago
  It also costs everyone else more to run the inefficient code that you produce.
- crote 3y ago
  But OP could have more RAM, and the end user doesn't care about how clever the algorithm is. They only care that the service is offline for a week while the developer is having fun implementing their new toy algorithm. Working on improvements like this makes a lot of sense in academia, when you are running thousands of servers, or when you need so much memory that you can't buy a big enough server. It's a nice backlog item for when there is literally nothing else to do. I have definitely fallen into this rabbit hole myself. Solving difficult problems with clever solutions is a lot of fun! But my manager - rightfully - chastised me for it because from a business PoV it was essentially a waste of time and money. Fine for a hobby project, not so much when you are trying to run a business.
  - mananaysiempre 3y ago
    > Working on improvements like this makes a lot of sense in academia, when you are running thousands of servers, or when you need so much memory that you can't buy a big enough server. It also makes sense for all desktop and mobile apps, per gnyman’s point on not hitting individual limits. Also for anything that other people will be running, like libraries. Because your thing might perform marvelously in isolation on a beefy dev machine and still run like ass on every user’s somewhat underpowered computer with ten other apps competing for RAM (hello Slack). See also: “craptop duty”[1]; how the 512K Mac reduced the competitive advantage of MS’s ruthlessly-optimized office apps but the MultiFinder, né Switcher, restored it[2]. [1] https://css-tricks.com/test-your-product-on-a-crappy-laptop/ https://css-tricks.com/test-your-product-on-a-crappy-laptop/ [2] https://www.folklore.org/StoryView.py?project=Macintosh&story=Switcher.txt https://www.folklore.org/StoryView.py?project=Macintosh&stor...
    - gnyman 3y ago
      Thanks for the links, interesting story about switcher. And yes, hearty recommendation on the craptop. I was discussing with a colleague a while back when he said he had to get one as he was unable to reproduce a problem on his M2 even with the 6x cpu throttling you can do in Chrome. On the old laptop he dug out of storage he managed to reproduce it instantly. I need to check with him how easily he could fix it, one argument I have to why some optimisation is always good is that it's easier to fix things as you go along vs having to go back and try to make things better. I mean if there is one thing making it slow then it's easy, but I have a feeling it's often a mix of things and at that point you might not be able to replace the slow parts as everything else depends on them.
dgb23 3y ago
What if runtime performance and developer performance aren’t inversely proportional? It might just be to a certain degree, we’re not actually getting any business efficiency from creating bloated and slow software? A lot of things, especially in business IT, are built on top of outdated and misleading assumptions and are leaning on patterns and norms touted as best practices. We sometimes get trapped in this belief that any form of performance improvement somehow costs us something. What if it’s baggage that we didn’t need in the first place?
- marginalia_nu 3y ago
  I say this having worked in the software business for coming on 15 years: I don't think an organization where business is calling the shots would be capable of building an Internet search engine. The entire project a fractal of this type of business-inscrutable engineering, and in any organization where engineers aren't calling the shots, that engineering isn't going to get done, and the project is going to be hideously slow and expensive as a result. In a parallel universe where I had gone the classic startup route and started out using the biggest off-the-shelf pieces I could find, gluing them together with python (instead of building a bespoke index); then thrown VC money at hardware at when it started struggling; even more when it struggled again; I'd be absolutely dumbfounded when it yet again hit a brick wall and my hardware cost is tens of thousands every month (as opposed to $100/mo now). Since I've instead built the solution from scratch, I've also built a deep understanding of both the domain and the solution, and when I'm faced with scaling problems, they're solvable in software rather than hardware. I can just change how the software works until it does. It's a slower route, but it's also able to take you places where conventional-wisdom-driven-development does not.
  - ColinHayhurst 3y ago
    Congratulations on your latest steps forward. We empathise. Resourcefulness, the ultimate asset which sparks creativity.
- crote 3y ago
  > We sometimes get trapped in this belief that any form of performance improvement somehow costs us something But it does always cost you something. Developer time isn't free, after all. If we only cared about performance, we would be handwriting SIMD intrinsics for baremetal applications. But we don't, because it is easily worth a 20% performance penalty to write code in a modern programming language. We're willing to trade 10% performance for a well-tested framework and library ecosystem which greatly reduces development time. Nobody cares how efficient your application is when it never ships. Even "bloated" and "slow" do not always mean the same thing. Just look at something like database indexes: they take up space (bloated), but make your application faster. Often that's a worthwhile tradeoff, but creating indexes for literally everything isn't a good idea either. It is all about finding the right balance. I do agree that a lot of user-facing applications have gone way too far, though. Even completely trivial Android apps are 150MB+ binaries these days, and the widespread use of memory-hogging Electron tools is a bit worrying. When your app runs on millions of devices, you should care about resource usage!
stavros 3y ago
> over 40 hours of developer time at at least $20 I think maybe you dropped a zero.
- crote 3y ago
  I intentionally underestimated the cost to provide a lower bound. In reality the listed changes once you consider all team members are likely going to run closer to 80-120 hours at $150 / hour - but that only confirms the final conclusion.
  - stavros 3y ago
    Agreed, it's a no-brainer to add some cheap hardware rather than spend two orders of magnitude more to fix the inefficient code.
Joeri 3y ago
In my opinion, a lot of "software bloat" we see these days for apps running on user machines comes from a mismatch between the developer machine and the user machine. The developer is often equipped with a high-end workstation as they simply need those resources to do their job, but they end up using the same machine to do basic testing. Incidentally, I think the reason they need those specs is the same: the people building the dev tools all have top end hardware, and what’s fast enough for them is good enough to ship. I don’t think the people building the dev tools at meta, or apple, or google are seriously considering the use case of a developer working on an old dual core 8 gb machine, but that’s the reality in large parts of the world.
- crote 3y ago
  To a certain extent, yes. On the other hand, development tools simply require a lot of resources: a debug build is always going to be more demanding than a production build, and a full recompilation of something like Firefox is never going to be fast. I reckon a lot of developers are more than happy to sacrifice 8GB of RAM for near-instant advanced autocomplete and typechecking, but we should keep in mind that it should not become a requirement to do basic development.
- marcosdumay 3y ago
  Oh, I agree that this is the cause, but no thanks. I will keep using the heavyweight development tools that run through all of my code discovering problems, analyzing its quality, changing it into more performant code, deducting all kinds of new code to complete my high-level view, and doing all kinds of the modern benefits that use my computer to replace my work. I can get behind testing the resulting executable on a low-power machine. But not on using it for everything.

by crote 3y ago

vincnetas 3y ago

So i guess that is the point GP is making. From practical perspective one would just spend 50$ on RAM and forget about it. But you miss opportunity to make something great in terms of algorithm improvements for example. Even if it costs you more. SO here artificial constraint is that "you cant have more RAM" and so you need to find other more creative solutions.

uoaei 3y ago
It also costs everyone else more to run the inefficient code that you produce.
crote 3y ago
But OP could have more RAM, and the end user doesn't care about how clever the algorithm is. They only care that the service is offline for a week while the developer is having fun implementing their new toy algorithm. Working on improvements like this makes a lot of sense in academia, when you are running thousands of servers, or when you need so much memory that you can't buy a big enough server. It's a nice backlog item for when there is literally nothing else to do. I have definitely fallen into this rabbit hole myself. Solving difficult problems with clever solutions is a lot of fun! But my manager - rightfully - chastised me for it because from a business PoV it was essentially a waste of time and money. Fine for a hobby project, not so much when you are trying to run a business.
- mananaysiempre 3y ago
  > Working on improvements like this makes a lot of sense in academia, when you are running thousands of servers, or when you need so much memory that you can't buy a big enough server. It also makes sense for all desktop and mobile apps, per gnyman’s point on not hitting individual limits. Also for anything that other people will be running, like libraries. Because your thing might perform marvelously in isolation on a beefy dev machine and still run like ass on every user’s somewhat underpowered computer with ten other apps competing for RAM (hello Slack). See also: “craptop duty”[1]; how the 512K Mac reduced the competitive advantage of MS’s ruthlessly-optimized office apps but the MultiFinder, né Switcher, restored it[2]. [1] https://css-tricks.com/test-your-product-on-a-crappy-laptop/ https://css-tricks.com/test-your-product-on-a-crappy-laptop/ [2] https://www.folklore.org/StoryView.py?project=Macintosh&story=Switcher.txt https://www.folklore.org/StoryView.py?project=Macintosh&stor...
  - gnyman 3y ago
    Thanks for the links, interesting story about switcher. And yes, hearty recommendation on the craptop. I was discussing with a colleague a while back when he said he had to get one as he was unable to reproduce a problem on his M2 even with the 6x cpu throttling you can do in Chrome. On the old laptop he dug out of storage he managed to reproduce it instantly. I need to check with him how easily he could fix it, one argument I have to why some optimisation is always good is that it's easier to fix things as you go along vs having to go back and try to make things better. I mean if there is one thing making it slow then it's easy, but I have a feeling it's often a mix of things and at that point you might not be able to replace the slow parts as everything else depends on them.

dgb23 3y ago

What if runtime performance and developer performance aren’t inversely proportional? It might just be to a certain degree, we’re not actually getting any business efficiency from creating bloated and slow software? A lot of things, especially in business IT, are built on top of outdated and misleading assumptions and are leaning on patterns and norms touted as best practices. We sometimes get trapped in this belief that any form of performance improvement somehow costs us something. What if it’s baggage that we didn’t need in the first place?

marginalia_nu 3y ago
I say this having worked in the software business for coming on 15 years: I don't think an organization where business is calling the shots would be capable of building an Internet search engine. The entire project a fractal of this type of business-inscrutable engineering, and in any organization where engineers aren't calling the shots, that engineering isn't going to get done, and the project is going to be hideously slow and expensive as a result. In a parallel universe where I had gone the classic startup route and started out using the biggest off-the-shelf pieces I could find, gluing them together with python (instead of building a bespoke index); then thrown VC money at hardware at when it started struggling; even more when it struggled again; I'd be absolutely dumbfounded when it yet again hit a brick wall and my hardware cost is tens of thousands every month (as opposed to $100/mo now). Since I've instead built the solution from scratch, I've also built a deep understanding of both the domain and the solution, and when I'm faced with scaling problems, they're solvable in software rather than hardware. I can just change how the software works until it does. It's a slower route, but it's also able to take you places where conventional-wisdom-driven-development does not.
- ColinHayhurst 3y ago
  Congratulations on your latest steps forward. We empathise. Resourcefulness, the ultimate asset which sparks creativity.
crote 3y ago
> We sometimes get trapped in this belief that any form of performance improvement somehow costs us something But it does always cost you something. Developer time isn't free, after all. If we only cared about performance, we would be handwriting SIMD intrinsics for baremetal applications. But we don't, because it is easily worth a 20% performance penalty to write code in a modern programming language. We're willing to trade 10% performance for a well-tested framework and library ecosystem which greatly reduces development time. Nobody cares how efficient your application is when it never ships. Even "bloated" and "slow" do not always mean the same thing. Just look at something like database indexes: they take up space (bloated), but make your application faster. Often that's a worthwhile tradeoff, but creating indexes for literally everything isn't a good idea either. It is all about finding the right balance. I do agree that a lot of user-facing applications have gone way too far, though. Even completely trivial Android apps are 150MB+ binaries these days, and the widespread use of memory-hogging Electron tools is a bit worrying. When your app runs on millions of devices, you should care about resource usage!

stavros 3y ago

> over 40 hours of developer time at at least $20 I think maybe you dropped a zero.

crote 3y ago
I intentionally underestimated the cost to provide a lower bound. In reality the listed changes once you consider all team members are likely going to run closer to 80-120 hours at $150 / hour - but that only confirms the final conclusion.
- stavros 3y ago
  Agreed, it's a no-brainer to add some cheap hardware rather than spend two orders of magnitude more to fix the inefficient code.

Joeri 3y ago

In my opinion, a lot of "software bloat" we see these days for apps running on user machines comes from a mismatch between the developer machine and the user machine. The developer is often equipped with a high-end workstation as they simply need those resources to do their job, but they end up using the same machine to do basic testing. Incidentally, I think the reason they need those specs is the same: the people building the dev tools all have top end hardware, and what’s fast enough for them is good enough to ship. I don’t think the people building the dev tools at meta, or apple, or google are seriously considering the use case of a developer working on an old dual core 8 gb machine, but that’s the reality in large parts of the world.

crote 3y ago
To a certain extent, yes. On the other hand, development tools simply require a lot of resources: a debug build is always going to be more demanding than a production build, and a full recompilation of something like Firefox is never going to be fast. I reckon a lot of developers are more than happy to sacrifice 8GB of RAM for near-instant advanced autocomplete and typechecking, but we should keep in mind that it should not become a requirement to do basic development.
marcosdumay 3y ago
Oh, I agree that this is the cause, but no thanks. I will keep using the heavyweight development tools that run through all of my code discovering problems, analyzing its quality, changing it into more performant code, deducting all kinds of new code to complete my high-level view, and doing all kinds of the modern benefits that use my computer to replace my work. I can get behind testing the resulting executable on a low-power machine. But not on using it for everything.