7 ms·

I'm just a little confused, With terabytes of disk and gigabytes of ram, why should someone spend time shrinking the size of a binary down? I understand if you

I'm just a little confused, With terabytes of disk and gigabytes of ram, why should someone spend time shrinking the size of a binary down? I understand if you are working in an environment where both of those could be limited such a tool would be extremely helpful. But I don't think I have ever decided to not use a program because of the size of the executable. I don't spend my day looking at the size of awk vs sed vs grep and then using the one that is smaller.

14113 10y ago
One reason to optimise is to try and get it to fit into cache. I don't know if this tool would help with that or not, but it's an excellent way to try and optimise cache residency of your binary.
- dan353hehe 10y ago
  That makes sense. if the hot code path in your executable can't all be in the cache at the same time then it will have a performance penalty.
barsonme 10y ago
Some of our services are basically just a Dockerfile and an executable. Shrinking a 14MB binary to 3MB takes 10 seconds with upx and is just another step in our build process. I'd rather send or receive a 3MB file instead of a 14MB.
legulere 10y ago
Smaller code/data fits better into lower level caches, which are faster than the disk/ram. Also think of smartphones. People often complain about app sizes, because they often just have 16 or 32 GB.
- dan353hehe 10y ago
  Fitting in the cache is important for high performance applications. I do agree that in resource constrained environments it is good to examine what it taking up space, but the exe may not be the first place to look, especially for image heavy smartphone apps.
bigiain 10y ago
Considering where it's come from, Id guess "webscale". Assuming "spare" terrabytes of disk and gigabytes of ram breaks down at Google/Amazon/Facebook/etc scale. (I've bookmarked this, because I'm trying to work out how to fit enough of OpenCV into the spare resources on a STM32 microcontroller to be useful... I doubt the author will be too concerned about bugs/doco for _my_ use case though...)
Peaker 10y ago
You might not care to shrink it down for 500KB to 200KB. But shrinking 200MiB binary to 40MiB is nice. See also: https://en.wikipedia.org/wiki/Wirth's_law https://en.wikipedia.org/wiki/Wirth's_law
- WalterGR 10y ago
  But shrinking 200MiB binary to 40MiB is nice. Is that the average size savings?
  - Normal_gaussian 10y ago
    You can make that kind of saving quite easily if you are statically linking a load of stuff together. Often the extra libs have a whole set of features you don't need and didn't realise were so large.
    - FnuGk 10y ago
      Should the linker not throw away dead code?
      - phaedrus 10y ago
        It depends on the linker and the command line options to the compiler and the linker. I was surprised to find out that, e.g., GCC, has this off by default and you have to pass options to enable it.
      - Peaker 10y ago
        Normally, if you use archive (.a) files, then any .o files within them that are not needed, are thrown away. But that is at .o file granularity. There's no per-symbol dead code elimination. And if you provide a list of .o files (no .a files) then it doesn't throw away anything at all.
ksk 10y ago
Hopefully so that Chrome can launch in under a second as opposed to the slow bloated mess that it currently is.
Cerium 10y ago
My team was just asked to do a feasibility study on shrinking a 3mb application into 64kb since it would allow us to access a considerable cost savings on the next generation of hardware. Even things that can be made limitless have associated costs.
- dan353hehe 10y ago
  what sort of hardware are you using? and how would shrinking the app to 64k save your team a considerable amount of money?
  - ajdlinux 10y ago
    Not GP, but it sounds awfully like they're trying to get rid of a >3MB flash chip on the next iteration of the product they're making by moving it into a more limited amount of flash that's already on another chip, which would mean one less component and could also reduce board size.
    - Cerium 10y ago
      Yes. Exactly the situation.
kevan 10y ago
The best example I can think of is mobile apps in severely bandwidth-constrained countries. A 10MB vs 50MB APK could make the difference between success and failure because not many people are willing to wait 30m for an app to download.
- dan353hehe 10y ago
  Right and I agree. however I'm sure that most of the APK would be images, etc. and not the actual executable.
  - toast0 10y ago
    Images are big, especially if you have multiple versions for different screens, but the Java code can also be pretty large (especially if you bundle in libraries). I think the play store can split downloads so people only get appropriate native code, but if you have a download version, it needs several copies. And if you've been a good developer and localized your strings, you'll be happy to know that the strings file isn't compressed (and it's utf16)
nitwit005 10y ago
It can be an issue even when there is plenty of space. I recall quite a bit of drama around Doom 4: https://steamcommunity.com/app/379720/discussions/0/357286119106271201/ https://steamcommunity.com/app/379720/discussions/0/35728611... But besides annoyance to customers, remember that bandwidth costs are quite large for popular products. Something like a browser or MS Office update is going to get downloaded billions of times.
- misnome 10y ago
  The binary. Presumably it was the assets and not Doom.exe taking most of that 43gb.
fnord123 10y ago
Organisations that use ball of mud releases have all the code in a large executable. This may exceed the standard ELF maximum size of 4gb. While this isn't a blocker, this is time-consuming to link, to copy to release servers, and, god forbid you break it out into shared libraries, the startup cost of linking will be unpleasant. This is also a reason behind moving from a ball mud to microservices (nee SOA).
pjc50 10y ago
I work on a WinCE environment where program download is rather slow; shrinking a 200MB debug executable to 20MB saved several minutes on every edit-rebuild-test cycle. (The solution was to make sure string deduplication was on in the linker.)

14113 10y ago

One reason to optimise is to try and get it to fit into cache. I don't know if this tool would help with that or not, but it's an excellent way to try and optimise cache residency of your binary.

dan353hehe 10y ago
That makes sense. if the hot code path in your executable can't all be in the cache at the same time then it will have a performance penalty.

barsonme 10y ago

Some of our services are basically just a Dockerfile and an executable. Shrinking a 14MB binary to 3MB takes 10 seconds with upx and is just another step in our build process. I'd rather send or receive a 3MB file instead of a 14MB.

legulere 10y ago

Smaller code/data fits better into lower level caches, which are faster than the disk/ram. Also think of smartphones. People often complain about app sizes, because they often just have 16 or 32 GB.

dan353hehe 10y ago
Fitting in the cache is important for high performance applications. I do agree that in resource constrained environments it is good to examine what it taking up space, but the exe may not be the first place to look, especially for image heavy smartphone apps.

bigiain 10y ago

Considering where it's come from, Id guess "webscale". Assuming "spare" terrabytes of disk and gigabytes of ram breaks down at Google/Amazon/Facebook/etc scale. (I've bookmarked this, because I'm trying to work out how to fit enough of OpenCV into the spare resources on a STM32 microcontroller to be useful... I doubt the author will be too concerned about bugs/doco for _my_ use case though...)

Peaker 10y ago

You might not care to shrink it down for 500KB to 200KB. But shrinking 200MiB binary to 40MiB is nice. See also: https://en.wikipedia.org/wiki/Wirth's_law https://en.wikipedia.org/wiki/Wirth's_law

WalterGR 10y ago
But shrinking 200MiB binary to 40MiB is nice. Is that the average size savings?
- Normal_gaussian 10y ago
  You can make that kind of saving quite easily if you are statically linking a load of stuff together. Often the extra libs have a whole set of features you don't need and didn't realise were so large.
  - FnuGk 10y ago
    Should the linker not throw away dead code?
    - phaedrus 10y ago
      It depends on the linker and the command line options to the compiler and the linker. I was surprised to find out that, e.g., GCC, has this off by default and you have to pass options to enable it.
    - Peaker 10y ago
      Normally, if you use archive (.a) files, then any .o files within them that are not needed, are thrown away. But that is at .o file granularity. There's no per-symbol dead code elimination. And if you provide a list of .o files (no .a files) then it doesn't throw away anything at all.

ksk 10y ago

Hopefully so that Chrome can launch in under a second as opposed to the slow bloated mess that it currently is.

Cerium 10y ago

My team was just asked to do a feasibility study on shrinking a 3mb application into 64kb since it would allow us to access a considerable cost savings on the next generation of hardware. Even things that can be made limitless have associated costs.

dan353hehe 10y ago
what sort of hardware are you using? and how would shrinking the app to 64k save your team a considerable amount of money?
- ajdlinux 10y ago
  Not GP, but it sounds awfully like they're trying to get rid of a >3MB flash chip on the next iteration of the product they're making by moving it into a more limited amount of flash that's already on another chip, which would mean one less component and could also reduce board size.
  - Cerium 10y ago
    Yes. Exactly the situation.

kevan 10y ago

The best example I can think of is mobile apps in severely bandwidth-constrained countries. A 10MB vs 50MB APK could make the difference between success and failure because not many people are willing to wait 30m for an app to download.

dan353hehe 10y ago
Right and I agree. however I'm sure that most of the APK would be images, etc. and not the actual executable.
- toast0 10y ago
  Images are big, especially if you have multiple versions for different screens, but the Java code can also be pretty large (especially if you bundle in libraries). I think the play store can split downloads so people only get appropriate native code, but if you have a download version, it needs several copies. And if you've been a good developer and localized your strings, you'll be happy to know that the strings file isn't compressed (and it's utf16)

nitwit005 10y ago

It can be an issue even when there is plenty of space. I recall quite a bit of drama around Doom 4: https://steamcommunity.com/app/379720/discussions/0/357286119106271201/ https://steamcommunity.com/app/379720/discussions/0/35728611... But besides annoyance to customers, remember that bandwidth costs are quite large for popular products. Something like a browser or MS Office update is going to get downloaded billions of times.

misnome 10y ago
The binary. Presumably it was the assets and not Doom.exe taking most of that 43gb.

fnord123 10y ago

Organisations that use ball of mud releases have all the code in a large executable. This may exceed the standard ELF maximum size of 4gb. While this isn't a blocker, this is time-consuming to link, to copy to release servers, and, god forbid you break it out into shared libraries, the startup cost of linking will be unpleasant. This is also a reason behind moving from a ball mud to microservices (nee SOA).

pjc50 10y ago

I work on a WinCE environment where program download is rather slow; shrinking a 200MB debug executable to 20MB saved several minutes on every edit-rebuild-test cycle. (The solution was to make sure string deduplication was on in the linker.)