7 ms·
Bloaty McBloatface: a size profiler for binaries
- dan353hehe 10y agoI'm just a little confused, With terabytes of disk and gigabytes of ram, why should someone spend time shrinking the size of a binary down? I understand if you are working in an environment where both of those could be limited such a tool would be extremely helpful. But I don't think I have ever decided to not use a program because of the size of the executable. I don't spend my day looking at the size of awk vs sed vs grep and then using the one that is smaller.
- 14113 10y agoOne reason to optimise is to try and get it to fit into cache. I don't know if this tool would help with that or not, but it's an excellent way to try and optimise cache residency of your binary.
- dan353hehe 10y agoThat makes sense. if the hot code path in your executable can't all be in the cache at the same time then it will have a performance penalty.
- barsonme 10y agoSome of our services are basically just a Dockerfile and an executable. Shrinking a 14MB binary to 3MB takes 10 seconds with upx and is just another step in our build process. I'd rather send or receive a 3MB file instead of a 14MB.
- legulere 10y agoSmaller code/data fits better into lower level caches, which are faster than the disk/ram. Also think of smartphones. People often complain about app sizes, because they often just have 16 or 32 GB.
- dan353hehe 10y agoFitting in the cache is important for high performance applications. I do agree that in resource constrained environments it is good to examine what it taking up space, but the exe may not be the first place to look, especially for image heavy smartphone apps.
- bigiain 10y agoConsidering where it's come from, Id guess "webscale". Assuming "spare" terrabytes of disk and gigabytes of ram breaks down at Google/Amazon/Facebook/etc scale. (I've bookmarked this, because I'm trying to work out how to fit enough of OpenCV into the spare resources on a STM32 microcontroller to be useful... I doubt the author will be too concerned about bugs/doco for _my_ use case though...)
- Peaker 10y agoYou might not care to shrink it down for 500KB to 200KB. But shrinking 200MiB binary to 40MiB is nice. See also: https://en.wikipedia.org/wiki/Wirth's_law https://en.wikipedia.org/wiki/Wirth's_law
- WalterGR 10y agoBut shrinking 200MiB binary to 40MiB is nice. Is that the average size savings?
- Normal_gaussian 10y agoYou can make that kind of saving quite easily if you are statically linking a load of stuff together. Often the extra libs have a whole set of features you don't need and didn't realise were so large.
- FnuGk 10y agoShould the linker not throw away dead code?
- phaedrus 10y agoIt depends on the linker and the command line options to the compiler and the linker. I was surprised to find out that, e.g., GCC, has this off by default and you have to pass options to enable it.
- Peaker 10y agoNormally, if you use archive (.a) files, then any .o files within them that are not needed, are thrown away. But that is at .o file granularity. There's no per-symbol dead code elimination. And if you provide a list of .o files (no .a files) then it doesn't throw away anything at all.
- ksk 10y agoHopefully so that Chrome can launch in under a second as opposed to the slow bloated mess that it currently is.
- Cerium 10y agoMy team was just asked to do a feasibility study on shrinking a 3mb application into 64kb since it would allow us to access a considerable cost savings on the next generation of hardware. Even things that can be made limitless have associated costs.
- dan353hehe 10y agowhat sort of hardware are you using? and how would shrinking the app to 64k save your team a considerable amount of money?
- ajdlinux 10y agoNot GP, but it sounds awfully like they're trying to get rid of a >3MB flash chip on the next iteration of the product they're making by moving it into a more limited amount of flash that's already on another chip, which would mean one less component and could also reduce board size.
- Cerium 10y agoYes. Exactly the situation.
- kevan 10y agoThe best example I can think of is mobile apps in severely bandwidth-constrained countries. A 10MB vs 50MB APK could make the difference between success and failure because not many people are willing to wait 30m for an app to download.
- dan353hehe 10y agoRight and I agree. however I'm sure that most of the APK would be images, etc. and not the actual executable.
- toast0 10y agoImages are big, especially if you have multiple versions for different screens, but the Java code can also be pretty large (especially if you bundle in libraries). I think the play store can split downloads so people only get appropriate native code, but if you have a download version, it needs several copies. And if you've been a good developer and localized your strings, you'll be happy to know that the strings file isn't compressed (and it's utf16)
- nitwit005 10y agoIt can be an issue even when there is plenty of space. I recall quite a bit of drama around Doom 4: https://steamcommunity.com/app/379720/discussions/0/357286119106271201/ https://steamcommunity.com/app/379720/discussions/0/35728611... But besides annoyance to customers, remember that bandwidth costs are quite large for popular products. Something like a browser or MS Office update is going to get downloaded billions of times.
- misnome 10y agoThe binary. Presumably it was the assets and not Doom.exe taking most of that 43gb.
- fnord123 10y agoOrganisations that use ball of mud releases have all the code in a large executable. This may exceed the standard ELF maximum size of 4gb. While this isn't a blocker, this is time-consuming to link, to copy to release servers, and, god forbid you break it out into shared libraries, the startup cost of linking will be unpleasant. This is also a reason behind moving from a ball mud to microservices (nee SOA).
- pjc50 10y agoI work on a WinCE environment where program download is rather slow; shrinking a 200MB debug executable to 20MB saved several minutes on every edit-rebuild-test cycle. (The solution was to make sure string deduplication was on in the linker.)
- tempodox 10y agoQuite predictably, that name made me lough out loudly. I like very much how cleanly it displays units. Since I routinely produce binary executables, I want to give it a try. It looks really useful.
- shaqbert 10y agoVincent Vanhoucke deserves a lot of credit for his naming skills... Bloaty McBloatface[1] is the best name I've seen in ages. [1]: https://en.wikipedia.org/wiki/Boaty_McBoatface https://en.wikipedia.org/wiki/Boaty_McBoatface
- geodel 10y agoAre you sure it is not like Judge Judgy McJudgerson to simply call this name best?
- tyingq 10y agoI suppose Parsey McParseface goes in the same bucket: http://www.telegraph.co.uk/technology/2016/05/13/google-names-its-latest-ai-parsey-mcparseface/ http://www.telegraph.co.uk/technology/2016/05/13/google-name...
- archgoon 10y agoWas Vincent Vanhoucke the mind behind Boaty? Or just Parsey?
- skinofstars 10y agoJames Hand, a former BBC Jersey presenter, came up with Boaty McBoatface http://www.bbc.co.uk/news/world-europe-jersey-35860760 http://www.bbc.co.uk/news/world-europe-jersey-35860760
- ryan-allen 10y agoTIL the submersibles aboard the RSS David Attenborough were actually named after the popular vote Boaty McBoatface, fantastic!
- ayuvar 10y agoThis is a great thing. I am currently tracking down a problem at work where we have a Win32 PE with a lot of DLL bloat. I've written a few hacky exploratory tools for it but nothing of this quality.
- deleted 10y ago[deleted]
- rwmj 10y agoI ran it and it gives an awful lot of "[None]": $ ~/d/bloaty/bloaty builder/virt-builder -d compileunits VM SIZE FILE SIZE -------------- -------------- 75.5% 1.96Mi [None] 3.67Mi 85.2% 8.7% 232Ki guestfs-c-actions.c 232Ki 5.3% 8.2% 219Ki guestfs.ml 219Ki 5.0% 2.0% 52.4Ki [Other] 52.4Ki 1.2% 1.3% 33.7Ki _none_ 33.7Ki 0.8% 0.7% 17.5Ki customize_cmdline.ml 17.5Ki 0.4% 0.6% 17.3Ki builder.ml 17.3Ki 0.4% 0.4% 11.8Ki customize_run.ml 11.8Ki 0.3% 0.4% 10.4Ki cmdline.ml 10.4Ki 0.2% 0.3% 7.08Ki firstboot.ml 7.08Ki 0.2% 0.2% 6.21Ki index-scan.c 6.21Ki 0.1% 0.2% 5.90Ki index_parser.ml 5.90Ki 0.1% 0.2% 5.15Ki sigchecker.ml 5.15Ki 0.1% 0.2% 4.87Ki getopt-c.c 4.87Ki 0.1% [...] It's a mixed OCaml/C executable, but I ran it on a build from the local directory and all debug symbols are still available. Edit: The README mentions this may be caused by incompletely parsed `.debug_aranges'.
- haberman 10y agoAuthor here. The quality of the reports that use debug info (like "compileunits") is entirely dependent on how complete said debug information is. There may be ways of mining the debug info more completely, to compensate for this. Theoretically .debug_aranges should be all we need for this report, but we could supplement this with info in other sections.
- dboreham 10y agoAlmost spat my latte into my keyboard reading the name. The first time I remember doing this exercise was when the product I worked on got so large it wouldn't fit on one CD. I think we found 10 copies of one of its shared libraries.
- proudofyou 10y agoLinker script fail!
- 21 10y agoXy McXface was hilarious the first time, fun the second time, but I feel that now it's just lame to name something like this.
- bigiain 10y agoDoubly so when you feel the need to explain the name in the footer...
- AnimalMuppet 10y agoGiven how close "Bloaty" is to "Boaty", this one gets a pass in my book...
- ryan-allen 10y agoIt got a good snort out of me, so I'm satisfied!
- ndesaulniers 10y agoIf you have debug symbols. Great tool, I have already used it for Android platform, and love it. But without debug symbols, you're going to have a bad time.
- deleted 10y ago[deleted]
- smnscu 10y agoI was curious so I ran bloaty against Go's hello world: bloaty main VM SIZE FILE SIZE -------------- -------------- 33.0% 536Ki __TEXT,__text 536Ki 33.7% 18.1% 293Ki __TEXT,__gopclntab 293Ki 18.5% 15.6% 253Ki __DWARF,__debug_info 253Ki 16.0% 13.1% 213Ki __TEXT,__rodata 213Ki 13.4% 6.5% 106Ki __DATA,__bss 0 0.0% 0.3% 5.62Ki [None] 93.1Ki 5.9% 4.4% 71.9Ki __DWARF,__debug_frame 71.9Ki 4.5% 4.2% 68.3Ki __DWARF,__debug_line 68.3Ki 4.3% 1.9% 30.5Ki __DWARF,__debug_pubtypes 30.5Ki 1.9% 1.1% 17.1Ki __DATA,__noptrbss 0 0.0% 0.6% 9.90Ki __DWARF,__debug_pubnames 9.90Ki 0.6% 0.6% 9.45Ki __DATA,__noptrdata 9.45Ki 0.6% 0.4% 6.34Ki __DATA,__data 6.34Ki 0.4% 0.2% 2.86Ki __TEXT,__typelink 2.86Ki 0.2% 0.0% 255 __DWARF,__debug_abbrev 255 0.0% 0.0% 72 __TEXT,__itablink 72 0.0% 0.0% 61 __DWARF,__debug_gdb_scri 61 0.0% 0.0% 48 __DWARF,__debug_aranges 48 0.0% 100.0% 1.59Mi TOTAL 1.55Mi 100.0%
- shultays 10y agoUh, what is "Mi", "Ki"? Is it Mebibyte, kibibyte? Wikipedia says correct abbreviation is MiB and KiB.