12 ms·
Why is a Rust executable large? (2016)
- new4thaccount 7y agoAlso, the author mentions this is in the Rust FAQ (https://prev.rust-lang.org/en-US/faq.html https://prev.rust-lang.org/en-US/faq.html) now under the following question: Why do Rust programs have larger binary sizes than C programs?
- jp3141 7y agohttps://prev.rust-lang.org/en-US/faq.html#why-do-rust-programs-have-larger-binary-sizes-than-C-programs https://prev.rust-lang.org/en-US/faq.html#why-do-rust-progra...
- masklinn 7y agoOne of the points has since been removed from reality: jemalloc was removed and the default allocator switched to system in 1.32 (January 2019)[0]. [0] https://github.com/rust-lang/rust/blob/master/RELEASES.md#compiler-2 https://github.com/rust-lang/rust/blob/master/RELEASES.md#co...
- Kurtz79 7y agoI find interesting though that the Rust code that generates a larger binary is about as complex and lengthy as the C code, while the Rust code which generates a smaller binary is clearly clunky and nobody would ever use it. Of course it is a toy example and I imagine in most real-world examples the size of the binary is a moot point.
- simias 7y agoI believe that rust now defaults to the system's malloc implementation instead of bundling jemalloc, so that bit is no longer necessary: #![feature(alloc_system)] extern crate alloc_system; As for the part that removes libstd it's how you'd generally do it for a bare metal program (i.e. bootloader/OS) since you can't really use it without an OS unless you manually reimplement all sorts of primitives (memory allocations being the most obvious one). Now if you do use libstd it's true that it's significantly larger that libc, but it's also vastly more powerful. You don't have anything like String, Vec or iterators in libc for instance, just a bunch of relatively low-level functions and wrappers around syscalls. It's also a fixed cost, so obviously for a simple Hello World there's a massive overhead but for a more complex application it should be less noticeable. I mean when you think about it even the C Hello World is ridiculously bloated. If you were to write the equivalent program in assembly without any dependency on the libc you could probably get a binary that would be significantly smaller (and most of its size would really be the ELF metadata).
- chaosite 7y agoWell, if you're going to go that route, you in fact can make an ELF exectuable that's smaller than the size of the ELF header (under Linux anyway) https://www.muppetlabs.com/~breadbox/software/tiny/teensy.html https://www.muppetlabs.com/~breadbox/software/tiny/teensy.ht...
- PudgePacket 7y agoThe smaller, similar to C code uses println, which is a macro, and expands out at compile time to be bigger than the clunky hand written code.
- snvzz 7y agotl;dr bloat.
- pornel 7y agoThe real answer is that they're not. It's just that C has a head start of having a 10MB libc already installed on your system, and Rust doesn't. If you compile C with libc statically linked in, it'll make executables as large as Rust's. If you compile Rust for dynamic linking, it'll give you hello world as small as you get from C.
- mahkoh 7y ago"If you compile C with libc statically linked in, it'll make executables as large as Rust's." $ cat test.c && musl-gcc -static -O2 test.c && echo && du -h a.out #include <stdio.h> int main(void) { printf("Hello World\n"); } 20K a.out
- pornel 7y agoI stand corrected. Congrats to MUSL!
- saagarjha 7y agoI would also imagine that the compiler is performing a couple of optimizations that would significantly reduce the resulting binary’s size: the call to printf would likely resolve to puts, which might be further optimized to a raw write call by a smart enough compiler.
- wbl 7y agoAnd possible dead code elimination in the linker.
- wdroz 7y agoWith an "old" gcc I use at work: echo $(gcc --version) && cat test.c && gcc -static -O2 test.c && echo && du -h a.out gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 ... #include <stdio.h> int main(void) { printf("Hello World\n"); } 796K a.out
- 7y ago
- nineteen999 7y agoThis is a good write up from the point of view of a C/C++ programmer, since it gives a fair breakdown of where that extra space is going and rightly points out that statically linked C/C++ executables are going to be large as well. It's also a fun tour of how to access the syscall interface directly from Rust and to perform optimizations that most C and C++ programmers wouldn't ever perform perhaps outside of the most space constrained embedded projects. I take a minor issue with the handwaving away (not just in this article but with others as well) of glibc and the suggestion that it can just be replaced with musl or another libc. There's a reason that glibc doesn't officially support static linking, and that is NSS, and PAM. If all you need is a static executable for your Docker container or whatever that reads user/group information and authenticates out of flat files in /etc, then go for it. But in the "enterprise" space things like LDAP, Active Directory, 2FA etc. are real, and if your application needs to support those, then you're going to need glibc and its dependency on dlopen() and friends. And this goes for every language which has a dependency on your chosen libc as well (which let's face it, is a large majority at least in the Linux world), if you want to use NSS and PAM modules.
- majewsky 7y ago> If all you need is a static executable for your Docker container or whatever that reads user/group information and authenticates out of flat files in /etc, then go for it. If you're using a Docker image, then that's your statically-linked binary, for all intents and purposes. It won't change sizes much if you link libc statically or dynamically (esp. if you're running in an Alpine container with musl).
- nineteen999 7y agoSubstitute Docker for another container technology, or even "IOT device" in my comment if that helps. The point i'm making is not about the size of the binary, size or type of container that it runs in (if at all), but about the features it can/will support if you rip out glibc and replace it in a fit of rage with musl or similar which seems to be somewhat of a fashion at the moment.
- nwmcsween 7y ago
- lifthrasiir 7y agoWow, I wrote this 3 years ago and I feel nostalgic. Nowadays I consider this to be obsoleted, as the official Q&A has the exactly same question and many things (especially jemalloc) have been changed in the Rust world. I personally found the HN discussion at that time [1] was greatly fruitful. [1] https://news.ycombinator.com/item?id=11823949 https://news.ycombinator.com/item?id=11823949
- klmr 7y agoUnfortunately that link to the Rust FAQ is broken.
- maddening 7y agoThey removed the FAQ as it didn't fit they newer site. You can find it at the old site version: https://prev.rust-lang.org/en-US/faq.html https://prev.rust-lang.org/en-US/faq.html
- nickcw 7y ago> I didn’t mention this because it doesn’t improve such a small program, but you can also try UPX and other executable compressors if you are working with a much larger application. I don't recommend this as it will increase the memory used by your program which is usually more important than how much disk space it takes up. OSes only page in the bits of the executable that is actually run, however when upx decompresses the binary it has to load it all into RAM. That RAM can be swapped out but for a normal executable backed by a file the kernel would just drop pages from the file so no write IO needed. In my tests with rclone, using upx made the binary 31% of the size, but made it use 42% more RAM. YMMV!
- lifthrasiir 7y agoWhile you are technically correct, nowadays the executable file size is minuscule compared to the available memory. Since upx uses the UCL algorithm which requires no additional memory besides the target buffer AFAIK, the memory increase can only be attributed to the original file size---I think rclone weighs about 20 MBs, so it is probably the case that rclone unusually consumes less memory! :p The real problem with upx is the invocation latency, which is a big blocker when your program starts a lot (common for CLIs). Executable compression has been much more common in Windows because the download size matters there and programs tend to last much longer.
- ChrisSD 7y agoI've found that UPX triggers AntiVirus programs more often than uncompressed binaries on Windows.
- edflsafoiewq 7y agoBoth executables have grown on my machine since 2016. On Linux, no optimization or stripping * C Hello World - 19K * Rust Hello World - 1.6M (!) The C gain is attributable to a change to ld: it puts read-only data and executable data in separate segments by default as a security hardening measure since ld 2.30. I don't see why this shouldn't produce three segments (R, RE, RW), but here I get four (R, RE, R, RW). (Anyone know why four?) If I pass -znoseparate-code to disable this the C goes back to the 8K shown in the article. (No effect on the Rust). The Rust gain appears to be mostly in debug info. Stripped I get 187K, which is similar to 121K the article gives after stripping and removing jemalloc (which isn't in by default anymore). Does anyone know why Rust has grown so much from 2016 to 2019?
- dathinab 7y agoDid you compile with the --release flag? Without it will build a debug build and put all the debug info in there (which mainly due to println is a lot).
- edflsafoiewq 7y agoIt doesn't make any difference for something as small as hello world (you can try it). A release build might not have debug info in the sense that -C debuginfo=0, but it still has plenty of stuff that can be stripped. You also get the same size if you bypass the println machinery with io::stdout().write_all(b"Hello, world!").
- Grollicus 7y ago$ cargo --version && cargo init hello && cd hello && cat ./src/main.rs && cargo build --release && du -h ./target/release/hello cargo 1.34.0 (6789d8a0a 2019-04-01) Created binary (application) package fn main() { println!("Hello, world!"); } Compiling hello v0.1.0 (/tmp/foo/hello) Finished release [optimized] target(s) in 0.31s 1,6M ./target/release/hello Edit: $ strip target/release/hello && du -h ./target/release/hello 192K ./target/release/hello
- 7y ago
- saagarjha 7y ago> it somehow aborted. Probably a libbacktrace issue, I don’t know, but that doesn’t harm much anyway. Are you talking about the SIGILL? That’s probably a ud2 inside of panic, which presumably tells LLVM it will not return and causes this instruction to be placed there.
- titzer 7y agoIt always amazes me when new languages start out and their implementations include a huge pile of crap by default. Virgil is different, it only includes the stuff which is reachable from main(). In fact the entire compiler is organized around only compiling reachable code into the binary. Even including the entire runtime and garbage collector, about (8KiB), there isn't much smaller you can get: HelloWorld.v3: def main(a: Array<string>) { System.puts("Hello World!\n"); } -rwxr-xr-x 1 titzer wheel 80 May 16 13:59 HelloWorld-jar -rwxr-xr-x 1 titzer wheel 9088 May 16 13:59 HelloWorld-x86-darwin -rwxr-xr-x 1 titzer wheel 5744 May 16 13:58 HelloWorld-x86-darwin-nogc -rwxr-xr-x 1 titzer wheel 4132 May 16 13:58 HelloWorld-x86-darwin-nort -rwxr-xr-x 1 titzer wheel 4692 May 16 13:59 HelloWorld-x86-linux-nogc -rwxr-xr-x 1 titzer wheel 340 May 16 13:59 HelloWorld-x86-linux-nort -rw-r--r-- 1 titzer wheel 6309 May 16 13:59 HelloWorld.jar -rw-r--r-- 1 titzer wheel 63 May 16 13:57 HelloWorld.v3 -rw-r--r-- 1 titzer wheel 3486 May 16 14:04 HelloWorld.wasm -rw-r--r-- 1 titzer wheel 256 May 16 14:46 HelloWorld-nogc.wasm (the executable files are, well, the executables). The Linux executable without runtime or GC is literally 340 bytes, the wasm executable without GC is 256 bytes. The x86-darwin-nort binary is large because apparently Mach-O executables don't work right unless they are at least one 4KiB page in size.
- rmu09 7y agoI wanted to check that out, but it seems the homepage at http://compilers.cs.ucla.edu/virgil/ http://compilers.cs.ucla.edu/virgil/ is broken, the webserver doesn't include the server side includes (e.g. linkbar.inc, footer.inc)
- titzer 7y agohttps://github.com/titzer/virgil https://github.com/titzer/virgil
- Narishma 7y agoWhy was this downvoted?
- ChrisSD 7y agoWhile comparing the sizes of "hello world" programs is cute, how do real world small utilities compare?
- JohnFen 7y agoThat's an interesting breakdown. The size of Rust binaries is probably the largest reason why Rust is not terribly compelling for me. Seeing that it's possible to make that more reasonable is enlightening!
- pslam 7y agoI have routinely created firmware written in Rust which are just a handful of KB. The issues people are having are related to system-integration, such as static linkage, can be mitigated in cases anyone really cares about, and will disappear over time.
- monocasa 7y agoFor those writing embedded firmware, one of the subpoints of a talk I did recently (about a rust dev kit for the Nintendo 64) was that you want to avoid a standard model where in C and C++ you might have most of your code in a static library and build the executable as a second step. Instead you really want all of your code including your asm in rust source files so it's as painless as possible to generate an LTOed binary. I had binary sizes go from ~1MB to about ~70KB. Rust really depends on LTO if you care about binary size.
- StrangeDoctor 7y agoWas this talk recorded anywhere? it sounds really interesting.
- monocasa 7y agoNo, unfortunately. I really just need to take the content and stick it in a blog, but life gets in the way...
- naikrovek 7y agoSo kick life's ass, tell it to back off, and blog that info somewhere! It would be good info for the rest of us, and it would be great if you could chronicle that stuff. (Don't tell Life I said to kick it's ass, please. It will crush me.)
- RobotCaleb 7y agoHi! I think I was at your talk. In Denver? I asked about emulation.
- monocasa 7y agoHey! Yeah, that was me. Sorry for being so rambley; I hope that you still enjoyed it. https://github.com/n64dev/cen64 https://github.com/n64dev/cen64 is the emulator that I use
- codedokode 7y agoThe author doesn't even understand what's wrong with large binaries and writes nonsence about Internet connection speed. Main problem is that statically built libraries are not shareable. For example, if you have two Electron applications, each with its own copy of Chrome, then every Chrome will take about 100 Mb only for code section (not counting data, stack which typically will be much larger) and together they take 200 Mb. But if those applications didn't ship their own copy of Chrome and used the same libraries then the code would be shared and took only 100 Mb of RAM. That is, 100 millions of bytes of RAM are saved. The more RAM such applications take, the higher is probability that the system will get out of RAM and start swapping. And in this case the system will become so slow that it won't matter what kind of Internet connection you have. The problem of non-shareable code is also actual for interpreted languages like Javascript, Python or Java. When running such applications, their code and byte-code generated from it often are not shareable. So if you run two copies of desktop JS application, that uses some JS library, you can get two copies of this library in memory. As I remember, Android uses some clever trick that allows sharing compiled Java code. Most other interpreted languages cannot do it. Also, a fun fact: if you are using Gnome-shell on Wayland and the system starts swapping, computer can stop responding even to mouse movement, and you won't be able even to switch to other VT.
- monocasa 7y agoBut part of the point of electron is a fixed base that you control. You only have to validate against one specific version of electron. Forcing people to use the same electron is directly against part of the value proposition to using it in the first place. Like, I say this as someone who hates the electron-ification of the desktop. Additionally there's some pretty good arguments when you're not in a dual interpreted/compiled world and can LTO fully like a pure Rust app is, that the benefits of having everything shared between processes are outweighed by stripping everything out that isn't necessary.
- ncmncm 7y agoThe REAL answer is still the same as in 2016: the language is not mature yet. It will be fixed, in time, after other things that matter more. Binary size has less priority than important goals such as feature completeness, compiler speed, optimization performance, target platform coverage, performance tooling support, library coverage, and myriad others. Memory has become cheap enough that excess binary size is not typically the limiting factor preventing deployment, where that happens. As the language matures, numerous impediments to industrial deployment will fall, one at a time. The language and its ecosystem are maturing with impressive, even stunning rapidity, but it will still be ten years before the language is an obviously safe choice for any random project -- if it gets there at all. Historically, odds are against that, but the only real risks for Rust are whether its adoption rate can be grown fast enough to retain relevance, and whether its development jumps some unforeseen shark not easily recovered from.