7 ms·

> Just to be cautious: Is it a valid question to ask after all? We have hundreds of gigabytes of storage, if not some terabytes, and people should be using dece

> Just to be cautious: Is it a valid question to ask after all? We have hundreds of gigabytes of storage, if not some terabytes, and people should be using decent ISPs nowadays, so the binary size should not be a concern, right? $ find /usr/bin -type f | wc -l 2254 $ du -sh /usr/bin 445M /usr/bin If all these programs were written in Rust and statically compiled, assuming only a 600K difference by binary, that would make my /usr/bin 1.3G (or 300%) larger. But in reality all those programs are dynamically linked against many libraries in /usr/lib, so the difference would be even bigger, with libraries duplicated between all those programs. Sure, you can dynamically link with Rust too, but then, you hit the other problem that there is no stable ABI (yet?), and that upgrading Rust (every 6 weeks) means recompiling everything.

jleahy 10y ago
Not necessarily. You could go the way of busybox or uutils and have a single binary with many hard links. So 'ls', 'wc', 'grep', etc can all point at a single executable which dispatches to different functionality based on argv[0]. Then you can even share code between the binaries, which should make them even smaller.
- glandium 10y ago
  ls and grep are in /bin on my system. wc, coming from coreutils is in /usr/bin. But that's really only a small part of /usr/bin. $ dpkg -L coreutils | grep -c /usr/bin 76 Are you suggesting that all of e.g. GNOME should be a single program?
- cgag 10y ago
  Why hard links over symlinks?
  - hueving 10y ago
    So someone doesn't unlink the main binary when they are cutting unused utils to save space.
yazaddaruvala 10y ago
Forgetting about Rust for a second. Talking about dynamically linked libs in general. Dynamic linking was an optimization which came about when memory was expensive. Memory is no longer expensive. Is 1.3GB (or even 13GB) a lot on your hardware[1]? According to "Do not prematurely optimize": Pretending we never had dynamic linking, and given today's hardware constraints[2], as a community would we choose to reimplement and universally adopt this optimization? [1] Keep in mind that a single "modern" application, on average, weighs in the 10s of MBs or GBs. [2] I'm talking about the general case, for the majority of OS distributions, ignoring the relatively exceptional case of embedded systems, which do in-fact still need it.
- Retra 10y ago
  Yes, we would. Not because it's 'premature optimization', but because it's easy optimization. "Do not prematurely optimize" is not a software design rule, it's a time-management rule. Dynamic linking has software design impacts.
  - kbenson 10y ago
    > "Do not prematurely optimize" is not a software design rule, it's a time-management rule. No, it's both. Optimization often affects the cost of later decisions, and the reason not to prematurely optimize is because it can easily take you to a local optimum which is not very optimal at all. This is a perfect example of that, as the GP comment point out. If memory is not constrained as it was when this trade-off became common, it may not have become prevalent. Static binaries are faster (the degree to which depending on a lot of factors), while dynamic binaries are smaller on disk and in memory, if the shared libraries are already used elsewhere. Modern optimizations at the OS level for forking and threading should make consideration of those negligible.
    - Retra 10y ago
      Dynamic linking wasn't invented as a premature optimization though. And if it didn't exist today, it would still not be premature to invent it, because dynamic linking does not only concern how large and fast your program is, but also how it is interacted with. So here is my point: optimization that can affect the relevant interfaces of your software is not premature because deciding on the interfaces your software exposes is not premature.
fpgaminer 10y ago
I would gladly sacrifice that disk space in exchange for system utilities written in Rust. We're still finding vulnerabilities in core utilities, even after all these decades.
- Aoreias 10y ago
  Most recent vulnerabilities in core utilities really don't have a lot to do with memory safety though - Shellshock & Imagemagick were input sanitization, other common ones though are still injection vulnerabilities or authentication weaknesses. Heartbleed excluded most major vulnerabilities these days aren't related to memory safety.
  - fpoling 10y ago
    Heartbleed is not a real memory safety bug when program reads beyond allocated memory. It is more of improper reuse of previously allocated buffer and could exist in safe Rust just as well.
    - rkangel 10y ago
      You're right, there isn't a classic simple buffer overrun that Rust would trivially catch, but you're missing two things: 1) The problem was really sending back uninitialised memory. In Rust you can't have uninitialised memory. The oversize allocated buffer would have to have initialisation data passed in (possibly zeroes) 2) You'd never write the Rust code like that anyway. The abstractions avaialble mean that you aren't separating the content of some data and the length to pass to allocators.
  - robryk 10y ago
    Well, there are for example all those libbfd-related issues that caused running strings(!) on unstrusted files to be unsafe.
  - fpgaminer 10y ago
    Sure, but Rust isn't just about dealing with memory safety. The language also lends itself well to solving other common mistakes by virtue of its design and by being built on modern principals. Idiomatic C promotes throwing around pointers/arrays and hoping that the next coder that comes along to consume a struct reads the docs/header and understands how the data in that struct is supposed to be used. Idiomatic Rust uses its type system to strictly enforce how a struct and its data can/should be used. It's a world of difference and results in drastically less bugs. Not to mention the rest of the Rust ecosystem works in harmony with the language to further reduce bugs. Testing as a first class citizen of the language and its tooling is one of the big ones. That isn't to say that you can't do something similar in C, but it is an order of magnitude more challenging to design a "module" in C that is explicit and robust compared to the effort to do the same in Rust. I've coded my fair share of cryptographic systems in both C and Rust. Bulletproof C is just _exhausting_ to code and work with. The same kind of code in Rust is, dare I say, fun to write. It's just a joy to use Rust's type system to enforce rules and invariants, and then codify those rules in the documentation comments above the structs/functions, and then have "cargo test" actually run the code in that documentation automatically to check it for validity. And yes, as you point out, some of the big bugs lately have been logic bugs resulting not necessarily from poor code but from poor design. Thing is, the less mental capacity a language requires from a coder the more mental capacity that coder has to use for thinking about the application logic. i.e. in C when you get a string you have to think about how to handle the UTF-8 encoding and what to do about path names that somehow ended up with a non UTF-8 character, and whether the string is NULL terminated or pascal, and is memmove (src, dst), or (dst, src)? In Rust, well, that's all handled, so you think about what the string actually means and, hopefully, you'll realize that hey you should probably sanitize that string so it can't be used to gain shell access from an SVG file.

jleahy 10y ago

Not necessarily. You could go the way of busybox or uutils and have a single binary with many hard links. So 'ls', 'wc', 'grep', etc can all point at a single executable which dispatches to different functionality based on argv[0]. Then you can even share code between the binaries, which should make them even smaller.

glandium 10y ago
ls and grep are in /bin on my system. wc, coming from coreutils is in /usr/bin. But that's really only a small part of /usr/bin. $ dpkg -L coreutils | grep -c /usr/bin 76 Are you suggesting that all of e.g. GNOME should be a single program?
cgag 10y ago
Why hard links over symlinks?
- hueving 10y ago
  So someone doesn't unlink the main binary when they are cutting unused utils to save space.

yazaddaruvala 10y ago

Forgetting about Rust for a second. Talking about dynamically linked libs in general. Dynamic linking was an optimization which came about when memory was expensive. Memory is no longer expensive. Is 1.3GB (or even 13GB) a lot on your hardware[1]? According to "Do not prematurely optimize": Pretending we never had dynamic linking, and given today's hardware constraints[2], as a community would we choose to reimplement and universally adopt this optimization? [1] Keep in mind that a single "modern" application, on average, weighs in the 10s of MBs or GBs. [2] I'm talking about the general case, for the majority of OS distributions, ignoring the relatively exceptional case of embedded systems, which do in-fact still need it.

Retra 10y ago
Yes, we would. Not because it's 'premature optimization', but because it's easy optimization. "Do not prematurely optimize" is not a software design rule, it's a time-management rule. Dynamic linking has software design impacts.
- kbenson 10y ago
  > "Do not prematurely optimize" is not a software design rule, it's a time-management rule. No, it's both. Optimization often affects the cost of later decisions, and the reason not to prematurely optimize is because it can easily take you to a local optimum which is not very optimal at all. This is a perfect example of that, as the GP comment point out. If memory is not constrained as it was when this trade-off became common, it may not have become prevalent. Static binaries are faster (the degree to which depending on a lot of factors), while dynamic binaries are smaller on disk and in memory, if the shared libraries are already used elsewhere. Modern optimizations at the OS level for forking and threading should make consideration of those negligible.
  - Retra 10y ago
    Dynamic linking wasn't invented as a premature optimization though. And if it didn't exist today, it would still not be premature to invent it, because dynamic linking does not only concern how large and fast your program is, but also how it is interacted with. So here is my point: optimization that can affect the relevant interfaces of your software is not premature because deciding on the interfaces your software exposes is not premature.

fpgaminer 10y ago

I would gladly sacrifice that disk space in exchange for system utilities written in Rust. We're still finding vulnerabilities in core utilities, even after all these decades.

Aoreias 10y ago
Most recent vulnerabilities in core utilities really don't have a lot to do with memory safety though - Shellshock & Imagemagick were input sanitization, other common ones though are still injection vulnerabilities or authentication weaknesses. Heartbleed excluded most major vulnerabilities these days aren't related to memory safety.
- fpoling 10y ago
  Heartbleed is not a real memory safety bug when program reads beyond allocated memory. It is more of improper reuse of previously allocated buffer and could exist in safe Rust just as well.
  - rkangel 10y ago
    You're right, there isn't a classic simple buffer overrun that Rust would trivially catch, but you're missing two things: 1) The problem was really sending back uninitialised memory. In Rust you can't have uninitialised memory. The oversize allocated buffer would have to have initialisation data passed in (possibly zeroes) 2) You'd never write the Rust code like that anyway. The abstractions avaialble mean that you aren't separating the content of some data and the length to pass to allocators.
- robryk 10y ago
  Well, there are for example all those libbfd-related issues that caused running strings(!) on unstrusted files to be unsafe.
- fpgaminer 10y ago
  Sure, but Rust isn't just about dealing with memory safety. The language also lends itself well to solving other common mistakes by virtue of its design and by being built on modern principals. Idiomatic C promotes throwing around pointers/arrays and hoping that the next coder that comes along to consume a struct reads the docs/header and understands how the data in that struct is supposed to be used. Idiomatic Rust uses its type system to strictly enforce how a struct and its data can/should be used. It's a world of difference and results in drastically less bugs. Not to mention the rest of the Rust ecosystem works in harmony with the language to further reduce bugs. Testing as a first class citizen of the language and its tooling is one of the big ones. That isn't to say that you can't do something similar in C, but it is an order of magnitude more challenging to design a "module" in C that is explicit and robust compared to the effort to do the same in Rust. I've coded my fair share of cryptographic systems in both C and Rust. Bulletproof C is just _exhausting_ to code and work with. The same kind of code in Rust is, dare I say, fun to write. It's just a joy to use Rust's type system to enforce rules and invariants, and then codify those rules in the documentation comments above the structs/functions, and then have "cargo test" actually run the code in that documentation automatically to check it for validity. And yes, as you point out, some of the big bugs lately have been logic bugs resulting not necessarily from poor code but from poor design. Thing is, the less mental capacity a language requires from a coder the more mental capacity that coder has to use for thinking about the application logic. i.e. in C when you get a string you have to think about how to handle the UTF-8 encoding and what to do about path names that somehow ended up with a non UTF-8 character, and whether the string is NULL terminated or pascal, and is memmove (src, dst), or (dst, src)? In Rust, well, that's all handled, so you think about what the string actually means and, hopefully, you'll realize that hey you should probably sanitize that string so it can't be used to gain shell access from an SVG file.