8 ms·
How Swift achieved dynamic linking where Rust couldn't (2019)
- rrll22 6y agoSo no surprise Rust is faster. edit nvm I thought dynamic linking slowed things down since static linking means code fits better in the cache. Looking at the points, it seems that either dyanmic linking makes code faster or people prefer slower dynamic code.
- auggierose 6y agoAnd I say you are a low down rusty coder.
- kungito 6y agoFrom reading the first few pages I get triggered because the whole "where Rust couldn't" feels like mentioning Rust in the title to piggy back on Rust's rising popularity. I personally don't find Swift interesting in any way while Rust revolutionized GCless (and GCless parallelized) typed programming.
- gpm 6y agoGankra was heavily involved in low level rust plumbing such as linking, ABIs, unsafe code, and so on. Comparing to rust is both speaking to the other similar language she has experience with, and speaking to many of the people who were likely to read this article (those people being from the rust community). This article also really does achieve the goal in the third paragraph (at least when read by the right audience) > Also some folks like to complain that Rust doesn't bother with ABI stability, and I think looking at how Swift does helps elucidate why that is.
- millstone 6y agoNote Gankra is “she” not “he”
- tux3 6y agoI care about Rust (mostly due to using it a lot), so I really hope more people spend the time to criticize it, and dig up the areas that need love and care compared to other languages. But in this particular case, you should probably know that the author is not some nobody trying to piggyback on Rust's popularity :) They're a pretty well known contributor, in fact.
- steveklabnik 6y agoGankra worked on both Rust and Swift, and additionally wrote several very beloved Rust documentation resources, such as “Learning Rust With Entirely Too Many Linked Lists” and “The Rustinomicon.” This is an extremely accomplished person writing about something she is possibly more qualified to than any other person on earth.
- saghm 6y agoI agree that the author is extremely qualified to write about this, but personally I also find the title a little jarring, mostly due to the word "couldn't". It seems more that there are number of tradeoffs where allowing slight runtime costs by default (like reference counting) unlocks a lot of stuff that's needed to be able to make dynamic linking easier, and Rust chose to go a different route than Swift. I understand that changing the title to read "where Rust chose to optimize for different priorities" won't get as many clicks, but the title to me paints the lack of dynamic linking support in Rust as some sort of failing rather than an explicit design choice. Realistically, I think the fact that Rust and Swift picked different routes here is a good thing! Having languages which optimize for different things gives people more options to pick one that suits their needs better; in a world with both nails and screws, it's better to have both hammers and screwdrivers available.
- steveklabnik 6y agoSure, that the author is an expert does not mean that she is infallible, or there's nothing to discuss. That's not what this sub-thread is about. This thread is about some sort of insinuation that this is born out of some sort of weird attempt at Swift to use Rust to gain prominence, or something.
- kortex 6y agoThose were some of the most memorable sections for me. I never really thought about the other side of monomorphizing. It made it really obvious where "zero-cost abstractions" start to fall short.
- kungito 6y agoI totally agree. I have a problem only with the wording of the title and it shouldn't matter who wrote it. It's a matter of integrity and we as software engineers should set higher standards to how we write professional articles
- pjmlp 6y agoIf by GCless you mean affine types, the only thing that Rust did was bringing affine types to the masses, which is already an achievement, agreed- However in the big context of application programming, and how Swift is used on Apple stack, any form of garbage collection alongside memory ownership is much more productive. Long term adoption for Rust will be places like where MISRA-C and SPARK are used nowadays. If I have to spray my code with Arc, Rc and RefCell, I rather let the compiler type them for me.
- JoshTriplett 6y agoI'm one of the leads of the Rust language team, and I love seeing writeups like this. When we're designing Rust, we look carefully at other languages, to see examples of precedent, different trade-offs, and experiences; we don't design in a vacuum. This is an extremely clear explanation of Swift making a different set of trade-offs to achieve a different goal, and it's worthy of consideration and evaluation. (I read it when it was first published.) There are no attacks against Rust here. This is a professional write-up discussing two languages, by someone experienced with both.
- DangitBobby 6y agoTo be fair, that sounds more like "didn't" than "couldn't," which is exactly what GP suggests.
- lehi 6y agoThe experience from a few years ago: "Then we hit a wall with the dynamic linker. At the time you could only link Swift libraries dynamically. Unfortunately the linker executed in polynomial time so Apple’s recommend maximum number of libraries in a single binary was 6. We had 92 and counting. As a result It took 8-12 seconds after tapping the app icon before main was even called. Our shinny new app was slower than the old clunky one. Then the binary size problem hit." - https://twitter.com/StanTwinB/status/1336890442768547845 https://twitter.com/StanTwinB/status/1336890442768547845
- judge2020 6y agohttps://threadreaderapp.com/thread/1336890442768547845.html https://threadreaderapp.com/thread/1336890442768547845.html
- dmit 6y agoWhy post this link? I only have first-hand experience reading Twitter threads on PC and phone (official app and web view), but in all cases it's just as easy to follow as the page you linked. (Except for the Georgia font - the typography is consistently better on the official sources.) The site you linked served me a huge ad. Will the author of the tweets ever see a single cent of the revenue my hypothetical ad click would have earned them? Of course not. Let's say I am an investigative reporter, looking for the original source of information that I saw in a tweet. Forget it, I'm not. Let's say instead that I sometimes like to check if I can poke holes in a news story I see during my morning scan of HN headlines. I'm sorry, I can't set the bar any lower. I think you need to be a corporation to be able to set bars lower than this. To my surprise, the site you linked did have a direct link to the first tweet of the thread it was displaying. It was embedded in the timestamp, styled to be shown in a small font, faint gray, no underline. The embodiment of "fine print". I was looking specifically for it, and it still took a spelunking session in the HTML code to figure out where the link was. The site you linked is bad. Unless you have good reasons to, please stop posting links to it. Thanks.
- zapzupnz 6y ago
- gok 6y agoPreviously https://news.ycombinator.com/item?id=21488415 https://news.ycombinator.com/item?id=21488415
- mark_l_watson 6y agoThere is a lot of great tech behind Swift and LLVM. I really enjoyed learning a bit (no pun intended) of Rust a few years ago, but it didn't stick for what I need. I really like Swift, but I have only done one SwiftUI app, all the Swift stuff I do is on the command line with swift build/test/run with a light weight editor. Apple's pre-trained deep learning models are very useful, and an edit/build/run iteration is just a second or two. I wish Rust had been around 30 years ago when I loved doing low level programming. Between my C++ books and gigs, C++ was good financially but Rust is just better (apologies to people who love C++).
- onei 6y agoWhile many of the ideas in Rust are not particularly radical in terms of compiler research, I think part of its success is because it had the past 30 years of lessons. We all stand on the shoulders of giants. Maybe in another 30 years we'll see a new language that learns from Rust's inevitable mistakes.
- RcouF1uZ4gsC 6y ago> I wish Rust had been around 30 years ago when I loved doing low level programming. I think Rust’s killer feature is lifetime analysis by the compiler. In 1991, you had 33 MHz 80486 with around 4 MB of RAM. I don’t think the average developer had the computing power to do Rust’s lifetime in any reasonable amount of time.
- pjmlp 6y agoIndeed, but they could have used a Pascal dialect, Basic compiler dialect, or Modula-2 to save many of the security headaches of C.
- fluffything 6y agoLifetime analysis doesn’t even show up in Rust compiler profiles. It takes 0 seconds. Rust lifetime analysis is a function-local pass. It never needs to access any state outside the function. It can be done on all functions in parallel, etc. You could do Rust alias analysis with 33 MHZ.
- RcouF1uZ4gsC 6y agoTo be honest, I am actually glad about this. Dynamic linking is the cause of a lot of complexity. First there is DLL hell on Windows and its equivalent on Linux. Second, having to maintain a stable ABI is a huge drain on language evolution. In C++, there have been many features that have been vetoed by the compiler/standard library implementers because it would cause an ABI break. Because of this ABI concerns, C++ is stuck with overheads for unique_ptr, map, unordered_map, and regex classes that are inefficient. Secondly, a lot of the benefits of dynamic linking don’t make as much sense now. First with regards to space savings. With modern disk capacity having extra copies of executable code is probably not that big of a deal. In addition with Link Time Optimization, you actually may not be saving space with dynamic linking since the linker can pretty aggressively throw away code that it knows is ever called. Second, with regards to security. One thing to consider is that dynamic linking is so complex that it is one of the reasons why people use docker, to make sure their binary and its dependencies are stable. Once you have something in a docker, it has a lot of the same update issues as a static linked binary, only less efficient. In addition, with security, Rust unlike C and C++ is a memory safe language. Given the vast majority of security issues are memory safety issues, I would expect Rust to have a lot fewer CVE’s. I think the experience with Go is likely enlightening in this regard. I am not aware of a huge issue of lack of security because it didn’t do dynamic linking and you couldn’t do security updates as easily. Finally, Rust as a new language is embracing the new way of programming and deploying. We are shifting away from using binary artifacts (often closed source) that once built were rarely changed, to build from source and continuously build/deploy. Cargo makes it relatively easy to do this. In that model, updating a binary for security is just a subset of the normal building and updating of a binary that is done daily. By avoiding dynamic linking, I think, Rust positions itself best as the no-compromise, high performance, modern, systems programming language.
- zozbot234 6y agoTo be extra clear about this, Rust does support dynamic linking; it's just limited to interfacing via the stable C ABI. The ABI-resilience conerns described in the article are ultimately punted to the users and/or the dev ecosystem, albeit some facilities (namely, bindgen/cbindgen) are also provided to ease the task.
- pjmlp 6y agoUsually when a language supports dynamic loading we refer to the language itself, as integration with OS ABI dynamic loader is a given for any language worth using.
- steveklabnik 6y agotechnically speaking, you can dynamic link regardless of the abi or not, using a stable ABI just provides you degrees of freedom with which compiler version compiles what. They're orthogonal concepts, even if they're most useful when combined.
- aseipp 6y agoYes, the same is true of e.g. GHC the Haskell compiler. There's nothing stopping you from using dynamic linking with GHC on Haskell code (i.e. your haskell app links to a haskell shared object), but if the language ABI isn't stable, you just rebuild all those shared objects with every new version of the compiler anyway. In fact you don't even need to use a different compiler version: aggressive cross-module inlining also reduces the amount of sharing you can do (even, potentially, down to zero sharing.) So you rarely save disk and every app has different shared objects loaded, and even assuming you use one compiler for everything, it may not matter at all. If anything it's worse because it can negatively impact startup time. In practice, ~everyone compiles Haskell code statically, and only dynamically links to libraries that have exposed a stable ABI. I'm not surprised people do the same for Rust.
- airhead969 6y agoIt seems like the fundamental issue is tradeoffs between compile-time and run-time efficiency. During development in almost every compiled language, it would make sense to have minimal cached, shared object (or shared intermediate) code in most use-cases while globally compiling and optimizing for releases. Invoking every run-time optimization and recompiling everything during development or first package installation makes zero sense because it's a waste of time. Also, shared common package library units for releases by default shrinks binaries rather than having zillions of exhaustive, slightly-different combinations of similar structures and machine code hoarding space in binaries. I've never understood how 300 MiB binaries would ever be acceptable to anyone, especially going against decades of shared libraries that reduced binary sizes and memory usage. Deliberate waste is never acceptable because there is no free lunch.
- mhh__ 6y agoIf I've read this correctly, D has been able to dynamic link properly for years, against it's own ABI, the C ABI, or the C++ ABI (e.g. templates, vtables up to single inheritance)
- millstone 6y agoDoes D have an ABI spec? I imagine the garbage collector complicates it considerably.
- mumblemumble 6y agohttps://dlang.org/spec/abi.html https://dlang.org/spec/abi.html
- mhh__ 6y agoIts both a blessing and a curse that the garbage collector doesn't complicate it all that much
- cakoose 6y agoThe neat thing about Swift's ABI compatibility is that it's almost as flexible as its API compatibility, e.g. - A library can add new fields to a struct. - A library's parameterized types can be instantiated with new arguments. Skimming over the D ABI docs, the compatibility seems much more rigid.
- mhh__ 6y agoWell you can't add do either of those things in D because, pure and simple, we'd never approve the change.
- cakoose 6y agoSome similar work: The GCC compiler for Java (GCJ) started out doing whole-program compilation, but that couldn't handle Java's flexible ABI and dynamic loading. In 2004 they introduced a compilation mode with an additional layer of indirection similar to Swift's: <ftp://gcc.gnu.org/pub/gcc/summit/2004/GCJ%20New%20ABI.pdf>. They didn't have to deal with monomorphizing, though. Go's does something similar to Swift to avoid code bloat in their built-in polymorphic map: <https://dave.cheney.net/2018/05/29/how-the-go-runtime-implements-maps-efficiently-without-generics https://dave.cheney.net/2018/05/29/how-the-go-runtime-implem...>. It's just a one-off manual thing, though.
- PoignardAzur 6y agoClickbait title aside, that post was really popular when it came out. I remember there was a period where it got reposted or mentioned on Rust Internals at least once every two weeks.