19 ms·

Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

207 points by alin23 4y ago

deleted 4y ago
[deleted]
apatheticonion 4y ago
I have heard great things about Swift but haven't spent a second looking into it because I assumed it's an Apple exclusive language - making it unsuitable for my use case of services deployed on Linux, developed from any OS. To what extent, if any, is Swift exclusive to Apple devices?
- zyang 4y ago
  Swift is backed by LLVM and Linux solid is pretty solid. The cross platform problem has always been the core libraries. Fortunately a pure Swift core foundation rewrite is underway. https://www.swift.org/blog/future-of-foundation/ https://www.swift.org/blog/future-of-foundation/
  - jeroenhd 4y ago
    Looks like I may eventually give Swift a try once the rewrite is done! I've been watching Swift for a while now but I don't have a Mac so I haven't had a reason to try it. Do you happen to have any idea about UI library support? Will UI still be Apple only?
    - charrondev 4y ago
      As far as I know all of the UI stuff is dependant on iOS and macOS specific libraries. Apples UIKit and AppKit libraries are still proprietary here (and even have completely different capabilities depending on your platform and OS version).
    - MBCook 4y ago
      I see no reason for Apple to ever make the UI stuff available on other platforms, I can’t see how that would benefit them. But there’s nothing stopping other UI libraries from making Swift bindings.
      - photonbucket 4y ago
        Apple surely must run a lot of linux machines internally, I imagine it would be nice for them to have internal services written in swift
        MBCook 4y ago
        That’s my understanding of why it’s on Linux (besides the community). Apple uses tons of Linux servers, cloud and DC. Swift means they can write high performance networking stuff with memory safety and modern features in a language/library they already use internally instead of using yet another new language (or losing safety with C/C++/Obj-C). Makes a lot of sense to me.
        giantrobot 4y ago
        Most of the Linux machines inside Apple are tucked away in data centers, they're not sitting on engineers' desks (there's always exceptions). The people writing stuff that will run on Linux are usually writing in languages that are already cross platform. The stuff running on Linux inside Apple has a pretty good chance of being Java or Python (or Ruby for some teams). These languages are usually chosen for their ecosystem. There's not much of a Swift ecosystem except the proprietary one on macOS and iOS. Without all the frameworks available on macOS there's not a huge Swift ecosystem to leverage.
      - crispyalmond 4y ago
        Tokamak[0] is really promising on this front [0] https://github.com/TokamakUI/Tokamak https://github.com/TokamakUI/Tokamak
    - Klonoar 4y ago
      It's not out of the question for someone to write a (e.g) GTK-backed SwiftUI API library.
- ChrisMarshallNY 4y ago
  I'd say that it isn't particularly useful, outside of the Apple ecosystem, but I do know that there has been Linux, and even Windows, support. In my case, I don't really care, as I program Apple native. I think it's an excellent language. I enjoy writing in it. I write Swift, every day.
  - thayne 4y ago
    Conversely, I don't really care about swift because it isn't particularly useful outside of the apple ecosystem.
    - ChrisMarshallNY 4y ago
      Exactly. I wasn't trying to be dismissive. It's just the way things are. Some people loathe everything Apple. They Just. Hate. Apple. I've been dealing with that for over thirty years. Some people hate me, for no reason, other than I write code for Apple devices. I don't hate anyone else, but I've tried programming a number of systems, and haven't enjoyed it as much, as I do, working with Apple's stuff.
- stalfosknight 4y ago
  It’s open source under the Apache license. It has support for Windows, Linux, and Android too.
- zokier 4y ago
  At one point IBM attempted to make swift viable on the sever-side: https://news.ycombinator.com/item?id=21808619 https://news.ycombinator.com/item?id=21808619
  - myko 4y ago
    Kitura is no longer maintained but one of its primary competitors is still going strong: https://vapor.codes/ https://vapor.codes/
- klodolph 4y ago
  It's Apple-exclusive in the same way that C# is Windows-exclusive or Go is Linux-exclusive. You'll have a much better time programming C# on Windows or Swift on macOS, because that's where all the best tooling is. The tools and libraries are open-source, cross-platform, and mostly maintained by one company.
  - chabad360 4y ago
    That's really not the case with C# or Go. C# has first class Linux support nowadays (minus GUI support) and using Go on windows is quite easy (I daily drive go on windows at work).
    - maldev 4y ago
      With Maui, you can now have first class Linux gui support, same with Blazor. You can also even do some of the more cutting edge stuff on linux like compiling straight to ASM.
      - KRAKRISMOTT 4y ago
        Are you sure Maui provides native Linux GUI support? (Not through a WebView)
      - lockhouse 4y ago
        .NET MAUI doesn't have first class Linux GUI support. Unless you count Android as Linux. [1] There were some unofficial efforts for Linux GUI support for MAUI's predecessor (Xamarin Forms), but never first party support. [1] https://learn.microsoft.com/en-us/dotnet/maui/supported-platforms?view=net-maui-7.0 https://learn.microsoft.com/en-us/dotnet/maui/supported-plat...
      - mpawelski 4y ago
        MAUI is not working on Linux, still Microsoft market it as "cross-platform" which is super misleading at best (and pure lie at worst).
        4y ago
- andsoitis 4y ago
  Not exclusive to Apple devices at all. More details about platform support: https://www.swift.org/platform-support/ https://www.swift.org/platform-support/ (tl;dr Apple platforms, Linux, Windows)
- alfor 4y ago
  I think that Apple control where the language goes. That mean that things will serve other use cases only accidentally. fast.ai was very interested in Swift but dropped that project for that reason.
  - likeabbas 4y ago
    I wish fast.ai would have succeeded with Swift. The world desperately needs the ML community to move on from Python and towards a fast, statically typed language with truly concurrent multi threading.
    - alfor 4y ago
      Having a common stack that is fast and coherent from the highest to the lowest level would be wonderful. But it seems it’s what make Python successful: it force a interface between the low level serious stuff and the configuration/use at higher level.
- deleted 4y ago
  [deleted]
  - deleted 4y ago
    [deleted]
WiSaGaN 4y ago
(2019)
ChrisMarshallNY 4y ago
This is an excellent write-up!
alin23 4y ago
This past week I’ve been reverse engineering the macOS IOMobileFramebuffer to find a way to disconnect displays in software for Lunar (https://lunar.fyi/ https://lunar.fyi/). And I kept stumbling upon these witness tables while debugging. This led me to find this write-up and the much more interesting story behind it. The end note was priceless ^_^ collapses
- sweetjuly 4y ago
  I feel your pain, reverse engineering Swift is such a nightmare. I don't know if Ida handles it better but both Ghidra and Binja produce entirely inscrutable disassembly.
  - saagarjha 4y ago
    Nobody really cares about it, unfortunately. There’s not much reason things have to be this bad, other than lack of investment.
  - riscy 4y ago
    Hopper has support for Swift: https://www.hopperapp.com/ https://www.hopperapp.com/
grantjpowell 4y ago
Love the article. In my mind I see the problem of dynamic linking in rust to have a bunch of overlap with the "I want this rust library to be exposed in my higher level GC'd language with minimal safety issues/tedious handmaintained bindings" problem. My hunch is that the lack of expressiveness of the C ABI is holding back both. the thing I'd love to see some sort of "higher level than the C ABI" come out. And something like `wasm-bindgen`[0] to exist for more languages. Here's a link to the rust "interopable_api" proposal! I don't understand all the implications, but it seems to be in the right direction https://github.com/rust-lang/rust/pull/105586 https://github.com/rust-lang/rust/pull/105586 [0]https://rustwasm.github.io/docs/wasm-bindgen/ https://rustwasm.github.io/docs/wasm-bindgen/
- tubs 4y ago
  So, COM?
  - Gankra 4y ago
    The greatest trick Swift's developers ever performed was convincing the world that it wasn't just a better UX layer on top of COM
    - pjmlp 4y ago
      IBM did it first with SOM, but unfortunely it went the way it did.
  - vrnvu 4y ago
    Sorry, does anyone mind to explain/reference what COM is?
    - imron 4y ago
      Component object model. It’s a windows thing from the 90’s that still permeates the lower levels of windows programming.
      - 4y ago
im_jerry87 4y ago
I didn't read the URL properly and for a second, thought it was Andy Hertzfeld's blog.
forrestthewoods 4y ago
My very vague understanding of Swift’s dynamic linking is that it works but comes with a huge performance penalty. To the point of it becoming unusable for high-performance apps. Is that wrong?
- dnadoba 4y ago
  Yes, it is wrong. The article explains this quite well how you can trade performance and ABI stability.
  - forrestthewoods 4y ago
    The article doesn’t have any performance metrics. It mentions surprising performance cliffs but I couldn’t find details. It says there is a perf cost you can opt out of but doesn’t detail the cost or what you lose by opting out. It sure seems from the article that there are a lot of negative performance implications. Which totally be a price worth paying! But I would not agree that the article explains what that price actually is.
    - Gankra 4y ago
      The biggest problem is that if you have a big """zero-cost-abstraction""" blob like iterator adaptors -- `Map<Filter<Fold<ArrayIter<MyType>>>>` -- and a single drop of Resilient Type is in there (i.e. MyType is resilient) then the whole thing gets polymorphically compiled and the compiler won't boil away any of the things that are supposed to be "zero cost". How much you get burned by this kind of thing really depends on how you design APIs and where the hotspots are. Like if the big iterator blob is only ever for like 10 items, whatever. If the iterator blob is iterated inside the dylib where it's not resilient and can be inlined away, whatever.
puffoflogic 4y ago
[flagged]
- saagarjha 4y ago
  I am honestly curious what kind of content you must be consuming to come to that sort of conclusion. Like, is there a secret “anti-dynamic linking league” somewhere who pushes people to think it’s the literal spawn of Satan?
  - david2ndaccount 4y ago
    At this point it seems like sour grapes by the rust people.
  - pjmlp 4y ago
    I never got the anti-dynamic linking crowd, being old enough that static linking was the only option (minus hacks like overlays), I don't miss those days.
greyw 4y ago
Dynamic linking is in my opinion not that useful anymore in this day and age where the few MB of RAM and disk space you save is not worth the hassle. The amount of dynamic linking issues I encountered on GNU/Linux was insane (fuck libstdc++). Not even glibc manages to keep forward compatibility working (breaking memcpy, breaking DT_HASH, ...)! It's much better to just statically link your binaries (unfortunately many programs on GNU/Linux do not support this). Before the security guys come out the woods: No you don't need to rebuilt all programs linked against your dynamic library of choice. Just keep the object files around and relink them against the updated version of the static libraries. If now package manager just allowed differential binary updates (actually why is this trivial optimization not a common thing?) the overhead in terms of download size is not that of a big deal too. The value you gain is immense: binaries just work (modulo linux ABI issues but they are doing quite a good job at keeping this interface stable)
- taeric 4y ago
  Security updates, though? I mean, you aren't wrong. But there were other advantages to dynamic linking.
  - Gigachad 4y ago
    You are going to see more security improvement by using Rust than the unlikely but possible chance of having your C library incidentally updated.
    - palata 4y ago
      Not convinced at all. Say a big exploit gets published for a very popular library: you probably want to patch it as fast as possible. Whereas Rust will help you have fewer security issues in your own code, which is likely not as popular as some of your critical deps.
  - jessermeyer 4y ago
    There is obviously a trade off here, but categorically speaking, new releases introduce new bugs and security exploits too.
jchw 4y ago
A lot of people who are invested in the Apple ecosystem have been trying to present Swift as a general competitor to Rust, but no matter how many bullet points you list, as far as I can tell, it hasn't made any progress at all on that front, and I haven't been convinced that it will change to become closer to that any time soon, either. As far as I can tell, there's really nothing wrong with Swift, and it probably has a lot to offer, but at some point someone's going to need to point out the obvious: basically nobody outside of the Apple ecosystem is considering using Swift for anything. That doesn't mean it's bad, but it's not a competitor to Rust today, and it's not trending in that direction.
- mr_00ff00 4y ago
  I know this might be opening a can of worms, but technically doesn’t Swift do reference counting? Which is faster than “mark and sweep” GC but will never compare to manual management (like C/C++ and rust ownership) I assume that’s the big reason.
  - convolvatron 4y ago
    indeed. RC isn't universally faster than GC - it depends on workload
  - rat9988 4y ago
    reference counting is not necessarily faster.
    - lll-o-lll 4y ago
      It’s not faster in the normal case, it’s generally quite a bit slower. It is more predictable however (you can actually use it with real-time constraints!), and performs better under memory pressure. Whether this is a net benefit depends on what hardware you’re running on and what the app is supposed to be doing.
  - Jweb_Guru 4y ago
    Reference counting is not faster than tracing GC for general purpose workloads, at least not without a lot of extensive tuning that includes stuff like using tracing and moving GC for the young generation (prior to the first collection), deferring reference count increment/decrements using write barriers, limiting reference count bits (and reclaiming stuck objects with some variant of full heap tracing), and so on. Many of these benefits cannot be effectively realized if you insist on prompt execution of finalizers, either. Languages like Rust and C++ gain their performance benefits not from using reference counting or immediate reclamation, but from avoiding creating garbage in the first place by stack allocating most stuff, creating large objects by-value and within vectors to reduce pointer chasing, using direct interior references into data structures in ways that it would be difficult to trace efficiently, and using data structures like arenas to emulate the benefits of generational GC in controlled contexts where a maximum lifetime can be statically determined. My personal feeling is that it's very hard to directly compare manual memory management and GC from a performance perspective because so many of the techniques that make tracing fast involve moving pointers, which is very costly to combine with many of the techniques that make manual memory management fast which rely on values not moving in memory. I think any such solution that is efficient for both kinds of memory will probably require the programmer to have very tight control over when the garbage collector can execute, whether pointers can move at particular times, etc., and the complexity of maintaining that kind of control (and getting it right!) seems so high that I'm not sure it would ever be usable for general purpose programming.
brink 4y ago
In 2020 I wanted to leverage the new apple silicon in my Rust app, so I needed to do some work in Swift, and maybe i was looking in the wrong spots - but the Swift community felt incredibly dead. Swift documentation is horrible, I asked for help in a few places online and got none. I eventually solved the problem by guessing to fill in the gaps of documentation. A very different experience than the rust community, which was extremely eager to help with every problem I had.
- deleted 4y ago
  [deleted]
- pjmlp 4y ago
  Did you bother to look into the Apple developer forums?
brundolf 4y ago
Aria is so good at putting things into accessible (and entertaining) words. One of the best at it I've ever seen (they're the same person who wrote the (in)famous "Learn Rust by writing Entirely Too Many Linked Lists", which is one of my favorite pieces of technical writing of all time)
galaxyLogic 4y ago
How do they handle "DLL Hell"?
- ridiculous_fish 4y ago
  DLL Hell comes when you have multiple versions of dynamically linked libraries from third parties. In practice, Swift's dynamic linking exists to support Apple's own first-party libraries which are shipped with the system, so there is no DLL Hell. There is only one UIKit at a time.
photonbucket 4y ago
It'd be interesting to see if Swift's stable abi could become a #[repr] in rust, so that you could write "swift" libs in rust
low_tech_punk 4y ago
Could someone help explain the difference between Rust ARC (Atomically Reference Counted) vs. Switch ARC (Automatic Reference Counting)? Do the language designers intentionally chosen the same acronym to confuse us?
- oconnor663 4y ago
  Rust's Arc is very similar to C++'s shared_ptr. It's a generic container type, which does reference counting on the inside. I haven't written Swift, but I believe reference counting is mostly automatic and transparent there, more like a GC strategy than like a container. Not sure whether anyone's to blame for the name collision :)
  - zaphar 4y ago
    With one very important difference. Unless you use a thread_safe interior mutability implementation like Mutex Arc won't allow you modify or change the wrapped value. C++'s shared_ptr does not enforce the same. All of which means that while the shared_ptr control block is thread safe what the shared_ptr contains may or may not be. This has caused, and will continue to cause, no end of confusion for folks doing C++ especially for less experienced devs.
    - oconnor663 4y ago
      There are many differences when it comes to safety: - Like you said, Rust doesn't let you data race on the contents of an Arc. - Rust also doesn't let you data race on the Arc itself (i.e. repointing a global Arc without synchronization). This is Bug #5 in this great talk: https://youtu.be/lkgszkPnV8g https://youtu.be/lkgszkPnV8g - Rust also doesn't let you retain a reference to the contents of an Arc longer than you retain the Arc. This is the bug that C++ Core Guideline R.37 is about: http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rr-smartptrget http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#... But I'm not sure I'd call these Arc vs shared_ptr differences per se, because almost any comparison between a Rust container and its C++ equivalent looks similar.
- 4y ago
cormacrelf 4y ago
If I’m reading it right, reabstraction is about allowing you to make a protocol implementation more generic in future. If it’s implemented for String today, then if you change it to be implemented for all T in future then the dylib is still usable. The reabstraction thunks are checking that the type you’ve called it with is actually covered. So they’re not free. The idea of calling polymorphically compiled functions with pointers to witness tables in extra arguments seems like something we could throw a simple JIT compiler at. A poly-function might have a list of available concrete implementations it could forward you to. You could, as a consumer of a dylib, for each call site on a poly function where you’re passing in a type known at compile time, initially write out a stub function, that the first time it runs, caches the result of this lookup and replaces the stub with something better. The stub would have a signature shaped optimally, the initial body of the stub would pad that out with the witness tables and code to initiate JIT using the poly function, and the JITed replacement would in the ideal case just be a jump instruction to a concrete implementation with a signature identical to the stub’s. Would that help?
gumby 4y ago
The TL;DR is: Swift strives for ABI stability, Rust does not. There's also a driveby comment that C++ doesn't really support dynamic linking because of all the templates in its standard library, which is a weird statement to make.
- ahartmetz 4y ago
  Yeah - the big, complicated parts of platform libraries for which post-release bugfixes make sense are usually not templated. The vector container class is a template, but Korean input method support is not.
  - ahartmetz 4y ago
    (Actually, Korean was not a great choice of example. The Korean writing system isn't very complicated. Japanese would have been better - they even have three writing systems!)
  - int_19h 4y ago
    Templates are not a problem if you're defining the complete set of valid specializations in advance, as would be the case here - that's what "extern template" is for.
- pjmlp 4y ago
  Indeed, on Windows C++ dynamic linking has always been available, sure there is the ABI issue, but that doesn't mean there aren't C++ frameworks being shipped as dynamic libraries, e.g. MFC, VCL, Qt,...
Shorel 4y ago
Couldn't or wouldn't? In some part, dynamic linking is one of the reasons for the travesty of dunces that's Docker and Kubernetes. With static linking, most of that redundancy is not needed.
- imron 4y ago
  Docker and kubernetes excel in a world of static binaries. Gotta love those 5mb bare images.
  - heinrich5991 4y ago
    Why use docker in that case?
travisgriggs 4y ago
What a great write up. > Swift reserves a callee-preserved register for a method’s self argument (pointer) to make repeated calls faster. Cool? Years ago (2000 I think), I remember Eliot Miranda explaining that the biggest optimization—-over and above all the jit stuff—in the VisualWorks Smalltalk VM basically came down to something very similar: very specific management of registers for the commonest of calling patterns.
singularity2001 4y ago
Also swift hides almost all the complexity of memory safety in an ergonomic way. My preferred language by far.
bsaul 4y ago
this post makes me realize why swift is starting to look more and more bloated from an end-user pov. It was designed to be right from the beginning. instead of carefully backing off complex problems and slashing features to keep the design elegant, they just "went for it" at full speed. History will tell if that was a good thing.

207 points by alin23 4y ago

deleted 4y ago

[deleted]

I have heard great things about Swift but haven't spent a second looking into it because I assumed it's an Apple exclusive language - making it unsuitable for my use case of services deployed on Linux, developed from any OS. To what extent, if any, is Swift exclusive to Apple devices?

zyang 4y ago
Swift is backed by LLVM and Linux solid is pretty solid. The cross platform problem has always been the core libraries. Fortunately a pure Swift core foundation rewrite is underway. https://www.swift.org/blog/future-of-foundation/ https://www.swift.org/blog/future-of-foundation/
- jeroenhd 4y ago
  Looks like I may eventually give Swift a try once the rewrite is done! I've been watching Swift for a while now but I don't have a Mac so I haven't had a reason to try it. Do you happen to have any idea about UI library support? Will UI still be Apple only?
  - charrondev 4y ago
    As far as I know all of the UI stuff is dependant on iOS and macOS specific libraries. Apples UIKit and AppKit libraries are still proprietary here (and even have completely different capabilities depending on your platform and OS version).
  - MBCook 4y ago
    I see no reason for Apple to ever make the UI stuff available on other platforms, I can’t see how that would benefit them. But there’s nothing stopping other UI libraries from making Swift bindings.
    - photonbucket 4y ago
      Apple surely must run a lot of linux machines internally, I imagine it would be nice for them to have internal services written in swift
      - MBCook 4y ago
        That’s my understanding of why it’s on Linux (besides the community). Apple uses tons of Linux servers, cloud and DC. Swift means they can write high performance networking stuff with memory safety and modern features in a language/library they already use internally instead of using yet another new language (or losing safety with C/C++/Obj-C). Makes a lot of sense to me.
      - giantrobot 4y ago
        Most of the Linux machines inside Apple are tucked away in data centers, they're not sitting on engineers' desks (there's always exceptions). The people writing stuff that will run on Linux are usually writing in languages that are already cross platform. The stuff running on Linux inside Apple has a pretty good chance of being Java or Python (or Ruby for some teams). These languages are usually chosen for their ecosystem. There's not much of a Swift ecosystem except the proprietary one on macOS and iOS. Without all the frameworks available on macOS there's not a huge Swift ecosystem to leverage.
    - crispyalmond 4y ago
      Tokamak[0] is really promising on this front [0] https://github.com/TokamakUI/Tokamak https://github.com/TokamakUI/Tokamak
  - Klonoar 4y ago
    It's not out of the question for someone to write a (e.g) GTK-backed SwiftUI API library.
ChrisMarshallNY 4y ago
I'd say that it isn't particularly useful, outside of the Apple ecosystem, but I do know that there has been Linux, and even Windows, support. In my case, I don't really care, as I program Apple native. I think it's an excellent language. I enjoy writing in it. I write Swift, every day.
- thayne 4y ago
  Conversely, I don't really care about swift because it isn't particularly useful outside of the apple ecosystem.
  - ChrisMarshallNY 4y ago
    Exactly. I wasn't trying to be dismissive. It's just the way things are. Some people loathe everything Apple. They Just. Hate. Apple. I've been dealing with that for over thirty years. Some people hate me, for no reason, other than I write code for Apple devices. I don't hate anyone else, but I've tried programming a number of systems, and haven't enjoyed it as much, as I do, working with Apple's stuff.
stalfosknight 4y ago
It’s open source under the Apache license. It has support for Windows, Linux, and Android too.
zokier 4y ago
At one point IBM attempted to make swift viable on the sever-side: https://news.ycombinator.com/item?id=21808619 https://news.ycombinator.com/item?id=21808619
- myko 4y ago
  Kitura is no longer maintained but one of its primary competitors is still going strong: https://vapor.codes/ https://vapor.codes/
klodolph 4y ago
It's Apple-exclusive in the same way that C# is Windows-exclusive or Go is Linux-exclusive. You'll have a much better time programming C# on Windows or Swift on macOS, because that's where all the best tooling is. The tools and libraries are open-source, cross-platform, and mostly maintained by one company.
- chabad360 4y ago
  That's really not the case with C# or Go. C# has first class Linux support nowadays (minus GUI support) and using Go on windows is quite easy (I daily drive go on windows at work).
  - maldev 4y ago
    With Maui, you can now have first class Linux gui support, same with Blazor. You can also even do some of the more cutting edge stuff on linux like compiling straight to ASM.
    - KRAKRISMOTT 4y ago
      Are you sure Maui provides native Linux GUI support? (Not through a WebView)
    - lockhouse 4y ago
      .NET MAUI doesn't have first class Linux GUI support. Unless you count Android as Linux. [1] There were some unofficial efforts for Linux GUI support for MAUI's predecessor (Xamarin Forms), but never first party support. [1] https://learn.microsoft.com/en-us/dotnet/maui/supported-platforms?view=net-maui-7.0 https://learn.microsoft.com/en-us/dotnet/maui/supported-plat...
    - mpawelski 4y ago
      MAUI is not working on Linux, still Microsoft market it as "cross-platform" which is super misleading at best (and pure lie at worst).
      - 4y ago
andsoitis 4y ago
Not exclusive to Apple devices at all. More details about platform support: https://www.swift.org/platform-support/ https://www.swift.org/platform-support/ (tl;dr Apple platforms, Linux, Windows)
alfor 4y ago
I think that Apple control where the language goes. That mean that things will serve other use cases only accidentally. fast.ai was very interested in Swift but dropped that project for that reason.
- likeabbas 4y ago
  I wish fast.ai would have succeeded with Swift. The world desperately needs the ML community to move on from Python and towards a fast, statically typed language with truly concurrent multi threading.
  - alfor 4y ago
    Having a common stack that is fast and coherent from the highest to the lowest level would be wonderful. But it seems it’s what make Python successful: it force a interface between the low level serious stuff and the configuration/use at higher level.
deleted 4y ago
[deleted]
- deleted 4y ago
  [deleted]

WiSaGaN 4y ago

(2019)

ChrisMarshallNY 4y ago

This is an excellent write-up!

alin23 4y ago

This past week I’ve been reverse engineering the macOS IOMobileFramebuffer to find a way to disconnect displays in software for Lunar (https://lunar.fyi/ https://lunar.fyi/). And I kept stumbling upon these witness tables while debugging. This led me to find this write-up and the much more interesting story behind it. The end note was priceless ^_^ collapses

sweetjuly 4y ago
I feel your pain, reverse engineering Swift is such a nightmare. I don't know if Ida handles it better but both Ghidra and Binja produce entirely inscrutable disassembly.
- saagarjha 4y ago
  Nobody really cares about it, unfortunately. There’s not much reason things have to be this bad, other than lack of investment.
- riscy 4y ago
  Hopper has support for Swift: https://www.hopperapp.com/ https://www.hopperapp.com/

grantjpowell 4y ago

Love the article. In my mind I see the problem of dynamic linking in rust to have a bunch of overlap with the "I want this rust library to be exposed in my higher level GC'd language with minimal safety issues/tedious handmaintained bindings" problem. My hunch is that the lack of expressiveness of the C ABI is holding back both. the thing I'd love to see some sort of "higher level than the C ABI" come out. And something like `wasm-bindgen`[0] to exist for more languages. Here's a link to the rust "interopable_api" proposal! I don't understand all the implications, but it seems to be in the right direction https://github.com/rust-lang/rust/pull/105586 https://github.com/rust-lang/rust/pull/105586 [0]https://rustwasm.github.io/docs/wasm-bindgen/ https://rustwasm.github.io/docs/wasm-bindgen/

tubs 4y ago
So, COM?
- Gankra 4y ago
  The greatest trick Swift's developers ever performed was convincing the world that it wasn't just a better UX layer on top of COM
  - pjmlp 4y ago
    IBM did it first with SOM, but unfortunely it went the way it did.
- vrnvu 4y ago
  Sorry, does anyone mind to explain/reference what COM is?
  - imron 4y ago
    Component object model. It’s a windows thing from the 90’s that still permeates the lower levels of windows programming.
    - 4y ago

im_jerry87 4y ago

I didn't read the URL properly and for a second, thought it was Andy Hertzfeld's blog.

forrestthewoods 4y ago

My very vague understanding of Swift’s dynamic linking is that it works but comes with a huge performance penalty. To the point of it becoming unusable for high-performance apps. Is that wrong?

dnadoba 4y ago
Yes, it is wrong. The article explains this quite well how you can trade performance and ABI stability.
- forrestthewoods 4y ago
  The article doesn’t have any performance metrics. It mentions surprising performance cliffs but I couldn’t find details. It says there is a perf cost you can opt out of but doesn’t detail the cost or what you lose by opting out. It sure seems from the article that there are a lot of negative performance implications. Which totally be a price worth paying! But I would not agree that the article explains what that price actually is.
  - Gankra 4y ago
    The biggest problem is that if you have a big """zero-cost-abstraction""" blob like iterator adaptors -- `Map<Filter<Fold<ArrayIter<MyType>>>>` -- and a single drop of Resilient Type is in there (i.e. MyType is resilient) then the whole thing gets polymorphically compiled and the compiler won't boil away any of the things that are supposed to be "zero cost". How much you get burned by this kind of thing really depends on how you design APIs and where the hotspots are. Like if the big iterator blob is only ever for like 10 items, whatever. If the iterator blob is iterated inside the dylib where it's not resilient and can be inlined away, whatever.

puffoflogic 4y ago

[flagged]

saagarjha 4y ago
I am honestly curious what kind of content you must be consuming to come to that sort of conclusion. Like, is there a secret “anti-dynamic linking league” somewhere who pushes people to think it’s the literal spawn of Satan?
- david2ndaccount 4y ago
  At this point it seems like sour grapes by the rust people.
- pjmlp 4y ago
  I never got the anti-dynamic linking crowd, being old enough that static linking was the only option (minus hacks like overlays), I don't miss those days.

greyw 4y ago

Dynamic linking is in my opinion not that useful anymore in this day and age where the few MB of RAM and disk space you save is not worth the hassle. The amount of dynamic linking issues I encountered on GNU/Linux was insane (fuck libstdc++). Not even glibc manages to keep forward compatibility working (breaking memcpy, breaking DT_HASH, ...)! It's much better to just statically link your binaries (unfortunately many programs on GNU/Linux do not support this). Before the security guys come out the woods: No you don't need to rebuilt all programs linked against your dynamic library of choice. Just keep the object files around and relink them against the updated version of the static libraries. If now package manager just allowed differential binary updates (actually why is this trivial optimization not a common thing?) the overhead in terms of download size is not that of a big deal too. The value you gain is immense: binaries just work (modulo linux ABI issues but they are doing quite a good job at keeping this interface stable)

taeric 4y ago
Security updates, though? I mean, you aren't wrong. But there were other advantages to dynamic linking.
- Gigachad 4y ago
  You are going to see more security improvement by using Rust than the unlikely but possible chance of having your C library incidentally updated.
  - palata 4y ago
    Not convinced at all. Say a big exploit gets published for a very popular library: you probably want to patch it as fast as possible. Whereas Rust will help you have fewer security issues in your own code, which is likely not as popular as some of your critical deps.
- jessermeyer 4y ago
  There is obviously a trade off here, but categorically speaking, new releases introduce new bugs and security exploits too.

jchw 4y ago

A lot of people who are invested in the Apple ecosystem have been trying to present Swift as a general competitor to Rust, but no matter how many bullet points you list, as far as I can tell, it hasn't made any progress at all on that front, and I haven't been convinced that it will change to become closer to that any time soon, either. As far as I can tell, there's really nothing wrong with Swift, and it probably has a lot to offer, but at some point someone's going to need to point out the obvious: basically nobody outside of the Apple ecosystem is considering using Swift for anything. That doesn't mean it's bad, but it's not a competitor to Rust today, and it's not trending in that direction.

mr_00ff00 4y ago
I know this might be opening a can of worms, but technically doesn’t Swift do reference counting? Which is faster than “mark and sweep” GC but will never compare to manual management (like C/C++ and rust ownership) I assume that’s the big reason.
- convolvatron 4y ago
  indeed. RC isn't universally faster than GC - it depends on workload
- rat9988 4y ago
  reference counting is not necessarily faster.
  - lll-o-lll 4y ago
    It’s not faster in the normal case, it’s generally quite a bit slower. It is more predictable however (you can actually use it with real-time constraints!), and performs better under memory pressure. Whether this is a net benefit depends on what hardware you’re running on and what the app is supposed to be doing.
- Jweb_Guru 4y ago
  Reference counting is not faster than tracing GC for general purpose workloads, at least not without a lot of extensive tuning that includes stuff like using tracing and moving GC for the young generation (prior to the first collection), deferring reference count increment/decrements using write barriers, limiting reference count bits (and reclaiming stuck objects with some variant of full heap tracing), and so on. Many of these benefits cannot be effectively realized if you insist on prompt execution of finalizers, either. Languages like Rust and C++ gain their performance benefits not from using reference counting or immediate reclamation, but from avoiding creating garbage in the first place by stack allocating most stuff, creating large objects by-value and within vectors to reduce pointer chasing, using direct interior references into data structures in ways that it would be difficult to trace efficiently, and using data structures like arenas to emulate the benefits of generational GC in controlled contexts where a maximum lifetime can be statically determined. My personal feeling is that it's very hard to directly compare manual memory management and GC from a performance perspective because so many of the techniques that make tracing fast involve moving pointers, which is very costly to combine with many of the techniques that make manual memory management fast which rely on values not moving in memory. I think any such solution that is efficient for both kinds of memory will probably require the programmer to have very tight control over when the garbage collector can execute, whether pointers can move at particular times, etc., and the complexity of maintaining that kind of control (and getting it right!) seems so high that I'm not sure it would ever be usable for general purpose programming.

brink 4y ago

In 2020 I wanted to leverage the new apple silicon in my Rust app, so I needed to do some work in Swift, and maybe i was looking in the wrong spots - but the Swift community felt incredibly dead. Swift documentation is horrible, I asked for help in a few places online and got none. I eventually solved the problem by guessing to fill in the gaps of documentation. A very different experience than the rust community, which was extremely eager to help with every problem I had.

deleted 4y ago
[deleted]
pjmlp 4y ago
Did you bother to look into the Apple developer forums?

brundolf 4y ago

Aria is so good at putting things into accessible (and entertaining) words. One of the best at it I've ever seen (they're the same person who wrote the (in)famous "Learn Rust by writing Entirely Too Many Linked Lists", which is one of my favorite pieces of technical writing of all time)

galaxyLogic 4y ago

How do they handle "DLL Hell"?

ridiculous_fish 4y ago
DLL Hell comes when you have multiple versions of dynamically linked libraries from third parties. In practice, Swift's dynamic linking exists to support Apple's own first-party libraries which are shipped with the system, so there is no DLL Hell. There is only one UIKit at a time.

photonbucket 4y ago

It'd be interesting to see if Swift's stable abi could become a #[repr] in rust, so that you could write "swift" libs in rust

low_tech_punk 4y ago

Could someone help explain the difference between Rust ARC (Atomically Reference Counted) vs. Switch ARC (Automatic Reference Counting)? Do the language designers intentionally chosen the same acronym to confuse us?

oconnor663 4y ago
Rust's Arc is very similar to C++'s shared_ptr. It's a generic container type, which does reference counting on the inside. I haven't written Swift, but I believe reference counting is mostly automatic and transparent there, more like a GC strategy than like a container. Not sure whether anyone's to blame for the name collision :)
- zaphar 4y ago
  With one very important difference. Unless you use a thread_safe interior mutability implementation like Mutex Arc won't allow you modify or change the wrapped value. C++'s shared_ptr does not enforce the same. All of which means that while the shared_ptr control block is thread safe what the shared_ptr contains may or may not be. This has caused, and will continue to cause, no end of confusion for folks doing C++ especially for less experienced devs.
  - oconnor663 4y ago
    There are many differences when it comes to safety: - Like you said, Rust doesn't let you data race on the contents of an Arc. - Rust also doesn't let you data race on the Arc itself (i.e. repointing a global Arc without synchronization). This is Bug #5 in this great talk: https://youtu.be/lkgszkPnV8g https://youtu.be/lkgszkPnV8g - Rust also doesn't let you retain a reference to the contents of an Arc longer than you retain the Arc. This is the bug that C++ Core Guideline R.37 is about: http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rr-smartptrget http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#... But I'm not sure I'd call these Arc vs shared_ptr differences per se, because almost any comparison between a Rust container and its C++ equivalent looks similar.
4y ago

cormacrelf 4y ago

If I’m reading it right, reabstraction is about allowing you to make a protocol implementation more generic in future. If it’s implemented for String today, then if you change it to be implemented for all T in future then the dylib is still usable. The reabstraction thunks are checking that the type you’ve called it with is actually covered. So they’re not free. The idea of calling polymorphically compiled functions with pointers to witness tables in extra arguments seems like something we could throw a simple JIT compiler at. A poly-function might have a list of available concrete implementations it could forward you to. You could, as a consumer of a dylib, for each call site on a poly function where you’re passing in a type known at compile time, initially write out a stub function, that the first time it runs, caches the result of this lookup and replaces the stub with something better. The stub would have a signature shaped optimally, the initial body of the stub would pad that out with the witness tables and code to initiate JIT using the poly function, and the JITed replacement would in the ideal case just be a jump instruction to a concrete implementation with a signature identical to the stub’s. Would that help?

gumby 4y ago

The TL;DR is: Swift strives for ABI stability, Rust does not. There's also a driveby comment that C++ doesn't really support dynamic linking because of all the templates in its standard library, which is a weird statement to make.

ahartmetz 4y ago
Yeah - the big, complicated parts of platform libraries for which post-release bugfixes make sense are usually not templated. The vector container class is a template, but Korean input method support is not.
- ahartmetz 4y ago
  (Actually, Korean was not a great choice of example. The Korean writing system isn't very complicated. Japanese would have been better - they even have three writing systems!)
- int_19h 4y ago
  Templates are not a problem if you're defining the complete set of valid specializations in advance, as would be the case here - that's what "extern template" is for.
pjmlp 4y ago
Indeed, on Windows C++ dynamic linking has always been available, sure there is the ABI issue, but that doesn't mean there aren't C++ frameworks being shipped as dynamic libraries, e.g. MFC, VCL, Qt,...

Shorel 4y ago

Couldn't or wouldn't? In some part, dynamic linking is one of the reasons for the travesty of dunces that's Docker and Kubernetes. With static linking, most of that redundancy is not needed.

imron 4y ago
Docker and kubernetes excel in a world of static binaries. Gotta love those 5mb bare images.
- heinrich5991 4y ago
  Why use docker in that case?

travisgriggs 4y ago

What a great write up. > Swift reserves a callee-preserved register for a method’s self argument (pointer) to make repeated calls faster. Cool? Years ago (2000 I think), I remember Eliot Miranda explaining that the biggest optimization—-over and above all the jit stuff—in the VisualWorks Smalltalk VM basically came down to something very similar: very specific management of registers for the commonest of calling patterns.

singularity2001 4y ago

Also swift hides almost all the complexity of memory safety in an ergonomic way. My preferred language by far.

bsaul 4y ago

this post makes me realize why swift is starting to look more and more bloated from an end-user pov. It was designed to be right from the beginning. instead of carefully backing off complex problems and slashing features to keep the design elegant, they just "went for it" at full speed. History will tell if that was a good thing.