7 ms·

Swift for TensorFlow Design Overview

198 points by asparagui 9y ago

yodon 8y ago
As a longtime fan of C#, this article and use case just pushed me from “why would anyone think they needed to invent Swift” to “wow that’s an extremely cool set of language design constraints!”
waleedka 8y ago
Does Swift have a large community beyond iOS apps? Last I used it was years ago, right after it was introduced by Apple. I'm curious if it has found growth in other areas.
- johnymontana 8y ago
  I've seen a few server-side Swift frameworks (for building web apps or API apps) such as Perfect[1] and Kitura[2] (which is backed by IBM). It's not clear to me how much uptake they have so far. [1] https://perfect.org/ https://perfect.org/ [2] https://www.kitura.io/ https://www.kitura.io/
  - deleted 8y ago
    [deleted]
  - thegayngler 8y ago
    I was starting to use Kitura but it seems the community is more in favor of Vapor right now. I moved on to Vapor as a result. IBM seems like it might abandon the project like they did their Swift Packages website.
    - alexashka 8y ago
      Vapor is the likely winner in terms of Swift frameworks, but this does not say much. The API's not stable, the framework is in very active development. I imagine things will settle down in a year or two - until then, there are so many great alternatives (other languages), that I don't see Swift gaining much ground.
    - pjmlp 8y ago
      For awhile it seemed IBM was about to pick Swift as they did before with Java, but then they lost steam.
      - geodel 8y ago
        Ah, was there some news about it or it just that IBM seems less active on Swift http/ssl stuff lately?
        pjmlp 8y ago
        They seem to be less active.
- stiGGG 8y ago
  Not really atm. There is a small community around server-side-swift which did some impressive work so far, most interesting project is IMO the Vapor Framework [1], but at least outside the Apple Dev community they didn’t get much attention so far. For general system programming, which would be also possible with swift, Rust seems currently a lot more popular. [1]: https://vapor.codes https://vapor.codes
  - rudedogg 8y ago
    For people not familiar, Vapor is close to releasing version 3.0, using https://github.com/apple/swift-nio https://github.com/apple/swift-nio. I've heard rumors that the performance is really good, but I haven't ran anything personally.
    - IAmEveryone 8y ago
      Swift itself in incredibly fast, at least in microbenchmarks. I last looked at the numbers a year or so ago, but IIRC it was about as fast as C, which is several (as in 5-6) orders of magnitude faster than, say, python. In terms of "level of abstraction" vs "Speed", Swift is definitely at a Pareto-Optimum (if that's the term for "can't get better in one dimension without losing something in the other). Microbenchmarks aren't the be-all yada yada yada... And it's somewhat ironic that the server framework is named Vapor. But for web apps that are computation-heavy, I think it would be a valuable option to have.
      - rudedogg 8y ago
        Yeah, here are some old Vapor benchmarks for curious people: https://medium.com/@codevapor/server-side-swift-vs-the-other-guys-2-speed-ca65b2f79505 https://medium.com/@codevapor/server-side-swift-vs-the-other... I drop in on the Vapor Slack group occasionally. I heard someone mention that version 3.0 was passing Go in benchmarks. Apparently they had some performance regressions integrating nio (it was just released last month) - not sure what the status is now.
  - deleted 8y ago
    [deleted]
- woolvalley 8y ago
  I wouldn't want to use it for anything that would evolve beyond 100k lines of swift code until the language improves it's build and IDE performance. And if your making a backend, why not use kotlin which is fairly similar, but you get the entire java ecosystem?
__erik 8y ago
At first I was fairly disappointed that Swift was chosen over Julia, and I still wish there was strong Julia support because Julia is a great language, but I've slowly been changing my mind and think Swift could be a really good choice long term. I also just really like Chris' work and trust him to make the right calls until proven otherwise.
- dnautics 8y ago
  I love julia, but it already has a TF library that does a lot of very nice things (like not having to specify a computational graph separately from the execution) and there are other really interesting machine learning libraries (like knet). Swift is a good choice because there are some reasonably good mobile targets, which Julia does not have. If you're deploying a ML model, you should use swift. If you're developing one, you should use julia.
  - Larrikin 8y ago
    What are the mobile targets besides an iOS device?
    - dnautics 8y ago
      Android?
    - fractallyte 8y ago
      I cannot understand why someone would downvote a perfectly valid question. At the moment, Swift is really only viable for Apple devices. Any other mobile targets are a distant dream.
  - jbrambleDC 8y ago
    pretty much the entire deep learning community is still using python 2.7. I have moved onto python 3.6, but Julia is still an unsafe bet because it has an unreliable community behind it.
    - 8y ago
gok 8y ago
> Automatic differentiation in Swift is a compiler IR transformation implemented with static analysis. Super cool to see this implemented at the language level like this.
AlexeyBrin 8y ago
Given the fact that the vast majority of people is still on Windows, using Swift (with zero official support for Windows) will artificially limit the use of the project outside the circle of the original developers. That being said, you can use Swift through WSL, but not directly on Windows.
- oscargrouch 8y ago
  > That being said, you can use Swift through WSL, but not directly on Windows. I thought so too, but i've checked the instructions to build on windows, and now there's a way to build it natively using Visual Studio cl(or clang-cl). Following the instructions here, it worked for me. https://github.com/apple/swift/blob/master/docs/WindowsBuild.md https://github.com/apple/swift/blob/master/docs/WindowsBuild...
  - favorited 8y ago
    For the record, "official" support is still not there because there's no CI environment for Windows (currently only macOS and a few versions of Ubuntu have CI). So changes aren't guaranteed to not break other platforms. There's upstream (in-tree) support for Darwin, Linux, Windows, Cygwin, FreeBSD, Android, PS4, and Haiku. But most of those don't have CI support yet. Same is true for architectures. There's support for ARM, i386, x86_64, PPC64 (BE + LE), and s390x. IIRC there's been some talk about how to extend the CI infrastructure to support more executor environments, but so far I don't think it's been a high enough priority.
  - J0-nas 8y ago
    I also tried it, and it didn't work for me. I asked about the error message in the swift forum and got no response... Using the prebuilt binary for ubuntu with the WSL is WAY more userfriendly. Another problem problem is proper tooling. So far my experience with the LSP implementation is a bit dissapointing..
bitL 8y ago
Still a bit more verbose than Python. What would you gain by doing the same in Swift actually? If you have to type more code to do some experiments, and still have to import Python libraries for extra functions as they do in that example, what's the selling point? (beside it being cool)
- favorited 8y ago
  Here's some context: https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md https://github.com/tensorflow/swift/blob/master/docs/WhySwif... Some benefits vs Python are static types (catching more errors at compile-time vs hours into a training run), no GIL, don't have to drop to C++ for higher performance. Some benefits vs some other (non-Python) languages are a shallow learning curve, small boilerplate, safety-by-default, and a growing community. Obviously there are places where Python and others have advantages over Swift too, and several of those are called out in the paper.
  - dnautics 8y ago
    Is it really likely that you'll catch a type exception hours into a training run? The way tf works it'll catch a shape exception before you even start training.
    - netheril96 8y ago
      I don't have experience of a type exception hours into run, but for me, deploying code to the test environment is long, so the edit-run cycle is very long. If the compiler can catch the errors locally, then edit-(compile)-run is shortened.
      - candiodari 8y ago
        This sort-of feedbacks on itself. I use estimators, so type errors occur far into a run, but not during training itself, and checkpoints mean very rapid recovery, so ... I think I actually stopped caring so much because the recovery is so fast. Big help for me was coding a little script that means I can just send a notification to my android from from cli and/or python, so I know when these things need attention.
sandGorgon 8y ago
Whoa...The Python interoperability is incredible. You can use numpy from swift..As well as load pickle files ! https://github.com/tensorflow/swift/blob/master/docs/DesignOverview.md#python-interoperability https://github.com/tensorflow/swift/blob/master/docs/DesignO...
Froyoh 8y ago
Swift for Tensorflow or Tensorflow for Swift?
- ScottBurson 8y ago
  Yes, it's right up there with "Windows Subsystem for Linux".

198 points by asparagui 9y ago

As a longtime fan of C#, this article and use case just pushed me from “why would anyone think they needed to invent Swift” to “wow that’s an extremely cool set of language design constraints!”

waleedka 8y ago

Does Swift have a large community beyond iOS apps? Last I used it was years ago, right after it was introduced by Apple. I'm curious if it has found growth in other areas.

johnymontana 8y ago
I've seen a few server-side Swift frameworks (for building web apps or API apps) such as Perfect[1] and Kitura[2] (which is backed by IBM). It's not clear to me how much uptake they have so far. [1] https://perfect.org/ https://perfect.org/ [2] https://www.kitura.io/ https://www.kitura.io/
- deleted 8y ago
  [deleted]
- thegayngler 8y ago
  I was starting to use Kitura but it seems the community is more in favor of Vapor right now. I moved on to Vapor as a result. IBM seems like it might abandon the project like they did their Swift Packages website.
  - alexashka 8y ago
    Vapor is the likely winner in terms of Swift frameworks, but this does not say much. The API's not stable, the framework is in very active development. I imagine things will settle down in a year or two - until then, there are so many great alternatives (other languages), that I don't see Swift gaining much ground.
  - pjmlp 8y ago
    For awhile it seemed IBM was about to pick Swift as they did before with Java, but then they lost steam.
    - geodel 8y ago
      Ah, was there some news about it or it just that IBM seems less active on Swift http/ssl stuff lately?
      - pjmlp 8y ago
        They seem to be less active.
stiGGG 8y ago
Not really atm. There is a small community around server-side-swift which did some impressive work so far, most interesting project is IMO the Vapor Framework [1], but at least outside the Apple Dev community they didn’t get much attention so far. For general system programming, which would be also possible with swift, Rust seems currently a lot more popular. [1]: https://vapor.codes https://vapor.codes
- rudedogg 8y ago
  For people not familiar, Vapor is close to releasing version 3.0, using https://github.com/apple/swift-nio https://github.com/apple/swift-nio. I've heard rumors that the performance is really good, but I haven't ran anything personally.
  - IAmEveryone 8y ago
    Swift itself in incredibly fast, at least in microbenchmarks. I last looked at the numbers a year or so ago, but IIRC it was about as fast as C, which is several (as in 5-6) orders of magnitude faster than, say, python. In terms of "level of abstraction" vs "Speed", Swift is definitely at a Pareto-Optimum (if that's the term for "can't get better in one dimension without losing something in the other). Microbenchmarks aren't the be-all yada yada yada... And it's somewhat ironic that the server framework is named Vapor. But for web apps that are computation-heavy, I think it would be a valuable option to have.
    - rudedogg 8y ago
      Yeah, here are some old Vapor benchmarks for curious people: https://medium.com/@codevapor/server-side-swift-vs-the-other-guys-2-speed-ca65b2f79505 https://medium.com/@codevapor/server-side-swift-vs-the-other... I drop in on the Vapor Slack group occasionally. I heard someone mention that version 3.0 was passing Go in benchmarks. Apparently they had some performance regressions integrating nio (it was just released last month) - not sure what the status is now.
- deleted 8y ago
  [deleted]
woolvalley 8y ago
I wouldn't want to use it for anything that would evolve beyond 100k lines of swift code until the language improves it's build and IDE performance. And if your making a backend, why not use kotlin which is fairly similar, but you get the entire java ecosystem?

__erik 8y ago

At first I was fairly disappointed that Swift was chosen over Julia, and I still wish there was strong Julia support because Julia is a great language, but I've slowly been changing my mind and think Swift could be a really good choice long term. I also just really like Chris' work and trust him to make the right calls until proven otherwise.

dnautics 8y ago
I love julia, but it already has a TF library that does a lot of very nice things (like not having to specify a computational graph separately from the execution) and there are other really interesting machine learning libraries (like knet). Swift is a good choice because there are some reasonably good mobile targets, which Julia does not have. If you're deploying a ML model, you should use swift. If you're developing one, you should use julia.
- Larrikin 8y ago
  What are the mobile targets besides an iOS device?
  - dnautics 8y ago
    Android?
  - fractallyte 8y ago
    I cannot understand why someone would downvote a perfectly valid question. At the moment, Swift is really only viable for Apple devices. Any other mobile targets are a distant dream.
- jbrambleDC 8y ago
  pretty much the entire deep learning community is still using python 2.7. I have moved onto python 3.6, but Julia is still an unsafe bet because it has an unreliable community behind it.
  - 8y ago

gok 8y ago

> Automatic differentiation in Swift is a compiler IR transformation implemented with static analysis. Super cool to see this implemented at the language level like this.

AlexeyBrin 8y ago

Given the fact that the vast majority of people is still on Windows, using Swift (with zero official support for Windows) will artificially limit the use of the project outside the circle of the original developers. That being said, you can use Swift through WSL, but not directly on Windows.

oscargrouch 8y ago
> That being said, you can use Swift through WSL, but not directly on Windows. I thought so too, but i've checked the instructions to build on windows, and now there's a way to build it natively using Visual Studio cl(or clang-cl). Following the instructions here, it worked for me. https://github.com/apple/swift/blob/master/docs/WindowsBuild.md https://github.com/apple/swift/blob/master/docs/WindowsBuild...
- favorited 8y ago
  For the record, "official" support is still not there because there's no CI environment for Windows (currently only macOS and a few versions of Ubuntu have CI). So changes aren't guaranteed to not break other platforms. There's upstream (in-tree) support for Darwin, Linux, Windows, Cygwin, FreeBSD, Android, PS4, and Haiku. But most of those don't have CI support yet. Same is true for architectures. There's support for ARM, i386, x86_64, PPC64 (BE + LE), and s390x. IIRC there's been some talk about how to extend the CI infrastructure to support more executor environments, but so far I don't think it's been a high enough priority.
- J0-nas 8y ago
  I also tried it, and it didn't work for me. I asked about the error message in the swift forum and got no response... Using the prebuilt binary for ubuntu with the WSL is WAY more userfriendly. Another problem problem is proper tooling. So far my experience with the LSP implementation is a bit dissapointing..

bitL 8y ago

Still a bit more verbose than Python. What would you gain by doing the same in Swift actually? If you have to type more code to do some experiments, and still have to import Python libraries for extra functions as they do in that example, what's the selling point? (beside it being cool)

favorited 8y ago
Here's some context: https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md https://github.com/tensorflow/swift/blob/master/docs/WhySwif... Some benefits vs Python are static types (catching more errors at compile-time vs hours into a training run), no GIL, don't have to drop to C++ for higher performance. Some benefits vs some other (non-Python) languages are a shallow learning curve, small boilerplate, safety-by-default, and a growing community. Obviously there are places where Python and others have advantages over Swift too, and several of those are called out in the paper.
- dnautics 8y ago
  Is it really likely that you'll catch a type exception hours into a training run? The way tf works it'll catch a shape exception before you even start training.
  - netheril96 8y ago
    I don't have experience of a type exception hours into run, but for me, deploying code to the test environment is long, so the edit-run cycle is very long. If the compiler can catch the errors locally, then edit-(compile)-run is shortened.
    - candiodari 8y ago
      This sort-of feedbacks on itself. I use estimators, so type errors occur far into a run, but not during training itself, and checkpoints mean very rapid recovery, so ... I think I actually stopped caring so much because the recovery is so fast. Big help for me was coding a little script that means I can just send a notification to my android from from cli and/or python, so I know when these things need attention.

sandGorgon 8y ago

Whoa...The Python interoperability is incredible. You can use numpy from swift..As well as load pickle files ! https://github.com/tensorflow/swift/blob/master/docs/DesignOverview.md#python-interoperability https://github.com/tensorflow/swift/blob/master/docs/DesignO...

Froyoh 8y ago

Swift for Tensorflow or Tensorflow for Swift?

ScottBurson 8y ago
Yes, it's right up there with "Windows Subsystem for Linux".