26 ms·

Swift for TensorFlow Shuts Down

nafizh 6y ago
It seems this was written on the wall once Chris Lattner left the project. It's a shame since this could have been a real breakthrough in machine learning combining performance and ease. But the success of JAX means Google probably doesn't feel the investment is worth it.
- swagonomixxx 6y ago
  I mean, I'm fairly certain that this project is the reason Chris Lattner joined Google; it was somehow in his contract somewhere. He just found a way to shoehorn himself in there and make sure that the language that he designed (Swift) is used for something more than just iOS apps. Nothing against him at all, I think most folks would want to push their brainchild as far as they can, but that's just what it looks like from the outside. Since ML has been a large rage for the last few years, and TensorFlow is one of the most popular frameworks that ML engineers use (although I use and prefer PyTorch) it seemed like the "perfect storm" for Swift to break out of the iOS mold. But it was clear to me (and many of my peers) that while this was an interesting project, the need for it is really low, and that nobody is going to throw away years of experience with libraries like TensorFlow and PyTorch (via Python) in the trash just so we can use Swift, which for me isn't really that crazy of a language as all the hype makes it out to be, and I'm sure the same is true for many ML devs and researchers alike.
msie 6y ago
Swift compared to Python was getting way too complex. Why would anyone learn Swift for data science when Python will do?
- jtdev 6y ago
  Agreed, Swift has an obsession with functional programming that is just so off-putting to me and I suspect many others.
  - jshier 6y ago
    This is hilariously wrong. Actual fans of functional programming would disagree with you, as there are several things Swift doesn't do to enable pure functional programming.
  - astrange 6y ago
    The type system is kind of sort of vaguely ML/Haskell-like despite looking like C. Is that what you mean? It certainly isn't a functional programming language. It's intentionally imperative.
    - jtdev 6y ago
      Yes, Swift is fundamentally imperative... but it has functional concepts littered throughout in ways that are obtuse, unhelpful, and unavoidable.
- geophile 6y ago
  I am not involved in data science at all, I have extensive Python experience, and did some work in Swift for a couple of years. Putting aside the Python 2/3 transition, I think that Python does a good job of introducing new features carefully. For example, the "X if P else Y" syntax. While I prefer the C approach, the Python approach works fine and fits in the language well. I had a mild dislike of Swift from the beginning, but it was a lot more palatable than working in Objective C. What bothered me was all the gimmicky syntax. Sometimes "let" unwraps an optional, sometimes it doesn't. ! and ? seem badly overused. ARC doesn't compose well with other language features. There is a ton of syntax whose only purpose appears to be to stuff logic into and around assignment syntax. Then it got worse, with major language changes across versions 3-5, each one introducing lots of new concepts and syntax. And of course, Swift is a proprietary Apple language, (in the same way that C# is tied to Microsoft). When I stopped doing iOS programming, I very happily but Swift behind me. I saw the TensorFlow additions to the language and thought the whole thing bizarre. I put my reaction down to my lack of involvement with data science. Maybe I'd appreciate those changes more if I knew more about data science. But I also remember thinking that it was quite odd to make such extensive additions to the language to support this one application area, which was very, very far from the language's sweet spot.
- smabie 6y ago
  I mean, python doesn't do it very well at all. that's not say I would use Swift, but I'd choose Julia any day over Python.
pixel_tracing 6y ago
Wow this project had so much potential going for it. What does Chris Lattner have to say about this?
- the_damn_nava 6y ago
  Same questions here, Jeez. Would love to know Jeremy Howard's too.
  - fuddle 6y ago
    I immediately thought of Jeremy Howard, he has mentioned this project in a couple of interviews.
    - Tenoke 6y ago
      He didn't just mentioned it, he thought the courses of an iteration of fast.ai in Swift as it was allegedly the way forward and more useful for students (paraphrasing).
  - BadInformatics 6y ago
    I remember an interview [1] with him a few months back where he seemed to have stopped pursuing it after Chris Lattner's departure. Sylvain Gugger, who was pretty involved in fastai and prototyping a Swift port, also left to join Huggingface a while ago. [1] https://www.youtube.com/watch?v=t2V2kf2gNnI https://www.youtube.com/watch?v=t2V2kf2gNnI
- throwaway1777 6y ago
  He left Google a while ago, so I doubt he’s surprised.
BadInformatics 6y ago
Looks like @throw6606 was right: https://news.ycombinator.com/item?id=24533937 https://news.ycombinator.com/item?id=24533937. Does anyone know the status of https://ai.facebook.com/blog/paving-the-way-for-software-20- https://ai.facebook.com/blog/paving-the-way-for-software-20-...?
- willxinc 6y ago
  Looks like your link was cut off: https://ai.facebook.com/blog/paving-the-way-for-software-20-with-kotlin/ https://ai.facebook.com/blog/paving-the-way-for-software-20-... (Missing the -with-kotlin at the end)
- xenihn 6y ago
  Reading that thread, I'm always so confused by HN posters who ask for evidence that would be impossible to provide without self-incriminating. Are they asking disingenuously, or are they just clueless?
  - bartleby_ 6y ago
    Always makes me laugh. Perhaps something like 10% clueless, 80% disingenuous and 10% feel they are somehow entitled to information that's obviously sensitive or private.
  - ergocoder 6y ago
    They aren't clueless. This kind of tactics has been used since the beginning of time. There was no way to produce a hard evidence, and everyone knew that.
- mikkelam 6y ago
  Sure. People were asserting that the second chris lattner left google, see https://news.ycombinator.com/item?id=22161780 https://news.ycombinator.com/item?id=22161780
bsaul 6y ago
Swift now being a 100% Apple-sponsored & owned project again makes me a bit nervous. Anyone knows if chris latner is at least using swift in his new company ? I have the feeling swift never really worked in the server side, data science is now officially a failure, and all that is left is now a very niche market of 100% native mobile development. I love this language, but i'm eager to see it handled by a proper foundation with big names behind, as it's a really great language.
- mottiden 6y ago
  What do you mean by data science being a failure?
  - nt591 6y ago
    Presumably "Swift for data science" is a failure, not the idea of data science as a whole
- MrPowers 6y ago
  It seemed like Swift looked like a promising language for data science a few years ago. I'm not familiar with Swift. Can anyone provide additional context as to why the language seemed promising for data?
  - SatvikBeri 6y ago
    I disagree with the pitch, but the pitch I heard when this was announced was: * Swift lets you write code that's as concise as Python while still being fast, and therefore allows you to get away from the problem of having to write ML code in multiple languages (TensorFlow is actually mostly C++, which makes it difficult for a Python developer to debug.) * That the lack of static typing is a major pain point in Python, and Swift's type system would help write code with fewer errors.
    - deleted 6y ago
      [deleted]
adamnemecek 6y ago
This was a matter of time.
- sedatk 6y ago
  Why?
  - adamnemecek 6y ago
    No one was using it.
sedatk 6y ago
I don’t see any justification for the shut down. Does anyone know why?
- marcinzm 6y ago
  It was Chris Lattner's baby and when he left Google no one cared to push it forward. The Google vs. Apple fights over ads and privacy probably didn't help the case.
- therealmarv 6y ago
  maybe Apple's own replacements with AI components?
high_derivative 6y ago
S4TF from my (external) perspective was always more a (very cool!) programming language research project than anything else, at least since Lattner left. I would personally assume the shutdown was due to a combination of reasons: - There simply being no good reason for Python users to ever move to Swift. There is no big painpoint being solved for the broad ML user community - Organisational momentum lost with Lattner leaving - General disarray of TensorFlow as a project, the parallel rise of Jax from/within prominent Google orgs, it being the cool new thing
- skohan 6y ago
  - There simply being no good reason for Python users to ever move to Swift. There is no big painpoint being solved for the broad ML user community As a swift and python user, I would have been really happy to be able to use swift for ML applications. Having a half way decent type system solves so many problems. But while I can see that from my vantage point, I know for a vast majority of the ML community python is "good enough" and there would be quite a lot of inertia to overcome in that regard which is probably not realistic.
  - marcinzm 6y ago
    I'm not sure if the Swift type system is half way decent in terms of machine learning applications. At least not without a lot of work. The data tends to be in specialized in-memory formats that tend to be opaque to the type system. The actual matrix operations involve dimension parameters that are likewise opaque to most type systems. So you still need to run code for the actual tensor graph compilation to validate if everything is correct. Python in notebooks does that instantly, Swift I suspect has a much longer delay.
    - skohan 6y ago
      I don't see any reason the swift type system could not accommodate the types required for ML. I did a lot of work with computer graphics in Swift, which is all linear algebra also, and the type system was a huge benefit. Also I don't see why Swift would perform worse than python in any case. If python is fast, it's because it's wrapping a C++ library. There's no reason Swift could not wrap the same library, but on top Swift could communicate better type information back to the user.
The_rationalist 6y ago
It would have been much more productive to improve the Kotlin programming language support instead.
- BadInformatics 6y ago
  Well, it's happening: https://news.ycombinator.com/item?id=25128251 https://news.ycombinator.com/item?id=25128251
joeblau 6y ago
I'm looking through the meeting notes[1] linked from the archive PR[2] and it looks like there hasn't been much activity on the project recently. Can anyone with more insight provide an explanation on what happened? [1] - https://docs.google.com/document/d/1Fm56p5rV1t2Euh6WLtBFKGqI43ozC3EIjReyLk-LCLU/edit https://docs.google.com/document/d/1Fm56p5rV1t2Euh6WLtBFKGqI... [2] - https://github.com/tensorflow/swift/commit/1b1381ccb89a342bababd1511cc39b8ba7b0708b https://github.com/tensorflow/swift/commit/1b1381ccb89a342ba...
- rezahussain 6y ago
  https://twitter.com/bradlarson/status/1360230872658698244 https://twitter.com/bradlarson/status/1360230872658698244 https://twitter.com/texasmichelle/status/1360287563898974211 https://twitter.com/texasmichelle/status/1360287563898974211 I think we just have to wait for the meeting video to be posted
A12-B 6y ago
I recall many users telling me swift would be the new python and that it would take over the machine learning domain.
BasedInfra 6y ago
How is swift for stuff beyond applications for the apple ecosystem?
- rezahussain 6y ago
  I was using this project for my stock trading bot. Way nicer to use Swift than python for this, and I have rewritten this bot several times in different languages + frameworks (objc php java python swift tf1.0 theano keras s4tf) since 2012. Doing authentication for brokerage apis + correctness from pulling from the db is easier to get right in Swift than python bc of the static typing + grand central dispatch for parallel queries. It is also way more productive for me to have the compiler catch issues at the beginning instead of 6 hours into a data generation job where the python interpreter sees a None for something I didn't think of.
ipsum2 6y ago
No surprises there. There were maybe a total of 5 people excited about adding automatic differentiation to Swift. Too bad they didn't try improving Julia instead.
- _Wintermute 6y ago
  I remember their terrible article trying to justify why Swift was the right choice out of a load of more appropriate languages. I'm still annoyed they didn't go with Julia.
  - cglong 6y ago
    Source? Curious to read that :D
    - _Wintermute 6y ago
      Here you are: https://github.com/tensorflow/swift/blob/main/docs/WhySwiftForTensorFlow.md#which-languages-fit-our-project-requirements https://github.com/tensorflow/swift/blob/main/docs/WhySwiftF...
      - sanderjd 6y ago
        I always figured they would have chosen rust if it weren't for Lattner's sway (real or perceived) in the Swift compiler development community. One underestimated advantage of rust over swift in my view is that it has interested practitioners across a much more diverse range of problems spaces. I think it was always going to be hard to convince people to use a language for for ML when most of its practitioners are solely mobile developers. But people already used rust for lots of different kinds of data processing heavy use cases.
        johnnycerberus 6y ago
        You can't be serious. Rust would just replace C++ in the current formula, you can't use Rust as if it was Python. ML people don't need to waste time proving memory properties at compile time, cmon! Indeed, there were alternatives. Two already come to my mind, one is Scala (which syntax-wise is strikingly similar to Python) and Julia. Both having features like type-safety, macros, JIT, FFIs, etc.
meteor333 6y ago
I love Swift and it's potential for use cases beyond just iOS ecosystem, but I think Apple never fostered or encouraged the open source community to involve in its development. It's another classic example where it's simply not enough to open source something, you have to really allow and encourage the developers to contribute to increase the adoption.
- Hamuko 6y ago
  What's Apple's incentive to do so?
  - darzu 6y ago
    Same as MSFT's with C#?
  - skohan 6y ago
    Theoretically if Swift was an extremely marketable language across many domains, there would be more qualified developers available to work on software for Apple products.
    - Hamuko 6y ago
      But hasn't Apple managed to force a lot of developers to pick up the language anyways?
      - skohan 6y ago
        Apple is one of the most successful organizations on the planet, and yes they have plenty of developers in tow. I am just trying to give an example of how it could help their bottom line to push open source swift more aggressively. If I had to guess, they don't do it because: 1. they don't think the resources it would take would represent a good ROI, 2. and/or they think it's good for them that developers who invest in their ecosystem have skills which are not transferrable to other domains
- skohan 6y ago
skohan 6y ago
It's a shame. I had high hopes at the beginning that S4TF - and the investment in Swift from Google - would help Swift break out of the iOS ghetto and cement it as a mainstream language. Swift's a delightful language to use. It has a lot of the nice things about Rust's type system, but is a heck of a lot easier to use at the expense of a bit of performance. For a lot of use cases, I think this is a great value proposition. ML/data science is a great example where practitioners could benefit from a more grown-up type system than what Python has on offer, but would still be able to keep low level details at arms length. I think Serverless would be another ideal use-case for Swift, where the productivity, clarity and correctness tools it offers would be a huge benefit. Some very interesting things came out of the S4TF project - like the work on autodiff, and python interop. It's a shame the project never really seemed to get legs, it seems like it just languished for years with no clear direction. These days I do most of my work in Rust, and I'm happy to do so because the tooling, community, and ecosystem is really amazing. But there are a lot of language concepts and features I miss from swift. I guess it goes to show that the governance and availability of a language have a lot more to do with adoption than the merits of the language itself.
- wrinkl3 6y ago
  > I think Serverless would be another ideal use-case for Swift, where the productivity, clarity and correctness tools it offers would be a huge benefit. I agree, but sadly none of the big cloud providers has any interest in pushing it - Google's got Go, AWS and Azure seem focused on Typescript.
  - chris_st 6y ago
    I know AWS lets you just deploy a docker container, so you can use whatever tools you want -- anyone know about Azure and Google cloud?
    - skohan 6y ago
      Yes theoretically I think there is nothing stopping you from shipping Swift code on AWS. I am pretty sure you could do it in a lambda and you don't even need a docker container. If you're talking about a container workflow, I'm pretty sure every cloud provider will support this just fine currently.
akmittal 6y ago
Swift for tensorflow never made any sense. I remember only reason it started was one main developer on team liked swift. Tensorflow for nodejs makes much more sense, node community is lit bigger and not sponsored by big corp.
- swagonomixxx 6y ago
  Well, there is already Tensorflow.js [0]. [0]: https://www.tensorflow.org/js https://www.tensorflow.org/js.
belval 6y ago
Swift for TensorFlow was a nice idea on paper, but you need more than ideas to push a project forward. You need people who actually want to use your product. For all the comments on HN about how great Swift was, there were hundreds if not thousands of ML engineer and research scientist who did not know it existed and frankly did not really care about it either. Swift for Tensorflow was not addressing the issues of ML in productions and felt like "Just another framework" but with the added handicap of needing its users to learn an entirely new language which is not something most scientists want to do. One might argue that engineering might be more interested, but in a lot of cases the science team will push the models and then the engineering team will put them in production, leaving very little room to migrate from TF/Pytorch to Swift for TF.
- SatvikBeri 6y ago
  It's interesting to compare Swift to Julia here – Julia had a base of people who absolutely loved it from the 0.2 days, and has managed to draw many users from Python/Matlab/R. There are many reasons for that, but overall Julia had something that really appealed to scientists: syntax they liked + competitive-with-C performance. In contrast, S4TF's target audience seems to have been Swift developers, and they didn't really appeal to the data science crowd.
  - belval 6y ago
    Precisely, S4TF felt like a startup product with no market research. It really felt like the only reason Swift was chosen was because the main dev created it.
  - Traster 6y ago
    We see this all the time right, the next big ML company is going to come from some scrappy startup with a precise focus on what to do (either because they're geniuses or because they're just the lucky idiot amongst 10,000 ML startups), not from some behemoth that decides "we'll eat that for breakfast". Swift4ML suffers from the fact that Swift exists. To make S4ML work you need to abandon atleast some of the things that makes Swift Swift.
jtdev 6y ago
Not surprised that this project had trouble gaining traction given Swift's obsession with shoe-horning functional concepts into every crevice of the language, e.g., I can't get through a single Swift tutorial without having to get into closures.
- jshier 6y ago
  Closures aren't really a functional concept, they can just be used functionally. You'll see a big reduction in closure usage when the new concurrency features ship.
- rudedogg 6y ago
  For a short time, maybe around Swift 2/3, functional features got some focus, but the language moved in a different direction. The only functional features I can think of are closures, map/reduce/etc., and value types. I have a lot of criticisms about Swift but this one seems weird/outdated to me.
layoutIfNeeded 6y ago
Cool! One special case less in the Swift language (automatic differentiation).
- eigenspace 6y ago
  Autodiff already landed into the upstream Swift implementation. Too late for that.
  - layoutIfNeeded 6y ago
    Crap :S Let’s hope it atrophies over time.
    - perturbation 6y ago
      Let's hope not, I like autodiff (and this project :( ).
chrisaiv 6y ago
Is if safe to assume that this is the main driver for this event? https://www.businessinsider.com/chris-lattner-former-apple-google-engineer-swift-ai-startup-sifive-2020-2 https://www.businessinsider.com/chris-lattner-former-apple-g...
- rezahussain 6y ago
  I think its the issues they are seeing here https://forums.swift.org/t/differentiable-programming-for-gradient-based-machine-learning/42147/97 https://forums.swift.org/t/differentiable-programming-for-gr... Until the video of the last meeting is posted im guessing maybe its bc the execs set a deadline and it got passed
- microtherion 6y ago
  It's kind of odd. Chris had a huge impact at Apple with LLVM and Swift still going strong and forming the backbone of the developer tools. But I'm not sure he has anything of lasting value to show for his jobs since then. Maybe evidence that for great engineers, just like other great people, fortuitous timing and circumstances are a big part of the story.
UncleOxidant 6y ago
This seems like good news for Julia. Some wondered why Google would develop S4TF with an LLVM backend when Julia already existed and had better support for things like linear algebra than Swift had.
cs702 6y ago
S4TF lost out, not to Python, but to Python's AI/ML ecosystem -- people, projects, libraries, frameworks. Despite its many shortcomings, Python has become the lingua franca of AI and ML, to the point that whenever I come across a newly published AI or ML paper that I find interesting, I expect to be able to find code implementing it in Python. For example, yesterday I saw a post on HN about approximating self-attention matrices in transformers, which have O(n²) computational cost, with a seemingly clever approach that has O(n) computational cost: https://news.ycombinator.com/item?id=26105455 https://news.ycombinator.com/item?id=26105455 . "Huh, that looks interesting," I thought, "let me see if I can find some code." Three clicks later, I found myself at https://github.com/mlpen/Nystromformer https://github.com/mlpen/Nystromformer -- the official implementation... in Python. A few moments later, I was playing with this thingamajiggy. No other language comes close to having this kind of ecosystem in AI and ML. Julia still has a shot at becoming a viable alternative to Python, especially if the Julia developers can shorten the "time to first interactive plot" to make it as fast as Python's, but they face an uphill battle against such an entrenched ecosystem.
- onelovetwo 6y ago
  Its becoming like the same grasp js has on the web...which is sad to see.
  - belval 6y ago
    This is a false equivalence of the first order. What makes JS impossible to dethrone is that whatever replaces it would need to be supported by a critical mass of web browsers, which just won't happen. AI/ML on the other hand has no such restriction. Your new startup could build all their ML models in Java with DeepLearning4J and it would probably work fine. Python is used because it's easy to work with for experimenting, which is a big part of research and for production your can extract your weight and graph structure to run on TVM/TensorRT/ONNX. There is no equivalent vendor lock-in.
    - cutler 6y ago
      Don't forget Web Assembly. That could easily dethrone JS but it will take time. All the major browsers have been evergreen for some time now so it's not like we're back in the early 2000s.
jhatemyjob 6y ago
Not surprising. Swift isn't a very good language. Objective-C is better in a lot of ways.
xiaodai 6y ago
Knew this was gonna happen. Doomed from the beginning. Should ve chosen Julia
amkkma 6y ago
Julia is now the only viable next gen ML language
Waterluvian 6y ago
Stupid question. Why is it not "TensorFlow for Swift"?
- davekeck 6y ago
  I had the same question for the naming of Windows Subsystem for Linux (WSL), which IMO should be named "Linux Subsystem for Windows" The best explanation I can think of is that the people naming it wanted the first word to emphasize the thing that's most important to them. In Microsoft's case, that's of course Windows. Perhaps the same is true for the S4TF folks.
- ddragon 6y ago
  In this case it's not just a library like the wrappers for tensorflow already present in many languages (including python), it's something like a Swift language fork in which the language constructs are analyzed by the forked compiler during compilation in order to directly generate an equivalent tensorflow execution graph (without manually creating a tape through library calls). Thus it's a Swift (compiler plugin) for tensorflow. The intention was to merge the fork into the main language making Swift differentiation tools first class within the language.
  - Waterluvian 6y ago
    That cleared it up. Thanks.
iamaziz 6y ago
Long live Python!
flipgimble 6y ago
Another sad lesson why no open source developer should trust Google with their time and effort. Swift is a gem of a language with an unfortunately Apple-centric ecosystem, but at least you know Apple won't abandon the effort.
- mcast 6y ago
  Let's be honest, Google still contributes more heavily to OSS, whitepapers, and security research than Apple does.
  - FridgeSeal 6y ago
    And that’s good. But they are notorious for being poor at keeping some projects and products alive.
FartyMcFarter 6y ago
Looks like Chris Lattner's "exec commitment and strong roadmaps" comment was not that meaningful: https://twitter.com/clattner_llvm/status/1222032740897284097?s=19 https://twitter.com/clattner_llvm/status/1222032740897284097...
tokipin 6y ago
I think people are deceived by Swift's syntax. It looks simple by syntactic sugar, but it's not actually any more approachable than other systems languages like C# or Java. Given that a lot of work in ML is exploratory, it doesn't seem like a good fit compared to a hackable scripting language like Python. I would bet against SwiftUI for similar reasons.
- socialdemocrat 6y ago
  The syntax makes sense in an Objective-C setting because it was matching Smalltalk. It is a very nice syntax for object-oriented programming. But for functional programming it is a terrible syntax, and I think Swift kind of missed the boat on this. I think they should have pushed the replacement to Objective-C into a far more Smalltalk oriented direction as that is what the ecosystem and syntax was really tailored towards. I do a lot of functional style programming in Julia today, but I also love how Smalltalk works. There is a place for both styles. Swift IMHO is the worst of both worlds. This very OO style syntax mixed in with functional programming makes everything kind of messy. I easily get confused when looking at Swift code for this reason. When you use anonymous functions, you don't want named arguments. That is an idiotic idea. However if you want to pass a message to a random object, then named arguments is quite nice. I feel Chris Lattner is not a guy who could really appreciate the Smalltalk heritage of Objective-C.
lostmsu 6y ago
And TensorFlow for C# is alive and kicking: https://losttech.software/gradient.html https://losttech.software/gradient.html I guess this is sort of an ad. Swift was a weird choice for statically typed TensorFlow, being only popular on the platform, that does not have GPU/TPU support in TensorFlow, which is, basically, a requirement for any serious work. The fact, that they had to fork the compiler did not help either. TensorFlow for C# is much like TensorFlow for Swift. There's a statically typed binding to the core types with the rest of the TensorFlow API available through an open source Python.NET project [1]. Unlike Swift version though, that second part is also mostly statically typed, so you can get IDE autocompletion hints. Also, .NET runtime has native support for dynamic languages. Like with Swift for TensorFlow, we ported all recent interesting neural network architectures (and maybe even more): Transformers (GPT-2) [2], CNNs (YOLOv4) [3], Q-learning (e.g. RL, actor-critic) [4], and even some cool ones, that have lots of unexplored potential like Siren [5] (this one uses sin/cos as activation functions). Although we have not worked on automatic differentiation yet, unlike Swift, it will not need compiler support. .NET (and Java) can inspect and generate code at runtime, so autodiff can be implemented in a library. We also have integration with Unity ML Agents for training robotic agents [4]. [1] https://github.com/pythonnet/pythonnet/ https://github.com/pythonnet/pythonnet/ [2] https://github.com/losttech/Gradient-Samples/tree/master/GPT-2 https://github.com/losttech/Gradient-Samples/tree/master/GPT... [3] https://github.com/losttech/YOLOv4 https://github.com/losttech/YOLOv4 [4] https://github.com/losttech/Gradient-Samples/tree/master/RL-MLAgents https://github.com/losttech/Gradient-Samples/tree/master/RL-... [5] https://github.com/losttech/Siren https://github.com/losttech/Siren https://vsitzmann.github.io/siren/ https://vsitzmann.github.io/siren/
- astrange 6y ago
  > Swift was a weird choice for statically typed TensorFlow, being only popular on the platform, that does not have GPU/TPU support in TensorFlow, which is, basically, a requirement for any serious work. The fact, that they had to fork the compiler did not help either. This is available now (not sure if it's in mainline). https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...
amelius 6y ago
Good. Nobody benefits from fragmentation in deep learning.
stmw 6y ago
This is sad for the project itself, but I predict that the coming programming competition will be Rust vs. Go, not Swift -- as Swift was always a distant third among the "new" languages. And of course the more mature languages will continue to be popular and widely used for decades. In my opinion, Rust will end up winning this contest, but others have perfectly good arguments against that view. Unfortunately for server-side Swift and the many good people who've worked on it, there seems little chance for it, in TensorFlow or other usecases.
- cutler 6y ago
  I think Kotlin will become the dominant JVM language in time.
  - nsonha 6y ago
    that's obvious to me, what sounds more like a prediction is kotlin to be a dominant application development language (fullstack/scripting) natively compiled or on the JVM
- yyyk 6y ago
  The idea there's one language for everything was false even during C's heyday. Your list contains languages for very different uses. Rust will eventually win low-level programming from C++, but massively networked applications will always be slightly easier in Go. Swift is a bytecode language, it competes against .NET (mainly C#) and JVM (Java and variants), not Rust or Go. IMHO Swift seems to losing when not tied to Apple's ecosystem, its future will be as some Apple-niche language.
  - stmw 6y ago
    I guess it's all in how you classify it... I'm not sure why "bytecode" vs. "non-bytecode" are an important distinction in the language's usecases, as opposed to implementation. For example, .Net supports both static compilation and bytecode interpretation, as do most LLVM-backed languages.
mark_l_watson 6y ago
I am disappointed, but so it goes. I started writing a Swift AI book a while back and I had planned on abut 20% of the book would be on S4TF. I still have lots of other material, but obviously I am not going to include S4TF material. BTW, the Haskell bindings for TensorFlow have been actively maintained for years. The latest update was 3 days ago.
- liuliu 6y ago
  It is much easier to maintain a language binding than a full toolchain. There are a lot of activities in Swift mainline, and I can only imagine how much work it must be to maintain the S4TF fork.
tmp538394722 6y ago
> Added language-integrated differentiable programming into the Swift language. This work continues in the official Swift compiler. added lots of novel paradigms and rarely trodden code paths to the compiler and then peaced out. I hope the inherent complexity added doesn’t impede future Swift development much, or is expeditiously deleted.
arensc 6y ago
Every time I see a post, about TensorFlow for Swift. There seems to be so many misconceptions about why this it was created. The top one being, oh it's Chris L's baby. As a Software Engineer, who's been working on teams for the past 15 years and seeing the craft devolve, and the market saturate with people whom don't understand that fundamentals but memorize the frameworks, you need a language like Swift to bridge that gap. It's hard enough to get people to program to an interface, let alone communicate what they are gonna return from a function or how a function behaves. 70% of people don't understand software engineering is legitimately just plumbing, they think its pretty much an artistic endeavour with no rhyme or reason. This makes it pretty difficult to stay on the same page, when building a product to scale up. Having a strong type system solves that problem. When working with ML people my question 100% of the time is what does that function, return what type?, so I can build off what you are doing, or even debugging requires understanding of the type and the values. It seems like team orientated programming is foreign to most people. Swift has a low cognitive load pushing you to solve problems, with Software Engineering and reducing communication lead time between engineers. All languages eventually converge Swift, I think Chris L has solved the language UI problem.
- nestorD 6y ago
  Most people regret that Google did not invest in Julia but, to my mind, it misses the point. Swift for TensorFlow's unique selling point was bringing a strong type system and ML+autodiferentiation together. That, for me, would be a huge boost to push ML code to production quality.
- pjmlp 6y ago
  There is nothing special about Swift that ML languages, or .NET eco-system didn't offered already, Chris L might be a very good technical person, with several achievements, however Swift without iOS is just yet another ML/Modula-3 language with "modern" syntax.
O_H_E 6y ago
I think it is important to note (for those who didn't read) that much of the work on the Core of the language is being upstreamed. https://forums.swift.org/t/differentiable-programming-for-gradient-based-machine-learning/42147 https://forums.swift.org/t/differentiable-programming-for-gr... https://github.com/rxwei/swift-evolution/blob/autodiff/proposals/0000-differentiable-programming.md https://github.com/rxwei/swift-evolution/blob/autodiff/propo...
socialdemocrat 6y ago
Back when I was doing a lot of Swift programming, this seemed like such a great idea. Since then I moved on to Julia. And when I looked at the ML code for Swift and compared it to Julia, I was thinking "How on earth could anyone think this was ever a good idea?" Doing machine learning stuff in Julia is simply much more user friendly than doing the same in Swift. Swift is nice for iOS development, but I think in data science and Machine Learning Julia will always be a much better choice. It is a pity Google did not go for Julia. Julia has gotten exceptionally far despite limited resources. With Google style resources, Julia could have been massive. Despite modest investment, the Julia JIT beats almost anything out there. It would have been amazing to see what they could have pulled off if they had put the kind of resources that was put into the JavaScript V8 JIT into the Julia JIT. Julia uses a method JIT, which are quick to implement, but which makes latency gets bad. Meaning if you load a whole new package, running the first function can cause some delay. With a JavaScript style tracer JIT added into the mix one could have probably managed to have both high performance and low latency. Alternatively with more investment they could have had better precompile caching, meaning libraries that had been loaded in the past would JIT really fast. Julia does that today, but it could have worked a lot better than it currently does. It isn't that it can't be done, but it just requires resources.
- freedomben 6y ago
  Julia is fascinating to me, and I don't do any ML (currently). Julia seems like such a well designed language. I'm excited to see where it goes.
  - sida 6y ago
    As weird as it sounds, the only thing that stops me from trying Julia is 1 indexing
    - adgjlsfhk1 6y ago
      IMO, Julia is good enough that you might not notice. you can use constructions like `for i in axes(V)` that will automatically give you the indexes, or `V[begin:begin+2]` to get the first 2 elements. Also higher order functions eg. `map` (and broadcasting) mean that you will rarely be manually working with indexes.
travisgriggs 6y ago
I'm curious which of the many "evolutions" that went into Swift in the last few years were motivated heavily by its dance with TensorFlow. If there were any, can we take them back out? Now that the marriage is over? One of the things that frustrates me with a lot of the languages I'm having to grok these days, is they often lack consistency, because it's one carrot after another to get buy in from different sub communities. And in the end it tastes like vegetable soup--edible, but not tasty (if you love vegetable soup, this reach of an analogy probably won't work for you).
- WoodenChair 6y ago
  I think this is a great observation, aka too many chefs. My criticism of Swift, is that it has become a “kitchen-sink” language over the past seven years (I’ve been using it since the first public beta), like C++. So many features have been added that it’s gone from being a “I can keep this language in my head” language to a “I have to routinely lookup how this thing works even though I program in Swift regularly” language. It’s gotten too big IMHO and could have used a good editor instead of design by committee. The recent function builder and property wrapper additions (added it seems because the SwiftUI folks wanted them) are particularly emblematic of how it has become less readable. The recent language additions have made me much less enthusiastic about it because like C++ it’s becoming a language where each person uses their own subset and style, instead of having a common elegance.
sriku 6y ago
It's awesome to see this whole comment section be about Julia! As a community, evebtually we'll realize that we dot have the bandwidth to keep inventing the same thing over and over just so a new technique can be used efficiently. Currently, Julia is the only language that makes this kind of deep reuse possible by bein efficient through just-ahead-of-time compilation that specialises well through open multiple-dispatch.
cutler 6y ago
So that's it then - another decade of unchallenged dominance of AI by a Betamax scripting language designed by accretion for early 90s CPU architecture. I just don't get the whole Python for ML craze.
paultopia 6y ago
wait, a google project got shut down? shocking.
pjmlp 6y ago
As expected since Chris Lattner left Google, and TensorFlow conferences kept having content on S4TF. They should just had chooen Julia.
nestorD 6y ago
> Added language-integrated differentiable programming into the Swift language. This work continues in the official Swift compiler. While the readme tells us that the project is in archive mode, it also tells us that the differentiable programming part is still worked on by the compiler team. Is there further information on that?
deleted 6y ago
[deleted]
max_ 6y ago
What is really needed in the ML industry is a code independent "neural net interpreter" with an api that binds to any programming language. How hard is it to have a program that takes in neural network architectures & trained waits so as to do inference?
mariokostelac 6y ago
It is unclear to me whether it's archived because they've done everything they wanted or they just stopped all the efforts?
- eigenspace 6y ago
  The latter. The project was not close to completion.
stadia42 6y ago
My reading of the discussion at https://forums.swift.org/t/swift-concurrency-roadmap/41611 https://forums.swift.org/t/swift-concurrency-roadmap/41611: the Google Swift team members expressed hope for a bit more community involvement in the Swift language planning, and Apple folks very politely told them to shove it where the sun don't shine. My guess: Google concluded that investing in a language developed behind closed doors of another company is a bad deal. I don't blame Apple, it may be a good business decision for them, and of course it is completely within their rights.

It seems this was written on the wall once Chris Lattner left the project. It's a shame since this could have been a real breakthrough in machine learning combining performance and ease. But the success of JAX means Google probably doesn't feel the investment is worth it.

swagonomixxx 6y ago
I mean, I'm fairly certain that this project is the reason Chris Lattner joined Google; it was somehow in his contract somewhere. He just found a way to shoehorn himself in there and make sure that the language that he designed (Swift) is used for something more than just iOS apps. Nothing against him at all, I think most folks would want to push their brainchild as far as they can, but that's just what it looks like from the outside. Since ML has been a large rage for the last few years, and TensorFlow is one of the most popular frameworks that ML engineers use (although I use and prefer PyTorch) it seemed like the "perfect storm" for Swift to break out of the iOS mold. But it was clear to me (and many of my peers) that while this was an interesting project, the need for it is really low, and that nobody is going to throw away years of experience with libraries like TensorFlow and PyTorch (via Python) in the trash just so we can use Swift, which for me isn't really that crazy of a language as all the hype makes it out to be, and I'm sure the same is true for many ML devs and researchers alike.

msie 6y ago

Swift compared to Python was getting way too complex. Why would anyone learn Swift for data science when Python will do?

jtdev 6y ago
Agreed, Swift has an obsession with functional programming that is just so off-putting to me and I suspect many others.
- jshier 6y ago
  This is hilariously wrong. Actual fans of functional programming would disagree with you, as there are several things Swift doesn't do to enable pure functional programming.
- astrange 6y ago
  The type system is kind of sort of vaguely ML/Haskell-like despite looking like C. Is that what you mean? It certainly isn't a functional programming language. It's intentionally imperative.
  - jtdev 6y ago
    Yes, Swift is fundamentally imperative... but it has functional concepts littered throughout in ways that are obtuse, unhelpful, and unavoidable.
geophile 6y ago
I am not involved in data science at all, I have extensive Python experience, and did some work in Swift for a couple of years. Putting aside the Python 2/3 transition, I think that Python does a good job of introducing new features carefully. For example, the "X if P else Y" syntax. While I prefer the C approach, the Python approach works fine and fits in the language well. I had a mild dislike of Swift from the beginning, but it was a lot more palatable than working in Objective C. What bothered me was all the gimmicky syntax. Sometimes "let" unwraps an optional, sometimes it doesn't. ! and ? seem badly overused. ARC doesn't compose well with other language features. There is a ton of syntax whose only purpose appears to be to stuff logic into and around assignment syntax. Then it got worse, with major language changes across versions 3-5, each one introducing lots of new concepts and syntax. And of course, Swift is a proprietary Apple language, (in the same way that C# is tied to Microsoft). When I stopped doing iOS programming, I very happily but Swift behind me. I saw the TensorFlow additions to the language and thought the whole thing bizarre. I put my reaction down to my lack of involvement with data science. Maybe I'd appreciate those changes more if I knew more about data science. But I also remember thinking that it was quite odd to make such extensive additions to the language to support this one application area, which was very, very far from the language's sweet spot.
smabie 6y ago
I mean, python doesn't do it very well at all. that's not say I would use Swift, but I'd choose Julia any day over Python.

pixel_tracing 6y ago

Wow this project had so much potential going for it. What does Chris Lattner have to say about this?

the_damn_nava 6y ago
Same questions here, Jeez. Would love to know Jeremy Howard's too.
- fuddle 6y ago
  I immediately thought of Jeremy Howard, he has mentioned this project in a couple of interviews.
  - Tenoke 6y ago
    He didn't just mentioned it, he thought the courses of an iteration of fast.ai in Swift as it was allegedly the way forward and more useful for students (paraphrasing).
- BadInformatics 6y ago
  I remember an interview [1] with him a few months back where he seemed to have stopped pursuing it after Chris Lattner's departure. Sylvain Gugger, who was pretty involved in fastai and prototyping a Swift port, also left to join Huggingface a while ago. [1] https://www.youtube.com/watch?v=t2V2kf2gNnI https://www.youtube.com/watch?v=t2V2kf2gNnI
throwaway1777 6y ago
He left Google a while ago, so I doubt he’s surprised.

BadInformatics 6y ago

Looks like @throw6606 was right: https://news.ycombinator.com/item?id=24533937 https://news.ycombinator.com/item?id=24533937. Does anyone know the status of https://ai.facebook.com/blog/paving-the-way-for-software-20- https://ai.facebook.com/blog/paving-the-way-for-software-20-...?

willxinc 6y ago
Looks like your link was cut off: https://ai.facebook.com/blog/paving-the-way-for-software-20-with-kotlin/ https://ai.facebook.com/blog/paving-the-way-for-software-20-... (Missing the -with-kotlin at the end)
xenihn 6y ago
Reading that thread, I'm always so confused by HN posters who ask for evidence that would be impossible to provide without self-incriminating. Are they asking disingenuously, or are they just clueless?
- bartleby_ 6y ago
  Always makes me laugh. Perhaps something like 10% clueless, 80% disingenuous and 10% feel they are somehow entitled to information that's obviously sensitive or private.
- ergocoder 6y ago
  They aren't clueless. This kind of tactics has been used since the beginning of time. There was no way to produce a hard evidence, and everyone knew that.
mikkelam 6y ago
Sure. People were asserting that the second chris lattner left google, see https://news.ycombinator.com/item?id=22161780 https://news.ycombinator.com/item?id=22161780

bsaul 6y ago

Swift now being a 100% Apple-sponsored & owned project again makes me a bit nervous. Anyone knows if chris latner is at least using swift in his new company ? I have the feeling swift never really worked in the server side, data science is now officially a failure, and all that is left is now a very niche market of 100% native mobile development. I love this language, but i'm eager to see it handled by a proper foundation with big names behind, as it's a really great language.

mottiden 6y ago
What do you mean by data science being a failure?
- nt591 6y ago
  Presumably "Swift for data science" is a failure, not the idea of data science as a whole
MrPowers 6y ago
It seemed like Swift looked like a promising language for data science a few years ago. I'm not familiar with Swift. Can anyone provide additional context as to why the language seemed promising for data?
- SatvikBeri 6y ago
  I disagree with the pitch, but the pitch I heard when this was announced was: * Swift lets you write code that's as concise as Python while still being fast, and therefore allows you to get away from the problem of having to write ML code in multiple languages (TensorFlow is actually mostly C++, which makes it difficult for a Python developer to debug.) * That the lack of static typing is a major pain point in Python, and Swift's type system would help write code with fewer errors.
  - deleted 6y ago
    [deleted]

adamnemecek 6y ago

This was a matter of time.

sedatk 6y ago
Why?
- adamnemecek 6y ago
  No one was using it.

sedatk 6y ago

I don’t see any justification for the shut down. Does anyone know why?

marcinzm 6y ago
It was Chris Lattner's baby and when he left Google no one cared to push it forward. The Google vs. Apple fights over ads and privacy probably didn't help the case.
therealmarv 6y ago
maybe Apple's own replacements with AI components?

high_derivative 6y ago

S4TF from my (external) perspective was always more a (very cool!) programming language research project than anything else, at least since Lattner left. I would personally assume the shutdown was due to a combination of reasons: - There simply being no good reason for Python users to ever move to Swift. There is no big painpoint being solved for the broad ML user community - Organisational momentum lost with Lattner leaving - General disarray of TensorFlow as a project, the parallel rise of Jax from/within prominent Google orgs, it being the cool new thing

skohan 6y ago
- There simply being no good reason for Python users to ever move to Swift. There is no big painpoint being solved for the broad ML user community As a swift and python user, I would have been really happy to be able to use swift for ML applications. Having a half way decent type system solves so many problems. But while I can see that from my vantage point, I know for a vast majority of the ML community python is "good enough" and there would be quite a lot of inertia to overcome in that regard which is probably not realistic.
- marcinzm 6y ago
  I'm not sure if the Swift type system is half way decent in terms of machine learning applications. At least not without a lot of work. The data tends to be in specialized in-memory formats that tend to be opaque to the type system. The actual matrix operations involve dimension parameters that are likewise opaque to most type systems. So you still need to run code for the actual tensor graph compilation to validate if everything is correct. Python in notebooks does that instantly, Swift I suspect has a much longer delay.
  - skohan 6y ago
    I don't see any reason the swift type system could not accommodate the types required for ML. I did a lot of work with computer graphics in Swift, which is all linear algebra also, and the type system was a huge benefit. Also I don't see why Swift would perform worse than python in any case. If python is fast, it's because it's wrapping a C++ library. There's no reason Swift could not wrap the same library, but on top Swift could communicate better type information back to the user.

The_rationalist 6y ago

It would have been much more productive to improve the Kotlin programming language support instead.

BadInformatics 6y ago
Well, it's happening: https://news.ycombinator.com/item?id=25128251 https://news.ycombinator.com/item?id=25128251

joeblau 6y ago

I'm looking through the meeting notes[1] linked from the archive PR[2] and it looks like there hasn't been much activity on the project recently. Can anyone with more insight provide an explanation on what happened? [1] - https://docs.google.com/document/d/1Fm56p5rV1t2Euh6WLtBFKGqI43ozC3EIjReyLk-LCLU/edit https://docs.google.com/document/d/1Fm56p5rV1t2Euh6WLtBFKGqI... [2] - https://github.com/tensorflow/swift/commit/1b1381ccb89a342bababd1511cc39b8ba7b0708b https://github.com/tensorflow/swift/commit/1b1381ccb89a342ba...

rezahussain 6y ago
https://twitter.com/bradlarson/status/1360230872658698244 https://twitter.com/bradlarson/status/1360230872658698244 https://twitter.com/texasmichelle/status/1360287563898974211 https://twitter.com/texasmichelle/status/1360287563898974211 I think we just have to wait for the meeting video to be posted

A12-B 6y ago

I recall many users telling me swift would be the new python and that it would take over the machine learning domain.

BasedInfra 6y ago

How is swift for stuff beyond applications for the apple ecosystem?

rezahussain 6y ago
I was using this project for my stock trading bot. Way nicer to use Swift than python for this, and I have rewritten this bot several times in different languages + frameworks (objc php java python swift tf1.0 theano keras s4tf) since 2012. Doing authentication for brokerage apis + correctness from pulling from the db is easier to get right in Swift than python bc of the static typing + grand central dispatch for parallel queries. It is also way more productive for me to have the compiler catch issues at the beginning instead of 6 hours into a data generation job where the python interpreter sees a None for something I didn't think of.

ipsum2 6y ago

No surprises there. There were maybe a total of 5 people excited about adding automatic differentiation to Swift. Too bad they didn't try improving Julia instead.

_Wintermute 6y ago
I remember their terrible article trying to justify why Swift was the right choice out of a load of more appropriate languages. I'm still annoyed they didn't go with Julia.
- cglong 6y ago
  Source? Curious to read that :D
  - _Wintermute 6y ago
    Here you are: https://github.com/tensorflow/swift/blob/main/docs/WhySwiftForTensorFlow.md#which-languages-fit-our-project-requirements https://github.com/tensorflow/swift/blob/main/docs/WhySwiftF...
    - sanderjd 6y ago
      I always figured they would have chosen rust if it weren't for Lattner's sway (real or perceived) in the Swift compiler development community. One underestimated advantage of rust over swift in my view is that it has interested practitioners across a much more diverse range of problems spaces. I think it was always going to be hard to convince people to use a language for for ML when most of its practitioners are solely mobile developers. But people already used rust for lots of different kinds of data processing heavy use cases.
      - johnnycerberus 6y ago
        You can't be serious. Rust would just replace C++ in the current formula, you can't use Rust as if it was Python. ML people don't need to waste time proving memory properties at compile time, cmon! Indeed, there were alternatives. Two already come to my mind, one is Scala (which syntax-wise is strikingly similar to Python) and Julia. Both having features like type-safety, macros, JIT, FFIs, etc.

meteor333 6y ago

I love Swift and it's potential for use cases beyond just iOS ecosystem, but I think Apple never fostered or encouraged the open source community to involve in its development. It's another classic example where it's simply not enough to open source something, you have to really allow and encourage the developers to contribute to increase the adoption.

Hamuko 6y ago
What's Apple's incentive to do so?
- darzu 6y ago
  Same as MSFT's with C#?
- skohan 6y ago
  Theoretically if Swift was an extremely marketable language across many domains, there would be more qualified developers available to work on software for Apple products.
  - Hamuko 6y ago
    But hasn't Apple managed to force a lot of developers to pick up the language anyways?
    - skohan 6y ago
      Apple is one of the most successful organizations on the planet, and yes they have plenty of developers in tow. I am just trying to give an example of how it could help their bottom line to push open source swift more aggressively. If I had to guess, they don't do it because: 1. they don't think the resources it would take would represent a good ROI, 2. and/or they think it's good for them that developers who invest in their ecosystem have skills which are not transferrable to other domains
skohan 6y ago

skohan 6y ago

It's a shame. I had high hopes at the beginning that S4TF - and the investment in Swift from Google - would help Swift break out of the iOS ghetto and cement it as a mainstream language. Swift's a delightful language to use. It has a lot of the nice things about Rust's type system, but is a heck of a lot easier to use at the expense of a bit of performance. For a lot of use cases, I think this is a great value proposition. ML/data science is a great example where practitioners could benefit from a more grown-up type system than what Python has on offer, but would still be able to keep low level details at arms length. I think Serverless would be another ideal use-case for Swift, where the productivity, clarity and correctness tools it offers would be a huge benefit. Some very interesting things came out of the S4TF project - like the work on autodiff, and python interop. It's a shame the project never really seemed to get legs, it seems like it just languished for years with no clear direction. These days I do most of my work in Rust, and I'm happy to do so because the tooling, community, and ecosystem is really amazing. But there are a lot of language concepts and features I miss from swift. I guess it goes to show that the governance and availability of a language have a lot more to do with adoption than the merits of the language itself.

wrinkl3 6y ago
> I think Serverless would be another ideal use-case for Swift, where the productivity, clarity and correctness tools it offers would be a huge benefit. I agree, but sadly none of the big cloud providers has any interest in pushing it - Google's got Go, AWS and Azure seem focused on Typescript.
- chris_st 6y ago
  I know AWS lets you just deploy a docker container, so you can use whatever tools you want -- anyone know about Azure and Google cloud?
  - skohan 6y ago
    Yes theoretically I think there is nothing stopping you from shipping Swift code on AWS. I am pretty sure you could do it in a lambda and you don't even need a docker container. If you're talking about a container workflow, I'm pretty sure every cloud provider will support this just fine currently.

akmittal 6y ago

Swift for tensorflow never made any sense. I remember only reason it started was one main developer on team liked swift. Tensorflow for nodejs makes much more sense, node community is lit bigger and not sponsored by big corp.

swagonomixxx 6y ago
Well, there is already Tensorflow.js [0]. [0]: https://www.tensorflow.org/js https://www.tensorflow.org/js.

belval 6y ago

Swift for TensorFlow was a nice idea on paper, but you need more than ideas to push a project forward. You need people who actually want to use your product. For all the comments on HN about how great Swift was, there were hundreds if not thousands of ML engineer and research scientist who did not know it existed and frankly did not really care about it either. Swift for Tensorflow was not addressing the issues of ML in productions and felt like "Just another framework" but with the added handicap of needing its users to learn an entirely new language which is not something most scientists want to do. One might argue that engineering might be more interested, but in a lot of cases the science team will push the models and then the engineering team will put them in production, leaving very little room to migrate from TF/Pytorch to Swift for TF.

SatvikBeri 6y ago
It's interesting to compare Swift to Julia here – Julia had a base of people who absolutely loved it from the 0.2 days, and has managed to draw many users from Python/Matlab/R. There are many reasons for that, but overall Julia had something that really appealed to scientists: syntax they liked + competitive-with-C performance. In contrast, S4TF's target audience seems to have been Swift developers, and they didn't really appeal to the data science crowd.
- belval 6y ago
  Precisely, S4TF felt like a startup product with no market research. It really felt like the only reason Swift was chosen was because the main dev created it.
- Traster 6y ago
  We see this all the time right, the next big ML company is going to come from some scrappy startup with a precise focus on what to do (either because they're geniuses or because they're just the lucky idiot amongst 10,000 ML startups), not from some behemoth that decides "we'll eat that for breakfast". Swift4ML suffers from the fact that Swift exists. To make S4ML work you need to abandon atleast some of the things that makes Swift Swift.

jtdev 6y ago

Not surprised that this project had trouble gaining traction given Swift's obsession with shoe-horning functional concepts into every crevice of the language, e.g., I can't get through a single Swift tutorial without having to get into closures.

jshier 6y ago
Closures aren't really a functional concept, they can just be used functionally. You'll see a big reduction in closure usage when the new concurrency features ship.
rudedogg 6y ago
For a short time, maybe around Swift 2/3, functional features got some focus, but the language moved in a different direction. The only functional features I can think of are closures, map/reduce/etc., and value types. I have a lot of criticisms about Swift but this one seems weird/outdated to me.

layoutIfNeeded 6y ago

Cool! One special case less in the Swift language (automatic differentiation).

eigenspace 6y ago
Autodiff already landed into the upstream Swift implementation. Too late for that.
- layoutIfNeeded 6y ago
  Crap :S Let’s hope it atrophies over time.
  - perturbation 6y ago
    Let's hope not, I like autodiff (and this project :( ).

chrisaiv 6y ago

Is if safe to assume that this is the main driver for this event? https://www.businessinsider.com/chris-lattner-former-apple-google-engineer-swift-ai-startup-sifive-2020-2 https://www.businessinsider.com/chris-lattner-former-apple-g...

rezahussain 6y ago
I think its the issues they are seeing here https://forums.swift.org/t/differentiable-programming-for-gradient-based-machine-learning/42147/97 https://forums.swift.org/t/differentiable-programming-for-gr... Until the video of the last meeting is posted im guessing maybe its bc the execs set a deadline and it got passed
microtherion 6y ago
It's kind of odd. Chris had a huge impact at Apple with LLVM and Swift still going strong and forming the backbone of the developer tools. But I'm not sure he has anything of lasting value to show for his jobs since then. Maybe evidence that for great engineers, just like other great people, fortuitous timing and circumstances are a big part of the story.

UncleOxidant 6y ago

This seems like good news for Julia. Some wondered why Google would develop S4TF with an LLVM backend when Julia already existed and had better support for things like linear algebra than Swift had.

cs702 6y ago

S4TF lost out, not to Python, but to Python's AI/ML ecosystem -- people, projects, libraries, frameworks. Despite its many shortcomings, Python has become the lingua franca of AI and ML, to the point that whenever I come across a newly published AI or ML paper that I find interesting, I expect to be able to find code implementing it in Python. For example, yesterday I saw a post on HN about approximating self-attention matrices in transformers, which have O(n²) computational cost, with a seemingly clever approach that has O(n) computational cost: https://news.ycombinator.com/item?id=26105455 https://news.ycombinator.com/item?id=26105455 . "Huh, that looks interesting," I thought, "let me see if I can find some code." Three clicks later, I found myself at https://github.com/mlpen/Nystromformer https://github.com/mlpen/Nystromformer -- the official implementation... in Python. A few moments later, I was playing with this thingamajiggy. No other language comes close to having this kind of ecosystem in AI and ML. Julia still has a shot at becoming a viable alternative to Python, especially if the Julia developers can shorten the "time to first interactive plot" to make it as fast as Python's, but they face an uphill battle against such an entrenched ecosystem.

onelovetwo 6y ago
Its becoming like the same grasp js has on the web...which is sad to see.
- belval 6y ago
  This is a false equivalence of the first order. What makes JS impossible to dethrone is that whatever replaces it would need to be supported by a critical mass of web browsers, which just won't happen. AI/ML on the other hand has no such restriction. Your new startup could build all their ML models in Java with DeepLearning4J and it would probably work fine. Python is used because it's easy to work with for experimenting, which is a big part of research and for production your can extract your weight and graph structure to run on TVM/TensorRT/ONNX. There is no equivalent vendor lock-in.
  - cutler 6y ago
    Don't forget Web Assembly. That could easily dethrone JS but it will take time. All the major browsers have been evergreen for some time now so it's not like we're back in the early 2000s.

jhatemyjob 6y ago

Not surprising. Swift isn't a very good language. Objective-C is better in a lot of ways.

xiaodai 6y ago

Knew this was gonna happen. Doomed from the beginning. Should ve chosen Julia

amkkma 6y ago

Julia is now the only viable next gen ML language

Waterluvian 6y ago

Stupid question. Why is it not "TensorFlow for Swift"?

davekeck 6y ago
I had the same question for the naming of Windows Subsystem for Linux (WSL), which IMO should be named "Linux Subsystem for Windows" The best explanation I can think of is that the people naming it wanted the first word to emphasize the thing that's most important to them. In Microsoft's case, that's of course Windows. Perhaps the same is true for the S4TF folks.
ddragon 6y ago
In this case it's not just a library like the wrappers for tensorflow already present in many languages (including python), it's something like a Swift language fork in which the language constructs are analyzed by the forked compiler during compilation in order to directly generate an equivalent tensorflow execution graph (without manually creating a tape through library calls). Thus it's a Swift (compiler plugin) for tensorflow. The intention was to merge the fork into the main language making Swift differentiation tools first class within the language.
- Waterluvian 6y ago
  That cleared it up. Thanks.

iamaziz 6y ago

Long live Python!

flipgimble 6y ago

Another sad lesson why no open source developer should trust Google with their time and effort. Swift is a gem of a language with an unfortunately Apple-centric ecosystem, but at least you know Apple won't abandon the effort.

mcast 6y ago
Let's be honest, Google still contributes more heavily to OSS, whitepapers, and security research than Apple does.
- FridgeSeal 6y ago
  And that’s good. But they are notorious for being poor at keeping some projects and products alive.

FartyMcFarter 6y ago

Looks like Chris Lattner's "exec commitment and strong roadmaps" comment was not that meaningful: https://twitter.com/clattner_llvm/status/1222032740897284097?s=19 https://twitter.com/clattner_llvm/status/1222032740897284097...

tokipin 6y ago

I think people are deceived by Swift's syntax. It looks simple by syntactic sugar, but it's not actually any more approachable than other systems languages like C# or Java. Given that a lot of work in ML is exploratory, it doesn't seem like a good fit compared to a hackable scripting language like Python. I would bet against SwiftUI for similar reasons.

socialdemocrat 6y ago
The syntax makes sense in an Objective-C setting because it was matching Smalltalk. It is a very nice syntax for object-oriented programming. But for functional programming it is a terrible syntax, and I think Swift kind of missed the boat on this. I think they should have pushed the replacement to Objective-C into a far more Smalltalk oriented direction as that is what the ecosystem and syntax was really tailored towards. I do a lot of functional style programming in Julia today, but I also love how Smalltalk works. There is a place for both styles. Swift IMHO is the worst of both worlds. This very OO style syntax mixed in with functional programming makes everything kind of messy. I easily get confused when looking at Swift code for this reason. When you use anonymous functions, you don't want named arguments. That is an idiotic idea. However if you want to pass a message to a random object, then named arguments is quite nice. I feel Chris Lattner is not a guy who could really appreciate the Smalltalk heritage of Objective-C.

lostmsu 6y ago

And TensorFlow for C# is alive and kicking: https://losttech.software/gradient.html https://losttech.software/gradient.html I guess this is sort of an ad. Swift was a weird choice for statically typed TensorFlow, being only popular on the platform, that does not have GPU/TPU support in TensorFlow, which is, basically, a requirement for any serious work. The fact, that they had to fork the compiler did not help either. TensorFlow for C# is much like TensorFlow for Swift. There's a statically typed binding to the core types with the rest of the TensorFlow API available through an open source Python.NET project [1]. Unlike Swift version though, that second part is also mostly statically typed, so you can get IDE autocompletion hints. Also, .NET runtime has native support for dynamic languages. Like with Swift for TensorFlow, we ported all recent interesting neural network architectures (and maybe even more): Transformers (GPT-2) [2], CNNs (YOLOv4) [3], Q-learning (e.g. RL, actor-critic) [4], and even some cool ones, that have lots of unexplored potential like Siren [5] (this one uses sin/cos as activation functions). Although we have not worked on automatic differentiation yet, unlike Swift, it will not need compiler support. .NET (and Java) can inspect and generate code at runtime, so autodiff can be implemented in a library. We also have integration with Unity ML Agents for training robotic agents [4]. [1] https://github.com/pythonnet/pythonnet/ https://github.com/pythonnet/pythonnet/ [2] https://github.com/losttech/Gradient-Samples/tree/master/GPT-2 https://github.com/losttech/Gradient-Samples/tree/master/GPT... [3] https://github.com/losttech/YOLOv4 https://github.com/losttech/YOLOv4 [4] https://github.com/losttech/Gradient-Samples/tree/master/RL-MLAgents https://github.com/losttech/Gradient-Samples/tree/master/RL-... [5] https://github.com/losttech/Siren https://github.com/losttech/Siren https://vsitzmann.github.io/siren/ https://vsitzmann.github.io/siren/

astrange 6y ago
> Swift was a weird choice for statically typed TensorFlow, being only popular on the platform, that does not have GPU/TPU support in TensorFlow, which is, basically, a requirement for any serious work. The fact, that they had to fork the compiler did not help either. This is available now (not sure if it's in mainline). https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...

amelius 6y ago

Good. Nobody benefits from fragmentation in deep learning.

stmw 6y ago

This is sad for the project itself, but I predict that the coming programming competition will be Rust vs. Go, not Swift -- as Swift was always a distant third among the "new" languages. And of course the more mature languages will continue to be popular and widely used for decades. In my opinion, Rust will end up winning this contest, but others have perfectly good arguments against that view. Unfortunately for server-side Swift and the many good people who've worked on it, there seems little chance for it, in TensorFlow or other usecases.

cutler 6y ago
I think Kotlin will become the dominant JVM language in time.
- nsonha 6y ago
  that's obvious to me, what sounds more like a prediction is kotlin to be a dominant application development language (fullstack/scripting) natively compiled or on the JVM
yyyk 6y ago
The idea there's one language for everything was false even during C's heyday. Your list contains languages for very different uses. Rust will eventually win low-level programming from C++, but massively networked applications will always be slightly easier in Go. Swift is a bytecode language, it competes against .NET (mainly C#) and JVM (Java and variants), not Rust or Go. IMHO Swift seems to losing when not tied to Apple's ecosystem, its future will be as some Apple-niche language.
- stmw 6y ago
  I guess it's all in how you classify it... I'm not sure why "bytecode" vs. "non-bytecode" are an important distinction in the language's usecases, as opposed to implementation. For example, .Net supports both static compilation and bytecode interpretation, as do most LLVM-backed languages.

mark_l_watson 6y ago

I am disappointed, but so it goes. I started writing a Swift AI book a while back and I had planned on abut 20% of the book would be on S4TF. I still have lots of other material, but obviously I am not going to include S4TF material. BTW, the Haskell bindings for TensorFlow have been actively maintained for years. The latest update was 3 days ago.

liuliu 6y ago
It is much easier to maintain a language binding than a full toolchain. There are a lot of activities in Swift mainline, and I can only imagine how much work it must be to maintain the S4TF fork.

tmp538394722 6y ago

> Added language-integrated differentiable programming into the Swift language. This work continues in the official Swift compiler. added lots of novel paradigms and rarely trodden code paths to the compiler and then peaced out. I hope the inherent complexity added doesn’t impede future Swift development much, or is expeditiously deleted.

arensc 6y ago

Every time I see a post, about TensorFlow for Swift. There seems to be so many misconceptions about why this it was created. The top one being, oh it's Chris L's baby. As a Software Engineer, who's been working on teams for the past 15 years and seeing the craft devolve, and the market saturate with people whom don't understand that fundamentals but memorize the frameworks, you need a language like Swift to bridge that gap. It's hard enough to get people to program to an interface, let alone communicate what they are gonna return from a function or how a function behaves. 70% of people don't understand software engineering is legitimately just plumbing, they think its pretty much an artistic endeavour with no rhyme or reason. This makes it pretty difficult to stay on the same page, when building a product to scale up. Having a strong type system solves that problem. When working with ML people my question 100% of the time is what does that function, return what type?, so I can build off what you are doing, or even debugging requires understanding of the type and the values. It seems like team orientated programming is foreign to most people. Swift has a low cognitive load pushing you to solve problems, with Software Engineering and reducing communication lead time between engineers. All languages eventually converge Swift, I think Chris L has solved the language UI problem.

nestorD 6y ago
Most people regret that Google did not invest in Julia but, to my mind, it misses the point. Swift for TensorFlow's unique selling point was bringing a strong type system and ML+autodiferentiation together. That, for me, would be a huge boost to push ML code to production quality.
pjmlp 6y ago
There is nothing special about Swift that ML languages, or .NET eco-system didn't offered already, Chris L might be a very good technical person, with several achievements, however Swift without iOS is just yet another ML/Modula-3 language with "modern" syntax.

O_H_E 6y ago

I think it is important to note (for those who didn't read) that much of the work on the Core of the language is being upstreamed. https://forums.swift.org/t/differentiable-programming-for-gradient-based-machine-learning/42147 https://forums.swift.org/t/differentiable-programming-for-gr... https://github.com/rxwei/swift-evolution/blob/autodiff/proposals/0000-differentiable-programming.md https://github.com/rxwei/swift-evolution/blob/autodiff/propo...

socialdemocrat 6y ago

Back when I was doing a lot of Swift programming, this seemed like such a great idea. Since then I moved on to Julia. And when I looked at the ML code for Swift and compared it to Julia, I was thinking "How on earth could anyone think this was ever a good idea?" Doing machine learning stuff in Julia is simply much more user friendly than doing the same in Swift. Swift is nice for iOS development, but I think in data science and Machine Learning Julia will always be a much better choice. It is a pity Google did not go for Julia. Julia has gotten exceptionally far despite limited resources. With Google style resources, Julia could have been massive. Despite modest investment, the Julia JIT beats almost anything out there. It would have been amazing to see what they could have pulled off if they had put the kind of resources that was put into the JavaScript V8 JIT into the Julia JIT. Julia uses a method JIT, which are quick to implement, but which makes latency gets bad. Meaning if you load a whole new package, running the first function can cause some delay. With a JavaScript style tracer JIT added into the mix one could have probably managed to have both high performance and low latency. Alternatively with more investment they could have had better precompile caching, meaning libraries that had been loaded in the past would JIT really fast. Julia does that today, but it could have worked a lot better than it currently does. It isn't that it can't be done, but it just requires resources.

freedomben 6y ago
Julia is fascinating to me, and I don't do any ML (currently). Julia seems like such a well designed language. I'm excited to see where it goes.
- sida 6y ago
  As weird as it sounds, the only thing that stops me from trying Julia is 1 indexing
  - adgjlsfhk1 6y ago
    IMO, Julia is good enough that you might not notice. you can use constructions like `for i in axes(V)` that will automatically give you the indexes, or `V[begin:begin+2]` to get the first 2 elements. Also higher order functions eg. `map` (and broadcasting) mean that you will rarely be manually working with indexes.

travisgriggs 6y ago

I'm curious which of the many "evolutions" that went into Swift in the last few years were motivated heavily by its dance with TensorFlow. If there were any, can we take them back out? Now that the marriage is over? One of the things that frustrates me with a lot of the languages I'm having to grok these days, is they often lack consistency, because it's one carrot after another to get buy in from different sub communities. And in the end it tastes like vegetable soup--edible, but not tasty (if you love vegetable soup, this reach of an analogy probably won't work for you).

WoodenChair 6y ago
I think this is a great observation, aka too many chefs. My criticism of Swift, is that it has become a “kitchen-sink” language over the past seven years (I’ve been using it since the first public beta), like C++. So many features have been added that it’s gone from being a “I can keep this language in my head” language to a “I have to routinely lookup how this thing works even though I program in Swift regularly” language. It’s gotten too big IMHO and could have used a good editor instead of design by committee. The recent function builder and property wrapper additions (added it seems because the SwiftUI folks wanted them) are particularly emblematic of how it has become less readable. The recent language additions have made me much less enthusiastic about it because like C++ it’s becoming a language where each person uses their own subset and style, instead of having a common elegance.

sriku 6y ago

It's awesome to see this whole comment section be about Julia! As a community, evebtually we'll realize that we dot have the bandwidth to keep inventing the same thing over and over just so a new technique can be used efficiently. Currently, Julia is the only language that makes this kind of deep reuse possible by bein efficient through just-ahead-of-time compilation that specialises well through open multiple-dispatch.

cutler 6y ago

So that's it then - another decade of unchallenged dominance of AI by a Betamax scripting language designed by accretion for early 90s CPU architecture. I just don't get the whole Python for ML craze.

paultopia 6y ago

wait, a google project got shut down? shocking.

pjmlp 6y ago

As expected since Chris Lattner left Google, and TensorFlow conferences kept having content on S4TF. They should just had chooen Julia.

nestorD 6y ago

> Added language-integrated differentiable programming into the Swift language. This work continues in the official Swift compiler. While the readme tells us that the project is in archive mode, it also tells us that the differentiable programming part is still worked on by the compiler team. Is there further information on that?

deleted 6y ago

[deleted]

max_ 6y ago

What is really needed in the ML industry is a code independent "neural net interpreter" with an api that binds to any programming language. How hard is it to have a program that takes in neural network architectures & trained waits so as to do inference?

mariokostelac 6y ago

It is unclear to me whether it's archived because they've done everything they wanted or they just stopped all the efforts?

eigenspace 6y ago
The latter. The project was not close to completion.

stadia42 6y ago

My reading of the discussion at https://forums.swift.org/t/swift-concurrency-roadmap/41611 https://forums.swift.org/t/swift-concurrency-roadmap/41611: the Google Swift team members expressed hope for a bit more community involvement in the Swift language planning, and Apple folks very politely told them to shove it where the sun don't shine. My guess: Google concluded that investing in a language developed behind closed doors of another company is a bad deal. I don't blame Apple, it may be a good business decision for them, and of course it is completely within their rights.