15 ms·

Julia 1.4

243 points by pella 7y ago

vasili111 7y ago
What is the state of Julia dataframe? Is it ready for production use? How is the Julia dataframe performance in comparison with R and Python dataframes?
- oxinabox 7y ago
  My employer uses DataFrames.jl in production. It's fine, nothing wrong with it. Used to be a bit unsafe to unwarey uses, now it's safe by default and you need to do a bit extra to get all the performance. It's worth knowing it is more like R's DataFrames than like Pandas. It is getting pretty close to a 1.0 release. Probably a few months out (One more minor, then if all goes well 1.0 a month or so later) Further one should know that there are many tabular data packages in Julia and they all use the interface defined by Tables.jl, and all interop very well. Query.jl (which is something like Linq or TidyR) works with all of them, and do packages for loadingand saving (CSV.jl, LibPQ.jl etc)
KenoFischer 7y ago
To answer the usual complaint whenever the release announcement gets linked here, the NEWS file is intended to give current users of Julia an overview of all the things that changed that they may want to adjust to. It is however, not designed to give people who are only casually following the project an overview of all the work that's going on and why it's happening. We've been talking about writing a more casual document like that for the releases, since they do tend to get a fair bit of attention, but that's just one more thing on an already-extensive list of things that need to be done for each release.
- Someone 7y ago
  ”It is however, not designed to give people who are only casually following the project an overview of all the work that's going on and why it's happening. The NEWS file may not be designed for that, but IMO, it still is way better than this text, which, for casual followers, doesn’t say much more than “1.4 replaces 1.3, no breaking changes, a few new features”, without even mentioning those features, or why they were added. Maybe, combining the two in a single document, and copy-pasting NEWS.md in the announcement, would decrease the amount of work and improve things?
  - disgruntledphd2 7y ago
    I dunno, the comments were pretty informative. Looks like Julia still haven't managed to handle updates well. I remember being so amazed that they'd blown their 1.0 announcement (went from 0.7 to 1.0 over a Juliacon). And because there were so many changes (most of which i thought were good), it was pretty broken for new users at that time, which I firmly believe limited their adoption. And to be clear, I love the idea of Julia, and that first document made me fall in love. And I think that the design for statistical computing is really, really good. It's just a shame that this ops/packaging stuff is holding them back.
    - dr_coffee 7y ago
      I really enjoy using Julia too but I have also been disappointed with how the updates have played out. Just a few weeks ago I rolled back my installation from v1.2 to the LTS version because of silly errors in the plotting package not allowing me to plot. Otherwise it really is a pleasure to use, and I have found that the LTS install has zero compatibility issues thus far.
      - ViralBShah 7y ago
        Glad to hear that the LTS version did not have an issue. I hope you did file an issue with the Plots package. The reason is that we generally like to make sure that regressions like these are not language level regressions. Sometimes package use undocumented internals - but we generally chase each and every single one of these. For those who haven't seen it yet, we have described our release process in great detail in this blog post: https://julialang.org/blog/2019/08/release-process/ https://julialang.org/blog/2019/08/release-process/
    - ViralBShah 7y ago
      The 0.7 to 1.0 was a planned move that was communicated for over a year. If anything, the 1.0 transition went very smoothly and the Julia community grew significantly soon after its release. All the download and community stats broadly demonstrate this. IMO, 1.0 was rough not for new users but people who had invested a bunch of time in Julia codes pre 1.0. We anticipated this and therefore had a very carefully planned release strategy to ease the transition with depreciation warnings and preparing 0.7 as a migration aiding release for 0.6 users to 1.0. In fact our release announcement discussed all of this at great length. https://julialang.org/blog/2018/08/one-point-zero/ https://julialang.org/blog/2018/08/one-point-zero/
- dilap 7y ago
  It's natural that you'll get publicity and curious onlookers when doing new releases -- might be worth considering that natural audience, too, when writing the NEWS file.
  - KenoFischer 7y ago
    Right, we're of course aware of that audience, which is why I posted that comment. The way the NEWS file gets written is just that each individual commit that makes a change that needs end user attention will add an entry in the relevant section. This is quite hand as people can just 'git blame' the file and automatically find a link to the relevant commit (or they can click through the linked issue). What's missing is a more high level narrative structure of changes in the release, but such a thing would have to be written once the release is done. Usually we do that in the form of a blog post announcing the release. However, invariably, these things end up on HN the second somebody thinks the release is done (or even just for early RCs).
pella 7y ago
Julia v1.4 Release Notes : https://github.com/JuliaLang/julia/blob/v1.4.0/NEWS.md https://github.com/JuliaLang/julia/blob/v1.4.0/NEWS.md
- dang 7y ago
  That seems more informative so we switched the URL from https://discourse.julialang.org/t/julia-v1-4-0-has-been-released/36324 https://discourse.julialang.org/t/julia-v1-4-0-has-been-rele... above.
sk0g 7y ago
I've been meaning to check Julia out for a while now. Are there any deep learning libraries that are as feature complete and user friendly as PyTorch/ Tensorflow?
- marmaduke 7y ago
  Flux.jl seems to be the flagship project.
- komuher 7y ago
  Flux but its not even close to PyTorch or TF in terms of features and performance
  - logicchains 7y ago
    It depends what you're doing. If you need to write your own kernels, or have small networks where the framework overhead is significant, then it's way faster than Tensorflow (unless you implement your own Tensorflow OP in C++/Cuda, but that's way more painful than just implementing it directly in Flux/Julia). It's hence quite nice for research on new architectures. Flux's autodiff also handles more language features than TF or PyTorch's.
    - mbeex 7y ago
      > unless you implement your own Tensorflow OP in C++/Cuda I don't know exactly what 'OP' means, but there are other ways to do ML in for example C++. I have some good experience with dlib, PyTorch's libtorch is on my todo list.
      - logicchains 7y ago
        >I don't know exactly what 'OP' means, but there are other ways to do ML in for example C++. I think Flux really excels when you're trying to do ML as "differentiable programming" (e.g. model-based reinforcement learning), because it can differentiate so much control flow via Zygote, which hooks into the compiler and differentiates the AST: https://github.com/FluxML/Zygote.jl https://github.com/FluxML/Zygote.jl
signaru 7y ago
I wish it had more dedicated IDE besides the atom/electron based ones (like GNU Octave has its own). I think Julia's performance deserves an equally performant IDE.
- newswasboring 7y ago
  This. I have been trying to learn Julia so I can promote it to replace Matlab in my company. I think everyone will like the performance, and a more modern and extensive library (also the fact that it's free will save the company a lot of money). But the atom Juno IDE looks so unprofessional, I think we will have a really hard time convincing scientists to use something which can't even undock the editor properly (I know we can open the file in a new window but it's really not the same).
  - mbeex 7y ago
    Seems to be a thing of electron, VS code cannot do this either (and the proper handling of multiple monitors in general).
    - newswasboring 7y ago
      Yeah, that is my guess too. But at least VS code can save your workspace in atom you have to install a package to do that. To programmers it might look like flexibility but to non programmers it just looks like a chore.
- logicchains 7y ago
  I've had quite a pleasant experience with just Jupyter Notebooks & Emacs for Julia development.
- ddragon 7y ago
  I don't really have issues with the performance of Visual Studio Code even though it's electron, but with Julia I mostly use it for the syntax highlighting, linting and autocompletion (plus version control and other external tools). I find it much easier to have a Julia instance running on a drop down terminal or in the second monitor for easy access, Revise so every file I save is automatically loaded in the REPL session and some extras like Rebugger, Infiltrator and OhMyREPL for debugging and some niceties. This way I can just immediately query and test things using the language itself without having to deal with any sort of constraints or quirks within the editors. And the Julia REPL allows for quickly switching to shell (just pressing ;) when needed. I would hope they keep improving the REPL (and the editor support features like linters) over focusing on a completely new IDE (there is even more potential when considering Lisp languages REPL, plus general usability improvements like first call latency), but I'm biased and that's not the workflow most developers use.
komuher 7y ago
Something about Julia from me. I was big Julia fan using it for last 1.5 year and in my company we are using Julia for data preprocessing and parsing hundred of GB's of data. (We also use Python for ML and start to moving julia code to Nim for data prepro) If u consider Julia as ur next language dont use it to any medium to big project outside pure scientific purposes it just isnt ready yet. We have about 3.5k loc in Julia at the moment for our preprocessing service and as much problems as we have with GC in Julia i never saw in any other language: random crashesh, memory leaks, big slowdowns with long running processes and there is a lot of topic on julia forum about that still not solved. Julia is beautiful and easy to use but it is nowhere near stable and/or fast as any other "new" languages (RUST, Nim, Golang, even Swift). And if u consider using Julia for Deep Learning then best performance and stability in this order are: Pytorch, Tensorflow/MXNet/(JAX), Nothing, Flux. Its a long way to go for Flux before it'll be even close in usability considering other flag frameworks.
- oxinabox 7y ago
  > If u consider Julia as ur next language dont use it to any medium to big project outside pure scientific purposes it just isnt ready yet My employer, Invenia, uses Julia for what I would say is at least a medium sized project. ~200k LOC, ~30 people concurrently working on that code base, literally the core product/system that makes us money. Julia's not perfect, but I would not blanket discount it out of hand for medium projects. Like all technologies the tradeoffs need to be investigated with reference to the task (and team) at hand.
- tomerbd 7y ago
  I have to admit that over the past 20 years, java is like the most stable robust, huge project, multiple teams, IDE friendly, type safe, infinite tooling and libraries language of choice!
  - komuher 7y ago
    Java is good for big projects but not for Machine Learning purposes (Also im C# fanboy so still will prefer C# over Java :D)
  - 7y ago
greendave 7y ago
The language has a lot of really nice stuff and the introduction of threads and support for them in the standard libraries (beginning in 1.3) has helped us a lot. That said, the tooling is still frustrating. Generating compiled binaries is a slow, painful process.
metreo 7y ago
I love Julia and the hard work of the team behind this is very exciting!
eigenspace 7y ago
Surprised by some of the negativity here! I've been extensively using julia for my graduate physics research and a lot of hobby programming for almost 3 years now and absolutely love it. It's a beautifully designed language with incredibly responsive and wise developers and 1.4.0 is a great release I've been on 1.4 release candidates for over a month and haven't had a single issue, and love the new features and improvements.
- systemvoltage 7y ago
  The negativity is warranted if you've ever run Julia or had to maintain it in production. Use it for your own Jupyter notebooks and personal analysis? Great! Need to debug some weird obscure error (which Julia does a poor job of reporting to the user, let alone which line in the stack trace) while having production pressure to get it up and running again? Julia is unquestionably, unarguably and utterly unsuitable. Do not use it in any production workflow, heed my advice as a maintainer of Julia repos and the tech debt that graduate students have created in our company (I am sorry but just stating the facts). You're still unsure? Let's switch sides, you maintain our company's production Julia repositories and I will take your spot to write beautiful code in a couple of hobby projects. I guarantee you, with absolute certainty, you will come down to your knees. You'll lose sleep. You'll hate management for letting this happen.
  - burntsushi 7y ago
    Do you have any links to other material where I could read up on some of the problems folks have faced using Julia in production? Thanks!
    - systemvoltage 7y ago
      I'd love to do a write up on the issues to guide the Julia team towards focusing on production and robustness, investing most of their efforts in a solid debugger support (Juno - totally incapable tool), and stopping all new features and "optimization" projects before getting these things sorted out. I am just too beaten, too bruised and injured to have the mental capacity to write an article. :( I am sorry if I am too harsh.
xor0110 7y ago
Julia is an amazing language and anyone who loves computers and/or science should be trying it right now.
dklend122 7y ago
Julia is an amazing, elegant and beautiful language. It's almost perfectly suited for scientific computing and ML. However, I'm not very bullish on its future given s4tf Swift can get you 90% of the way there, and that extra 10% can be more than made up the by the efforts of apple, google and other companies (including money, network/clout, kaggle which is owned by google etc). Despite predictions to the contrary, Chris Lattner's departure doesn't seem to have slowed down the project, and more team members from google have been added since. Swift is rapidly approaching usability on windows with investment from google. Further, at some point google will facilitate Swift's use for android apps, and then Swift's popularity will skyrocket, and all those developers will be naturally inclined to check out the ML stuff. Even facebook is getting in on the party: https://twitter.com/nadavrot/status/1241150682104606720 https://twitter.com/nadavrot/status/1241150682104606720 In addition, Swift has its own benefits over julia for production and large codebases, such as compilation to small binaries and static typing. Julia doesn't have a good story for either of these (yet?), and chasing down type instabilities in larger code isn't fun.
- ddragon 7y ago
  You mean the future of Julia or the future of Flux? While an amazing accomplishment (that is still on the way of becoming truly mature, just like s4tf), Flux is just one of Julia's current ML libraries, and it definitely doesn't feel like a Rails (or maybe Flutter) situation in which the library is larger than the language. ML isn't even Julia's core target (it just happens to fits extremely well with numerical and scientific processing). Julia will be just fine even if s4tf somehow steals all the mindshare (especially since a mature differentiable programming library will inevitably serve as inspiration for Flux itself) as the language and target audience is not very similar to Swift's and as such Flux and s4tf will also find different niches (for example one can be more used on high performance scientific research thanks to Julia's ecosystem and focus while the other can focus on mobile deployment of ML models).
  - dklend122 7y ago
    If my scenario holds, at some point Swift's scientific computing ecosystem will rival and overtake Julia's. I don't see the ML ecosystem developing in isolation because there's going to be overlap, especially as more and more code can be differentiated.

243 points by pella 7y ago

What is the state of Julia dataframe? Is it ready for production use? How is the Julia dataframe performance in comparison with R and Python dataframes?

oxinabox 7y ago
My employer uses DataFrames.jl in production. It's fine, nothing wrong with it. Used to be a bit unsafe to unwarey uses, now it's safe by default and you need to do a bit extra to get all the performance. It's worth knowing it is more like R's DataFrames than like Pandas. It is getting pretty close to a 1.0 release. Probably a few months out (One more minor, then if all goes well 1.0 a month or so later) Further one should know that there are many tabular data packages in Julia and they all use the interface defined by Tables.jl, and all interop very well. Query.jl (which is something like Linq or TidyR) works with all of them, and do packages for loadingand saving (CSV.jl, LibPQ.jl etc)

KenoFischer 7y ago

To answer the usual complaint whenever the release announcement gets linked here, the NEWS file is intended to give current users of Julia an overview of all the things that changed that they may want to adjust to. It is however, not designed to give people who are only casually following the project an overview of all the work that's going on and why it's happening. We've been talking about writing a more casual document like that for the releases, since they do tend to get a fair bit of attention, but that's just one more thing on an already-extensive list of things that need to be done for each release.

Someone 7y ago
”It is however, not designed to give people who are only casually following the project an overview of all the work that's going on and why it's happening. The NEWS file may not be designed for that, but IMO, it still is way better than this text, which, for casual followers, doesn’t say much more than “1.4 replaces 1.3, no breaking changes, a few new features”, without even mentioning those features, or why they were added. Maybe, combining the two in a single document, and copy-pasting NEWS.md in the announcement, would decrease the amount of work and improve things?
- disgruntledphd2 7y ago
  I dunno, the comments were pretty informative. Looks like Julia still haven't managed to handle updates well. I remember being so amazed that they'd blown their 1.0 announcement (went from 0.7 to 1.0 over a Juliacon). And because there were so many changes (most of which i thought were good), it was pretty broken for new users at that time, which I firmly believe limited their adoption. And to be clear, I love the idea of Julia, and that first document made me fall in love. And I think that the design for statistical computing is really, really good. It's just a shame that this ops/packaging stuff is holding them back.
  - dr_coffee 7y ago
    I really enjoy using Julia too but I have also been disappointed with how the updates have played out. Just a few weeks ago I rolled back my installation from v1.2 to the LTS version because of silly errors in the plotting package not allowing me to plot. Otherwise it really is a pleasure to use, and I have found that the LTS install has zero compatibility issues thus far.
    - ViralBShah 7y ago
      Glad to hear that the LTS version did not have an issue. I hope you did file an issue with the Plots package. The reason is that we generally like to make sure that regressions like these are not language level regressions. Sometimes package use undocumented internals - but we generally chase each and every single one of these. For those who haven't seen it yet, we have described our release process in great detail in this blog post: https://julialang.org/blog/2019/08/release-process/ https://julialang.org/blog/2019/08/release-process/
  - ViralBShah 7y ago
    The 0.7 to 1.0 was a planned move that was communicated for over a year. If anything, the 1.0 transition went very smoothly and the Julia community grew significantly soon after its release. All the download and community stats broadly demonstrate this. IMO, 1.0 was rough not for new users but people who had invested a bunch of time in Julia codes pre 1.0. We anticipated this and therefore had a very carefully planned release strategy to ease the transition with depreciation warnings and preparing 0.7 as a migration aiding release for 0.6 users to 1.0. In fact our release announcement discussed all of this at great length. https://julialang.org/blog/2018/08/one-point-zero/ https://julialang.org/blog/2018/08/one-point-zero/
dilap 7y ago
It's natural that you'll get publicity and curious onlookers when doing new releases -- might be worth considering that natural audience, too, when writing the NEWS file.
- KenoFischer 7y ago
  Right, we're of course aware of that audience, which is why I posted that comment. The way the NEWS file gets written is just that each individual commit that makes a change that needs end user attention will add an entry in the relevant section. This is quite hand as people can just 'git blame' the file and automatically find a link to the relevant commit (or they can click through the linked issue). What's missing is a more high level narrative structure of changes in the release, but such a thing would have to be written once the release is done. Usually we do that in the form of a blog post announcing the release. However, invariably, these things end up on HN the second somebody thinks the release is done (or even just for early RCs).

pella 7y ago

Julia v1.4 Release Notes : https://github.com/JuliaLang/julia/blob/v1.4.0/NEWS.md https://github.com/JuliaLang/julia/blob/v1.4.0/NEWS.md

dang 7y ago
That seems more informative so we switched the URL from https://discourse.julialang.org/t/julia-v1-4-0-has-been-released/36324 https://discourse.julialang.org/t/julia-v1-4-0-has-been-rele... above.

sk0g 7y ago

I've been meaning to check Julia out for a while now. Are there any deep learning libraries that are as feature complete and user friendly as PyTorch/ Tensorflow?

marmaduke 7y ago
Flux.jl seems to be the flagship project.
komuher 7y ago
Flux but its not even close to PyTorch or TF in terms of features and performance
- logicchains 7y ago
  It depends what you're doing. If you need to write your own kernels, or have small networks where the framework overhead is significant, then it's way faster than Tensorflow (unless you implement your own Tensorflow OP in C++/Cuda, but that's way more painful than just implementing it directly in Flux/Julia). It's hence quite nice for research on new architectures. Flux's autodiff also handles more language features than TF or PyTorch's.
  - mbeex 7y ago
    > unless you implement your own Tensorflow OP in C++/Cuda I don't know exactly what 'OP' means, but there are other ways to do ML in for example C++. I have some good experience with dlib, PyTorch's libtorch is on my todo list.
    - logicchains 7y ago
      >I don't know exactly what 'OP' means, but there are other ways to do ML in for example C++. I think Flux really excels when you're trying to do ML as "differentiable programming" (e.g. model-based reinforcement learning), because it can differentiate so much control flow via Zygote, which hooks into the compiler and differentiates the AST: https://github.com/FluxML/Zygote.jl https://github.com/FluxML/Zygote.jl

signaru 7y ago

I wish it had more dedicated IDE besides the atom/electron based ones (like GNU Octave has its own). I think Julia's performance deserves an equally performant IDE.

newswasboring 7y ago
This. I have been trying to learn Julia so I can promote it to replace Matlab in my company. I think everyone will like the performance, and a more modern and extensive library (also the fact that it's free will save the company a lot of money). But the atom Juno IDE looks so unprofessional, I think we will have a really hard time convincing scientists to use something which can't even undock the editor properly (I know we can open the file in a new window but it's really not the same).
- mbeex 7y ago
  Seems to be a thing of electron, VS code cannot do this either (and the proper handling of multiple monitors in general).
  - newswasboring 7y ago
    Yeah, that is my guess too. But at least VS code can save your workspace in atom you have to install a package to do that. To programmers it might look like flexibility but to non programmers it just looks like a chore.
logicchains 7y ago
I've had quite a pleasant experience with just Jupyter Notebooks & Emacs for Julia development.
ddragon 7y ago
I don't really have issues with the performance of Visual Studio Code even though it's electron, but with Julia I mostly use it for the syntax highlighting, linting and autocompletion (plus version control and other external tools). I find it much easier to have a Julia instance running on a drop down terminal or in the second monitor for easy access, Revise so every file I save is automatically loaded in the REPL session and some extras like Rebugger, Infiltrator and OhMyREPL for debugging and some niceties. This way I can just immediately query and test things using the language itself without having to deal with any sort of constraints or quirks within the editors. And the Julia REPL allows for quickly switching to shell (just pressing ;) when needed. I would hope they keep improving the REPL (and the editor support features like linters) over focusing on a completely new IDE (there is even more potential when considering Lisp languages REPL, plus general usability improvements like first call latency), but I'm biased and that's not the workflow most developers use.

komuher 7y ago

Something about Julia from me. I was big Julia fan using it for last 1.5 year and in my company we are using Julia for data preprocessing and parsing hundred of GB's of data. (We also use Python for ML and start to moving julia code to Nim for data prepro) If u consider Julia as ur next language dont use it to any medium to big project outside pure scientific purposes it just isnt ready yet. We have about 3.5k loc in Julia at the moment for our preprocessing service and as much problems as we have with GC in Julia i never saw in any other language: random crashesh, memory leaks, big slowdowns with long running processes and there is a lot of topic on julia forum about that still not solved. Julia is beautiful and easy to use but it is nowhere near stable and/or fast as any other "new" languages (RUST, Nim, Golang, even Swift). And if u consider using Julia for Deep Learning then best performance and stability in this order are: Pytorch, Tensorflow/MXNet/(JAX), Nothing, Flux. Its a long way to go for Flux before it'll be even close in usability considering other flag frameworks.

oxinabox 7y ago
> If u consider Julia as ur next language dont use it to any medium to big project outside pure scientific purposes it just isnt ready yet My employer, Invenia, uses Julia for what I would say is at least a medium sized project. ~200k LOC, ~30 people concurrently working on that code base, literally the core product/system that makes us money. Julia's not perfect, but I would not blanket discount it out of hand for medium projects. Like all technologies the tradeoffs need to be investigated with reference to the task (and team) at hand.
tomerbd 7y ago
I have to admit that over the past 20 years, java is like the most stable robust, huge project, multiple teams, IDE friendly, type safe, infinite tooling and libraries language of choice!
- komuher 7y ago
  Java is good for big projects but not for Machine Learning purposes (Also im C# fanboy so still will prefer C# over Java :D)
- 7y ago

greendave 7y ago

The language has a lot of really nice stuff and the introduction of threads and support for them in the standard libraries (beginning in 1.3) has helped us a lot. That said, the tooling is still frustrating. Generating compiled binaries is a slow, painful process.

metreo 7y ago

I love Julia and the hard work of the team behind this is very exciting!

eigenspace 7y ago

Surprised by some of the negativity here! I've been extensively using julia for my graduate physics research and a lot of hobby programming for almost 3 years now and absolutely love it. It's a beautifully designed language with incredibly responsive and wise developers and 1.4.0 is a great release I've been on 1.4 release candidates for over a month and haven't had a single issue, and love the new features and improvements.

systemvoltage 7y ago
The negativity is warranted if you've ever run Julia or had to maintain it in production. Use it for your own Jupyter notebooks and personal analysis? Great! Need to debug some weird obscure error (which Julia does a poor job of reporting to the user, let alone which line in the stack trace) while having production pressure to get it up and running again? Julia is unquestionably, unarguably and utterly unsuitable. Do not use it in any production workflow, heed my advice as a maintainer of Julia repos and the tech debt that graduate students have created in our company (I am sorry but just stating the facts). You're still unsure? Let's switch sides, you maintain our company's production Julia repositories and I will take your spot to write beautiful code in a couple of hobby projects. I guarantee you, with absolute certainty, you will come down to your knees. You'll lose sleep. You'll hate management for letting this happen.
- burntsushi 7y ago
  Do you have any links to other material where I could read up on some of the problems folks have faced using Julia in production? Thanks!
  - systemvoltage 7y ago
    I'd love to do a write up on the issues to guide the Julia team towards focusing on production and robustness, investing most of their efforts in a solid debugger support (Juno - totally incapable tool), and stopping all new features and "optimization" projects before getting these things sorted out. I am just too beaten, too bruised and injured to have the mental capacity to write an article. :( I am sorry if I am too harsh.

xor0110 7y ago

Julia is an amazing language and anyone who loves computers and/or science should be trying it right now.

dklend122 7y ago

Julia is an amazing, elegant and beautiful language. It's almost perfectly suited for scientific computing and ML. However, I'm not very bullish on its future given s4tf Swift can get you 90% of the way there, and that extra 10% can be more than made up the by the efforts of apple, google and other companies (including money, network/clout, kaggle which is owned by google etc). Despite predictions to the contrary, Chris Lattner's departure doesn't seem to have slowed down the project, and more team members from google have been added since. Swift is rapidly approaching usability on windows with investment from google. Further, at some point google will facilitate Swift's use for android apps, and then Swift's popularity will skyrocket, and all those developers will be naturally inclined to check out the ML stuff. Even facebook is getting in on the party: https://twitter.com/nadavrot/status/1241150682104606720 https://twitter.com/nadavrot/status/1241150682104606720 In addition, Swift has its own benefits over julia for production and large codebases, such as compilation to small binaries and static typing. Julia doesn't have a good story for either of these (yet?), and chasing down type instabilities in larger code isn't fun.

ddragon 7y ago
You mean the future of Julia or the future of Flux? While an amazing accomplishment (that is still on the way of becoming truly mature, just like s4tf), Flux is just one of Julia's current ML libraries, and it definitely doesn't feel like a Rails (or maybe Flutter) situation in which the library is larger than the language. ML isn't even Julia's core target (it just happens to fits extremely well with numerical and scientific processing). Julia will be just fine even if s4tf somehow steals all the mindshare (especially since a mature differentiable programming library will inevitably serve as inspiration for Flux itself) as the language and target audience is not very similar to Swift's and as such Flux and s4tf will also find different niches (for example one can be more used on high performance scientific research thanks to Julia's ecosystem and focus while the other can focus on mobile deployment of ML models).
- dklend122 7y ago
  If my scenario holds, at some point Swift's scientific computing ecosystem will rival and overtake Julia's. I don't see the ML ecosystem developing in isolation because there's going to be overlap, especially as more and more code can be differentiated.