5 ms·

This seems so silly to me. It’s PyTorch-if they said “the next version of PyTorch will be in Julia, the ecosystem would shift accordingly. They’re practically

This seems so silly to me. It’s PyTorch-if they said “the next version of PyTorch will be in Julia, the ecosystem would shift accordingly. They’re practically saying “this language has every feature we need and want, most of them already existing, but we’re going to continue re-inventing them in this objectively less suitable language because we clearly wish to make life harder for ourselves”

driscoll42 5y ago
Or I read it as "We want to make life as easy for our userbase as possible, so we will put more work on ourselves to make our users lives easier" which is an attitude I very much appreciate.
- fault1 5y ago
  agreed, and this always been the driving philosophy of pytorch, and perhaps why it kind of won so much brainshare against tensorflow despite _long_ odds when torch was ported from lua. Soumith Chintala had a keynote talk in juliacon where he focused on these points; https://www.youtube.com/watch?v=6V6jk_OdH-w https://www.youtube.com/watch?v=6V6jk_OdH-w
  - dnautics 5y ago
    were those odds really long? TensorFlow is a beastly mess; and there were some very serious breaking issues between minor version revisions, it was so bad that I used to work at a company where we joked "this company only exists because people can't install tensorflow". We also used to joke that in order to install TensorFlow, step 1: install the JVM.
- rg111 5y ago
  Exactly. PyTorch is not only easy, but is a joy to work with. Among researchers, TensorFlow is rapidly losing ground to PyTorch, and, I think, will keep losing ground until it becomes a niche and only used by Googlers and some others. https://horace.io/pytorch-vs-tensorflow/ https://horace.io/pytorch-vs-tensorflow/
- nextos 5y ago
  In the long run I think moving to Julia would make a lot of sense. I have used MATLAB, R, Python and Julia extensively for doing all sorts of data related things during the last 20 years. Julia is incredibly easy to work with, very elegant and really efficient. R and Python have always felt clumsy in some ways, and hard to write really performant code, even if you are more proficient in Python! As a seasoned Lisper and MLer, even after having a lot of Python experience in my belt, Julia felt much easier to work with from the very beginning. Furthermore, most Julia libraries are written in pure Julia which simplifies things a lot and enhances composability. While there are great libraries around, the DL landscape is a bit lacking. Flux is great, but I would not use it to build e.g. transformers as it changes too often and has too few maintainers behind it. Hence a potential migration of Torch to Julia would be fantastic.
  - dg96 5y ago
    Flux dev here. There's Transformers.jl which has prebuilt transformers built on top of Flux. While the package does change, we have been more careful about ensuring we don't break code all that often.
  - Daishiman 5y ago
    But Julia doesn't have a good story for anything else besides that. You can take a Python web server process, have a request call a task that uses NumPy and OpenCV and scikit-learn, get that back, and you're done, all in the same language. Julia's community does not seem to have aspirations beyond high-performance math code, which is great for its use, but I'm not going to learn Julia just for that when I can implement the entirety of a development pipeline in Python and have all the other niceties that come with it.
    - amkkma 5y ago
      That's just not true. https://github.com/GenieFramework/Genie.jl https://github.com/GenieFramework/Genie.jl https://github.com/JuliaWeb/HTTP.jl https://github.com/JuliaWeb/HTTP.jl https://www.youtube.com/watch?v=xsxJt4prFG4 https://www.youtube.com/watch?v=xsxJt4prFG4 And with upcoming improvements to binary size and structured concurrency (it already does go-like lightweight threads) it will get even better.
- tfehring 5y ago
  Does keeping as much of the codebase as possible in Python (or keeping the fast parts in C++) actually make things easier for the userbase, or do they just care about having a first-class interface in Python regardless of the implementation language?
  - pm90 5y ago
    Almost certainly the latter. Python excels in this because it’s really easy to learn, so non python libraries provide python functions that you can call which may not be implemented in pure python.
Mehdi2277 5y ago
Language choices are less about language to me and more about ecosystem of libraries. Python has generally been very strong in the ml/data science realm. I know Julia is catching up but am unsure just how much it covers. Example of python libraries I would consider needed as part of the data ecosystem, numpy, pytorch/tf, batch and stream processing frameworks like spark/flink/beam, workflow orchestration like kubeflow/airflow, data formats like pyarrow, etc. My company does most of our ml in google cloud so gcp libraries are also quite helpful. How much of that does Julia have equivalents to? How much coverage do those equivalents have? We’ve done some ml things outside python before and one general issue for most languages is leaving python there’s a high risk that something will be missing. Because of that if we use another language it’s preferred to keep scope of it’s project small. I think only big exception is on the data engineering side sometimes Java is better as a lot of batch/stream frameworks have best coverage in Java, although ml libraries is weaker there so our usage of Java is mainly data pipelines. Another issue is pytorch/tf in python are very dominant in research/projects. Often we clone relevant recent projects and try experimenting with them to see if they help. Swapping to Julia would hurt a ton in that area. edit: Also while I'm fond of python I'd be very open to seeing another language win. There are language design choices I dislike in python, but I like enough of the language and ecosystem as been too strong to leave most other languages worth pondering. If Julia grows enough that my coworkers start asking for Julia support I'd be happy to explore it. My pet preferred language is crystal (ruby like readability + types + good performance) but ecosystem wise it's tiny.
- queuebert 5y ago
  Python is the middle manager of languages. It sucks at everything, but always knows a guy.
  - dgfitz 5y ago
    I saw a quote once that stuck with me: python is the second best language for everything, first at nothing.
    - d0mine 5y ago
_tom_ 5y ago
By "the ecosystem" they mean the python ecosystem, not the PyTorch ecosystem. PyTorch is a small part of the python ecosystem. The python ecosystem is not going to change at all if PyTorch moves to Julia.

driscoll42 5y ago

Or I read it as "We want to make life as easy for our userbase as possible, so we will put more work on ourselves to make our users lives easier" which is an attitude I very much appreciate.

fault1 5y ago
agreed, and this always been the driving philosophy of pytorch, and perhaps why it kind of won so much brainshare against tensorflow despite _long_ odds when torch was ported from lua. Soumith Chintala had a keynote talk in juliacon where he focused on these points; https://www.youtube.com/watch?v=6V6jk_OdH-w https://www.youtube.com/watch?v=6V6jk_OdH-w
- dnautics 5y ago
  were those odds really long? TensorFlow is a beastly mess; and there were some very serious breaking issues between minor version revisions, it was so bad that I used to work at a company where we joked "this company only exists because people can't install tensorflow". We also used to joke that in order to install TensorFlow, step 1: install the JVM.
rg111 5y ago
Exactly. PyTorch is not only easy, but is a joy to work with. Among researchers, TensorFlow is rapidly losing ground to PyTorch, and, I think, will keep losing ground until it becomes a niche and only used by Googlers and some others. https://horace.io/pytorch-vs-tensorflow/ https://horace.io/pytorch-vs-tensorflow/
nextos 5y ago
In the long run I think moving to Julia would make a lot of sense. I have used MATLAB, R, Python and Julia extensively for doing all sorts of data related things during the last 20 years. Julia is incredibly easy to work with, very elegant and really efficient. R and Python have always felt clumsy in some ways, and hard to write really performant code, even if you are more proficient in Python! As a seasoned Lisper and MLer, even after having a lot of Python experience in my belt, Julia felt much easier to work with from the very beginning. Furthermore, most Julia libraries are written in pure Julia which simplifies things a lot and enhances composability. While there are great libraries around, the DL landscape is a bit lacking. Flux is great, but I would not use it to build e.g. transformers as it changes too often and has too few maintainers behind it. Hence a potential migration of Torch to Julia would be fantastic.
- dg96 5y ago
  Flux dev here. There's Transformers.jl which has prebuilt transformers built on top of Flux. While the package does change, we have been more careful about ensuring we don't break code all that often.
- Daishiman 5y ago
  But Julia doesn't have a good story for anything else besides that. You can take a Python web server process, have a request call a task that uses NumPy and OpenCV and scikit-learn, get that back, and you're done, all in the same language. Julia's community does not seem to have aspirations beyond high-performance math code, which is great for its use, but I'm not going to learn Julia just for that when I can implement the entirety of a development pipeline in Python and have all the other niceties that come with it.
  - amkkma 5y ago
    That's just not true. https://github.com/GenieFramework/Genie.jl https://github.com/GenieFramework/Genie.jl https://github.com/JuliaWeb/HTTP.jl https://github.com/JuliaWeb/HTTP.jl https://www.youtube.com/watch?v=xsxJt4prFG4 https://www.youtube.com/watch?v=xsxJt4prFG4 And with upcoming improvements to binary size and structured concurrency (it already does go-like lightweight threads) it will get even better.
tfehring 5y ago
Does keeping as much of the codebase as possible in Python (or keeping the fast parts in C++) actually make things easier for the userbase, or do they just care about having a first-class interface in Python regardless of the implementation language?
- pm90 5y ago
  Almost certainly the latter. Python excels in this because it’s really easy to learn, so non python libraries provide python functions that you can call which may not be implemented in pure python.

Mehdi2277 5y ago

Language choices are less about language to me and more about ecosystem of libraries. Python has generally been very strong in the ml/data science realm. I know Julia is catching up but am unsure just how much it covers. Example of python libraries I would consider needed as part of the data ecosystem, numpy, pytorch/tf, batch and stream processing frameworks like spark/flink/beam, workflow orchestration like kubeflow/airflow, data formats like pyarrow, etc. My company does most of our ml in google cloud so gcp libraries are also quite helpful. How much of that does Julia have equivalents to? How much coverage do those equivalents have? We’ve done some ml things outside python before and one general issue for most languages is leaving python there’s a high risk that something will be missing. Because of that if we use another language it’s preferred to keep scope of it’s project small. I think only big exception is on the data engineering side sometimes Java is better as a lot of batch/stream frameworks have best coverage in Java, although ml libraries is weaker there so our usage of Java is mainly data pipelines. Another issue is pytorch/tf in python are very dominant in research/projects. Often we clone relevant recent projects and try experimenting with them to see if they help. Swapping to Julia would hurt a ton in that area. edit: Also while I'm fond of python I'd be very open to seeing another language win. There are language design choices I dislike in python, but I like enough of the language and ecosystem as been too strong to leave most other languages worth pondering. If Julia grows enough that my coworkers start asking for Julia support I'd be happy to explore it. My pet preferred language is crystal (ruby like readability + types + good performance) but ecosystem wise it's tiny.

queuebert 5y ago
Python is the middle manager of languages. It sucks at everything, but always knows a guy.
- dgfitz 5y ago
  I saw a quote once that stuck with me: python is the second best language for everything, first at nothing.
  - d0mine 5y ago

_tom_ 5y ago

By "the ecosystem" they mean the python ecosystem, not the PyTorch ecosystem. PyTorch is a small part of the python ecosystem. The python ecosystem is not going to change at all if PyTorch moves to Julia.