6 ms·

I feel like this misses the point. Python is used as the language to describe things, but nobody is executing Python on the GPU. Instead, things are transpiled

I feel like this misses the point. Python is used as the language to describe things, but nobody is executing Python on the GPU. Instead, things are transpiled into highly optimized byte code, C++ source, or chains of optimized GPU sharers. See TensorFlow XLA and AOT. So when I set up a training pipeline with TensorFlow, then it'll barely run any python while training. Instead, the TF C++ core will execute a network of TF CUDA operators for me, all inside one python function call. That's why pythons global interpreter lock is pretty much a non-issue by now. Since only 1% of my training time is even running inside python, I don't expect any performance benefits from replacing it. And if both are equally fast, I prefer the familiar option, meaning python.

dgellow 6y ago
You can be sure that they are both well aware of this, Jeremy is one of the creator of the popular python library, book, and online course fast.ai built on top of PyTorch. The relevant part of the their discussion is this one: > Like Python is not the future of machine learning. It can't be. You know, it's so nicely hackable, but it's so frustrating to work with a language where you can't do anything fast enough unless you call out to some external code or C code, and you can't run anything in parallel unless you put in a whole other process. Like I find working with Python, there's just so much overhead in my brain to try to get it to work fast enough. It's obviously fine for a lot of things, but not really in the deep learning world or not really in the machine learning world. So, like, I really hope that Julia is really successful because there's a language with a nicely designed type system and a nicely designed dispatch system and most importantly, it's Julia all the way down so you can get in and write your GPU colonel in Julia, or all the basic stuff is implemented in Júlia all the way down until you hit the LLVM. Here the point is that when using other languages (such as Julia) you actually can use the same language for your GPU and for the glue that keep your scripts together.
- p1esk 6y ago
  I feel like you (and them) are still missing the point. 99% of users don't write GPU kernels. They use simple function calls and that's all they see. 99% of users don't care if they call CUDA code from Python, or Julia code from Julia, as long as both are equally fast and API is clean (e.g. Pytorch).
  - ed_elliott_asc 6y ago
    I watched an interview with him, his end goal is no code machine learning so when he is saying python isn’t the future it is because he prefers a type system that can’t be achieved with python. I agree that users don’t care, I use Spark a lot and don’t care if I write scala, python or even c# now. Have a listen: https://www.wandb.com/podcast/jeremy-howard https://www.wandb.com/podcast/jeremy-howard (I think I have paraphrased the podcast correctly :) )
    - p1esk 6y ago
      end goal is no code machine learning Now that I agree on. I suspect we will see something like GPT-5 reliably converting plain English into code sooner than Julia displacing Python as king of ML.
  - layoric 6y ago
    I don’t think they are missing the point. These abstractions are far from perfect and trying to hunt down why something is not performing and tracing it back to how syntax is being interpreted as data is stored (GPU memory vs system memory vs sent over the pcie bus) or passed between other glue code is a source of frustration I’m sure many people have experienced. It will take a lot for another language eco system to catch-up to something as broad and deep as python but these issues combined with the horrible experience that is dependency management with pure python vs compiled etc as well as the need for real-time/close to real-time systems, it seems very plausible that something like Julia or other llvm language will become the default over python for _these specific_ use cases, not in general though. Tooling on top of Julia can likely be far better and more easily made than living in python which is another aspect.
    - p1esk 6y ago
      trying to hunt down why something is not performing Again, 99% of the users won't be doing that no matter how good the tracing abilities are. They will post a question on SO/GitHub. Regardless of the language, modern DL frameworks are just too complex for regular users.
  - 6y ago
nl 6y ago
Jeremy's literal statement is: "Like Python is not the future of machine learning. It can't be. You know, it's so nicely hackable, but it's so frustrating to work with a language where you can't do anything fast enough unless you call out to some external code or C code, and you can't run anything in parallel unless you put in a whole other process." So exactly the point made here.
searchableguy 6y ago
Can someone tell me why ml/dl projects don't include even a requirement.txt or list of dependencies often? I have always wondered why. I see so many projects that miss those. Projects from big companies and teams. Are they intentionally adding hurdles to using the code?
t-vi 6y ago
Indeed. The main questionable implicit assumption that is made is that Python can't "the future" (which is severely underspecified without a time horizon) because of what is bad about it today. It even is literally there "Python isn't the future... It can't be" <because something in Python is bad>. That's highly questionable, because it leaves out - that people appear to have reasons to adopt Python, likely because of something that is good about it, (this complements the observation that not that many people might be impacted by the disadvantages). - that Python and the ML tools in Python might not be static and actually evolve in "the future". It seems to me a curious lack of reflection substituted for by being particularly assertive about it. To my mind Python not being the future is more likely to happen if Python becomes less likable or the attempts to improve the shortcomings (which are there, HPy as a medium-term way to get out of the "C extension vs. parallelism/speedup/GIL" corner, mypyc/numba/TorchScript/JAX pushing the "what can be efficiently expressed in Python" boundary) fail. Julia, Swift, Dex-Lang, HaskTorch all get things right that Python doesn't. But do they get all the things right that Python does get right? Disclaimer: I'm biased because I like Python and have some involvement in Python ML tools.

dgellow 6y ago

You can be sure that they are both well aware of this, Jeremy is one of the creator of the popular python library, book, and online course fast.ai built on top of PyTorch. The relevant part of the their discussion is this one: > Like Python is not the future of machine learning. It can't be. You know, it's so nicely hackable, but it's so frustrating to work with a language where you can't do anything fast enough unless you call out to some external code or C code, and you can't run anything in parallel unless you put in a whole other process. Like I find working with Python, there's just so much overhead in my brain to try to get it to work fast enough. It's obviously fine for a lot of things, but not really in the deep learning world or not really in the machine learning world. So, like, I really hope that Julia is really successful because there's a language with a nicely designed type system and a nicely designed dispatch system and most importantly, it's Julia all the way down so you can get in and write your GPU colonel in Julia, or all the basic stuff is implemented in Júlia all the way down until you hit the LLVM. Here the point is that when using other languages (such as Julia) you actually can use the same language for your GPU and for the glue that keep your scripts together.

p1esk 6y ago
I feel like you (and them) are still missing the point. 99% of users don't write GPU kernels. They use simple function calls and that's all they see. 99% of users don't care if they call CUDA code from Python, or Julia code from Julia, as long as both are equally fast and API is clean (e.g. Pytorch).
- ed_elliott_asc 6y ago
  I watched an interview with him, his end goal is no code machine learning so when he is saying python isn’t the future it is because he prefers a type system that can’t be achieved with python. I agree that users don’t care, I use Spark a lot and don’t care if I write scala, python or even c# now. Have a listen: https://www.wandb.com/podcast/jeremy-howard https://www.wandb.com/podcast/jeremy-howard (I think I have paraphrased the podcast correctly :) )
  - p1esk 6y ago
    end goal is no code machine learning Now that I agree on. I suspect we will see something like GPT-5 reliably converting plain English into code sooner than Julia displacing Python as king of ML.
- layoric 6y ago
  I don’t think they are missing the point. These abstractions are far from perfect and trying to hunt down why something is not performing and tracing it back to how syntax is being interpreted as data is stored (GPU memory vs system memory vs sent over the pcie bus) or passed between other glue code is a source of frustration I’m sure many people have experienced. It will take a lot for another language eco system to catch-up to something as broad and deep as python but these issues combined with the horrible experience that is dependency management with pure python vs compiled etc as well as the need for real-time/close to real-time systems, it seems very plausible that something like Julia or other llvm language will become the default over python for _these specific_ use cases, not in general though. Tooling on top of Julia can likely be far better and more easily made than living in python which is another aspect.
  - p1esk 6y ago
    trying to hunt down why something is not performing Again, 99% of the users won't be doing that no matter how good the tracing abilities are. They will post a question on SO/GitHub. Regardless of the language, modern DL frameworks are just too complex for regular users.
- 6y ago

nl 6y ago

Jeremy's literal statement is: "Like Python is not the future of machine learning. It can't be. You know, it's so nicely hackable, but it's so frustrating to work with a language where you can't do anything fast enough unless you call out to some external code or C code, and you can't run anything in parallel unless you put in a whole other process." So exactly the point made here.

searchableguy 6y ago

Can someone tell me why ml/dl projects don't include even a requirement.txt or list of dependencies often? I have always wondered why. I see so many projects that miss those. Projects from big companies and teams. Are they intentionally adding hurdles to using the code?

t-vi 6y ago

Indeed. The main questionable implicit assumption that is made is that Python can't "the future" (which is severely underspecified without a time horizon) because of what is bad about it today. It even is literally there "Python isn't the future... It can't be" <because something in Python is bad>. That's highly questionable, because it leaves out - that people appear to have reasons to adopt Python, likely because of something that is good about it, (this complements the observation that not that many people might be impacted by the disadvantages). - that Python and the ML tools in Python might not be static and actually evolve in "the future". It seems to me a curious lack of reflection substituted for by being particularly assertive about it. To my mind Python not being the future is more likely to happen if Python becomes less likable or the attempts to improve the shortcomings (which are there, HPy as a medium-term way to get out of the "C extension vs. parallelism/speedup/GIL" corner, mypyc/numba/TorchScript/JAX pushing the "what can be efficiently expressed in Python" boundary) fail. Julia, Swift, Dex-Lang, HaskTorch all get things right that Python doesn't. But do they get all the things right that Python does get right? Disclaimer: I'm biased because I like Python and have some involvement in Python ML tools.