4 ms·
In the long run i expect this to move away from python entirely. Yes large piece of key loops are compiled but still doesn’t fit like a good fit for something t
by Havoc 4y ago
In the long run i expect this to move away from python entirely. Yes large piece of key loops are compiled but still doesn’t fit like a good fit for something this performance critical
- thom 4y agoI hope we end up with rich, cross-platform dataframe capabilities via Arrow, but even then you need a whole ecosystem of tools to work with and visualise data. Perhaps Julia takes over but we’re some way off truly polyglot data science.
- phonebucket 4y agoIt's more than a large piece of key loops that are compiled. Almost everything is compiled. Python hardly does anything beyond orchestration in modern scientific pipelines. Even loops written in Python can be compiled away. It's almost like you are coding in a separate language altogether sometimes. E.g. you write something in Cython that gets compiled to C/C++. Same holds for things like numba/LLVM or Jax/XLA. PyTorch is going bigger on compilation in PyTorch 2.0. Python can be seen as a common interface to write code that can be JIT compiled to multiple backends. It's quite convenient to be able to interact with all these backends, and bring the results back to a universally common interface (NumPy). From this perspective, scientific Python itself is somewhat evolving by itself into something like Julia, which is JIT compiled to LLVM. Except Python's momentum has generated a huge number of backends for users to choose from. I'm all for a new language dethroning Python in scientific computing. But I don't think it will be for reasons of speed. I think it'll likely be things like typing (e.g. from Dex lang) or library modularity (e.g. Julia).
- amelius 4y agoYes, Python is much like a .BAT file that calls .EXE and .COM files.
- matsemann 4y agoDoesn't really matter, as Python is only the glue, and the hot paths happen in native code or straight on the metal/gpu. But I'd wish there were as good libraries for typed languages. Exploratory coding in python is so hard, because you don't know what you can do with a library or your tensors etc.
- brap 4y agoI like languages with strong static typeung too, but aren’t Python’s type hints enough for this purpose? Or are these libraries just untyped?
- matsemann 4y agoThe libraries can often be a bit too liberal in what they accept, so it's hard to write typings for them.
- whalesalad 4y agotype annotations and an IDE with support for that goes a long long way here.
- zwaps 4y agoBut what then? Python is the only language everyone can agree on. I'd love to code in Julia, and I'd happily use something like Rust. But realistically? Everyone does Python, and getting Python code out of critical path's is less effort than trying to get everyone to write Rust. I mean, Apple is trying to force people to code ML in Swift and I don't think I have ever even seen anyone in this space who even knows what Swift is. I mean, they provided all this nice ML framework to run stuff on M1+, and all everyone was asking: "Yeah but how do we do this from PyTorch?"
- xvilka 4y agoNobody uses Swift outside Apple.