3 ms·

The problem is: plain Julia is faster than plain CPython. But Numba has solved this problem for me, by compiling my hot loop in-place. Other people report simil

The problem is: plain Julia is faster than plain CPython. But Numba has solved this problem for me, by compiling my hot loop in-place. Other people report similar success with PyPy, Cython, or Pyston. Furthermore, libraries like scikit-learn, pandas, matplotlib, or scipy are incredibly powerful, and usually implement in fast languages. Python is only there to glue them together. For my applications, I just don't see any compelling reason why I should use Julia over Python. In practice, Julia is (for the above reasons) not faster in practice, and the libraries are a lot less mature. I try Julia every few months though, and there is progress. Maybe in a few years.

one-more-minute 10y ago
If you're really happy with your current tools, Julia's probably not for you right now, and that's totally fine. Julia's users tend to have the opposite of the "established libraries + glue" use case; something more like "custom data structures + unvectorisable numerics". It's better for the people who are writing the next scikit-learn than for those who are using it. The main advantage of Julia's JIT is that the performance of a given piece of code is easy to predict and debug – this is a classic problem with tracing JITs.
conjectures 10y ago
Between ccall, PyCall and RCall you can access a huge range of existing code from inside Julia. So the libraries issue is moot. Having used both, I much prefer Julia to python+cython. Three examples: multiprocessing is much less restrictive, there's no edit-compile loop so development is quicker, and no awkward pyx/pxd system. If you have to go deep with a cython project, you basically end up writing C, and that's slow - not cython's fault, it's a great tool, just a limitation of that platform.
- Derbasti 10y ago
  I don't like Cython very much. I much prefer CFFI or Numba, which have none of your stated problems.
  - conjectures 10y ago
    A quick look at the numba docs suggests that it doesn't parallelise any better than regular python - so there's one of my problems. Numba also seems to be restricted in the functionality it offers, e.g. currently looks like no user defined types so good for hot loops but not your whole codebase. If you touch the python C API it suggests it can't do full JIT compilation, i.e. sounds like this would happen with any C-extension library code outside the range of supported numpy features. No strings?... I'd love to be shown wrong about this.
jpfr 10y ago
Julia has a lot of room for improvements. For example with better type inference (programs get slooow when the type cannot be inferred, they are working on it) and stack-allocation of arrays. Currently, all arrays require a malloc. Once the JIT-compiler is sufficiently smart, Julia could use stack-allocation of temporary arrays (that cannot escape the current context). Algorithms making use of temporary arrays will become faster by possibly an order of magnitude. Julia might become competitive with C / Fortran for general purpose computing. And that is a level of speed all your other examples cannot hope to reach.

one-more-minute 10y ago

If you're really happy with your current tools, Julia's probably not for you right now, and that's totally fine. Julia's users tend to have the opposite of the "established libraries + glue" use case; something more like "custom data structures + unvectorisable numerics". It's better for the people who are writing the next scikit-learn than for those who are using it. The main advantage of Julia's JIT is that the performance of a given piece of code is easy to predict and debug – this is a classic problem with tracing JITs.

conjectures 10y ago

Between ccall, PyCall and RCall you can access a huge range of existing code from inside Julia. So the libraries issue is moot. Having used both, I much prefer Julia to python+cython. Three examples: multiprocessing is much less restrictive, there's no edit-compile loop so development is quicker, and no awkward pyx/pxd system. If you have to go deep with a cython project, you basically end up writing C, and that's slow - not cython's fault, it's a great tool, just a limitation of that platform.

Derbasti 10y ago
I don't like Cython very much. I much prefer CFFI or Numba, which have none of your stated problems.
- conjectures 10y ago
  A quick look at the numba docs suggests that it doesn't parallelise any better than regular python - so there's one of my problems. Numba also seems to be restricted in the functionality it offers, e.g. currently looks like no user defined types so good for hot loops but not your whole codebase. If you touch the python C API it suggests it can't do full JIT compilation, i.e. sounds like this would happen with any C-extension library code outside the range of supported numpy features. No strings?... I'd love to be shown wrong about this.

jpfr 10y ago

Julia has a lot of room for improvements. For example with better type inference (programs get slooow when the type cannot be inferred, they are working on it) and stack-allocation of arrays. Currently, all arrays require a malloc. Once the JIT-compiler is sufficiently smart, Julia could use stack-allocation of temporary arrays (that cannot escape the current context). Algorithms making use of temporary arrays will become faster by possibly an order of magnitude. Julia might become competitive with C / Fortran for general purpose computing. And that is a level of speed all your other examples cannot hope to reach.