5 ms·

I started using Julia recently. It seems like Julia has been able to take the good parts of Python and iron out the quirks. For example, I'm guessing the Julia

I started using Julia recently. It seems like Julia has been able to take the good parts of Python and iron out the quirks. For example, I'm guessing the Julia DataFrame library is a knock off of Pandas, but the syntax more intuitive and concise - and I can remember it. For Julia itself, the syntax is very similar to Python but doesn't have the weird lambda functions. It has the Javascript style arrow for short anonymous functions and the Ruby style "do" for longer functions. And finally, Julia is fast. I have a python/pandas script that take 3 days to run. Moving it over to Julia now.

ImaCake 7y ago
> I have a python/pandas script that take 3 days to run. Pandas .map() and .apply() get real slow on big datasets. I found it quicker to solve a problem with a million line dataset by just using base python iterables instead, so nothing needed to fit into my RAM and I didn't have to work with slow pandas mapping.
- appleiigs 7y ago
  Yes, i was thinking about removing the Pandas code and using python iterables. The issue is I did a lot exploration with pandas, which it was good at. If I started with Julia, i wouldn’t need to refactor seeking performance by removing pandas, or for numba, dask, etc. For the existing project i’m thinking the switch to Julia + DataFrames library (despite it being a completely different language) is more of a 1:1 port. In contrast would need to use brain power to build arrays or dicts to mimic Pandas (and probably get it wrong and introduce bugs)
  - mirekrusin 7y ago
    I hear you can use python from Julia, have you tried this as, possibly temporary/transition, approach?
    - appleiigs 7y ago
      Yes, thanks. That ability definitely does give me a bit of comfort in case Julia is missing something. I plan not to use it though. When you use it, it’s actually running python, so same speed and such.
spectramax 7y ago
Julia doesn't feel production ready at all. Its fine to mess around in notebooks but I would never recommend it for production use. Not even at a gunpoint. Debugger support is almost non-existent. Using Atom/Juno IDE is a D-grade experience. Julia offers little help to debug problems - errors are almost always without failure - completely tangent to what the real issue is. Julia takes forever to start, syntax was wonderful pre 0.4 days, now the syntax just looks absolutely jarring to my eyes. Julia's speedups can be offset by using many many far better technologies - Numba, Numpy, Cython, PyPy etc. Julia's ecosystem of libraries is a deserted land - that's expected for a new language. I hope it improves, but the core Julia experience needs to improve first. On the other hand, I've been following Rust development and the developers made absolutely sure from the get-go to focus on debugging/errors that show what the actual problem is, provide a stack trace and figure out where the problem started. Julia absolutely sucks at this.
- endoftime5 7y ago
  Just wanted to jump in here and say that for numerical computing Julia’s ecosystem of libraries are absolutely fantastic! DifferentialEquations.jl alone was worth switching to Julia! Although I spent a long time optimising numerical code with Numba the speedup I got (whilst significant) wasn’t really comparable to the speed of a Julia implementation.
  - spectramax 7y ago
    I love some of their libraries as you mentioned. Just that when the language itself is painful to write code and debug it, the whole value proposition is diminished. The core developers are of Julia are very smart folks, they want to develop a great language that's fast and easy to use. They missed the opportunity to restrict syntax, provide useful exception message (just look at Rust! it is a thing of art when you get an exception, it is beautiful), and generally provide good documentation. For example, just creating a Julia local registry requires significant overhead and time investment. Spinning up a registry should not take more than 30 mins. All these aspects of Julia are prohibitive and in my opinion, Julia should not be used in any company or production use until perhaps version 2. People who have dealt with Julia issues will tell you the truth - not the academicians or researchers. The people that maintain infrastructure/maintenance support for Julia apps are almost ready to quit their jobs. It sucks so bad.
rmah 7y ago
I took a program that used to take multiple days to run in python/numpy/scipy/pandas moved it to julia... and got it down to just a few hours. Something like a 20x speed improvement. It seems the biggest speedups are gained if you happened to have implemented custom functions in python since those cannot take as much advantage of C numpy/scipy/pandas cores. That said, I still find numpy/scipy/pandas more natural. But that's probably simply because I'm more familiar with them and python.

ImaCake 7y ago

> I have a python/pandas script that take 3 days to run. Pandas .map() and .apply() get real slow on big datasets. I found it quicker to solve a problem with a million line dataset by just using base python iterables instead, so nothing needed to fit into my RAM and I didn't have to work with slow pandas mapping.

appleiigs 7y ago
Yes, i was thinking about removing the Pandas code and using python iterables. The issue is I did a lot exploration with pandas, which it was good at. If I started with Julia, i wouldn’t need to refactor seeking performance by removing pandas, or for numba, dask, etc. For the existing project i’m thinking the switch to Julia + DataFrames library (despite it being a completely different language) is more of a 1:1 port. In contrast would need to use brain power to build arrays or dicts to mimic Pandas (and probably get it wrong and introduce bugs)
- mirekrusin 7y ago
  I hear you can use python from Julia, have you tried this as, possibly temporary/transition, approach?
  - appleiigs 7y ago
    Yes, thanks. That ability definitely does give me a bit of comfort in case Julia is missing something. I plan not to use it though. When you use it, it’s actually running python, so same speed and such.

spectramax 7y ago

Julia doesn't feel production ready at all. Its fine to mess around in notebooks but I would never recommend it for production use. Not even at a gunpoint. Debugger support is almost non-existent. Using Atom/Juno IDE is a D-grade experience. Julia offers little help to debug problems - errors are almost always without failure - completely tangent to what the real issue is. Julia takes forever to start, syntax was wonderful pre 0.4 days, now the syntax just looks absolutely jarring to my eyes. Julia's speedups can be offset by using many many far better technologies - Numba, Numpy, Cython, PyPy etc. Julia's ecosystem of libraries is a deserted land - that's expected for a new language. I hope it improves, but the core Julia experience needs to improve first. On the other hand, I've been following Rust development and the developers made absolutely sure from the get-go to focus on debugging/errors that show what the actual problem is, provide a stack trace and figure out where the problem started. Julia absolutely sucks at this.

endoftime5 7y ago
Just wanted to jump in here and say that for numerical computing Julia’s ecosystem of libraries are absolutely fantastic! DifferentialEquations.jl alone was worth switching to Julia! Although I spent a long time optimising numerical code with Numba the speedup I got (whilst significant) wasn’t really comparable to the speed of a Julia implementation.
- spectramax 7y ago
  I love some of their libraries as you mentioned. Just that when the language itself is painful to write code and debug it, the whole value proposition is diminished. The core developers are of Julia are very smart folks, they want to develop a great language that's fast and easy to use. They missed the opportunity to restrict syntax, provide useful exception message (just look at Rust! it is a thing of art when you get an exception, it is beautiful), and generally provide good documentation. For example, just creating a Julia local registry requires significant overhead and time investment. Spinning up a registry should not take more than 30 mins. All these aspects of Julia are prohibitive and in my opinion, Julia should not be used in any company or production use until perhaps version 2. People who have dealt with Julia issues will tell you the truth - not the academicians or researchers. The people that maintain infrastructure/maintenance support for Julia apps are almost ready to quit their jobs. It sucks so bad.

rmah 7y ago

I took a program that used to take multiple days to run in python/numpy/scipy/pandas moved it to julia... and got it down to just a few hours. Something like a 20x speed improvement. It seems the biggest speedups are gained if you happened to have implemented custom functions in python since those cannot take as much advantage of C numpy/scipy/pandas cores. That said, I still find numpy/scipy/pandas more natural. But that's probably simply because I'm more familiar with them and python.