4 ms·

I think you may be overgeneralizing. If the speed of number crunching is the only concern, then sure, few scientists would use Python, R, or MATLAB. This is n

I think you may be overgeneralizing. If the speed of number crunching is the only concern, then sure, few scientists would use Python, R, or MATLAB. This is not the case in reality. Whether you would run into speed limit of course depends on the size of the problem. It's fine to use a "slow" scripting language to process a small data set on a personal laptop. And I suspect that this is what most everyday numpy users do. Once you move to the other end of the spectrum where the size of the problem requires a supercomputer, that is when the Python ecosystem becomes clunky and painful. I think that could be an area that Julia shines in. But even that, I'm not sure if Julia is a clear winner yet, as the field is still largely dominated by C, C++, and Fortran. Also, check ref. 56 in the article. They did cite Julia, among others.

smabie 6y ago
The point is that Julia is easier to use and faster than Python for just about any scientific task. Even if Python is fast enough for you (which it probably isn't), it's harder to use and less expressive than Julia. This is why I consider it strictly inferior: it is not better at anything at all, only worse. Python is so slow it is even unusable on trivially small data sets. Also it uses a massive amount of ram. I loaded a 2gb csv file the other day in Pandas, and it consumed over 40gb of ram. That's totally unacceptable in my view.
- geoalchimista 6y ago
  > The point is that Julia is easier to use and faster than Python for just about any scientific task. Try to make a plot. Last time I checked, the time to first plot was still a pain point of Julia. > it's harder to use and less expressive than Julia. I agree that Julia is more expressive because it's a Lisp in disguise. This is definitely a plus. But in practice, I don't find Python to be "harder to use." There is good consistency among the mainstream packages in terms of API. For example, if you have a NumPy function np.mean, you can assume that Pandas would have a method .mean() for the DataFrame, and Dask would have that for DaskArray as well. Not always, but things are moving toward that direction. > I loaded a 2gb csv file the other day in Pandas, and it consumed over 40gb of ram. Have you tried the `engine='c'` option when calling pandas.read_csv? Pretty sure there is also another option of chunking that may be useful.
  - rsfern 6y ago
    Time to first plot is still painful even in julia 1.5, sure. But after paying that startup cost, the speed boost can be transformative in terms of the flexibility and dynamism that it buys. I've been using Pluto.jl lately, and interactive data analysis feels like way less of a burden than working in jupyter. As to harder to use, writing fast vectorized numpy code can take a lot of mental effort. For one project I ended up forcing things into this 5-D array broadcasting mess, where in julia it's way simpler to just write intuitive code that's performant without bending over backwards to avoid explicit loops. For numpy code you can "just" drop down to cython, but for pytorch you can get stuck with thinking up some clever broadcasting solution.
- naveen99 6y ago
  Yeah, string processing in python is slow. But for csv files of size, you are probably better of using SQLite or sql server over even Julia. If pytorch wasn’t able to keep the gpu busy, I would be more inclined to try Julia or lisp again.
  - eigenspace 6y ago
    > If pytorch wasn’t able to keep the gpu busy, I would be more inclined to try Julia or lisp again. Julia's GPU programming stack is fantastic and very advanced btw.
rsfern 6y ago
to be fair, the Julia citation is sort of a token sentence in the discussion section. > New generation languages, interpreters and compilers, such as Rust [55], Julia [56] and LLVM [57], will create new concepts and data structures, and determine their viability. I don't disagree with the statement, but there's not a lot of meat there either.

smabie 6y ago

The point is that Julia is easier to use and faster than Python for just about any scientific task. Even if Python is fast enough for you (which it probably isn't), it's harder to use and less expressive than Julia. This is why I consider it strictly inferior: it is not better at anything at all, only worse. Python is so slow it is even unusable on trivially small data sets. Also it uses a massive amount of ram. I loaded a 2gb csv file the other day in Pandas, and it consumed over 40gb of ram. That's totally unacceptable in my view.

geoalchimista 6y ago
> The point is that Julia is easier to use and faster than Python for just about any scientific task. Try to make a plot. Last time I checked, the time to first plot was still a pain point of Julia. > it's harder to use and less expressive than Julia. I agree that Julia is more expressive because it's a Lisp in disguise. This is definitely a plus. But in practice, I don't find Python to be "harder to use." There is good consistency among the mainstream packages in terms of API. For example, if you have a NumPy function np.mean, you can assume that Pandas would have a method .mean() for the DataFrame, and Dask would have that for DaskArray as well. Not always, but things are moving toward that direction. > I loaded a 2gb csv file the other day in Pandas, and it consumed over 40gb of ram. Have you tried the `engine='c'` option when calling pandas.read_csv? Pretty sure there is also another option of chunking that may be useful.
- rsfern 6y ago
  Time to first plot is still painful even in julia 1.5, sure. But after paying that startup cost, the speed boost can be transformative in terms of the flexibility and dynamism that it buys. I've been using Pluto.jl lately, and interactive data analysis feels like way less of a burden than working in jupyter. As to harder to use, writing fast vectorized numpy code can take a lot of mental effort. For one project I ended up forcing things into this 5-D array broadcasting mess, where in julia it's way simpler to just write intuitive code that's performant without bending over backwards to avoid explicit loops. For numpy code you can "just" drop down to cython, but for pytorch you can get stuck with thinking up some clever broadcasting solution.
naveen99 6y ago
Yeah, string processing in python is slow. But for csv files of size, you are probably better of using SQLite or sql server over even Julia. If pytorch wasn’t able to keep the gpu busy, I would be more inclined to try Julia or lisp again.
- eigenspace 6y ago
  > If pytorch wasn’t able to keep the gpu busy, I would be more inclined to try Julia or lisp again. Julia's GPU programming stack is fantastic and very advanced btw.

rsfern 6y ago

to be fair, the Julia citation is sort of a token sentence in the discussion section. > New generation languages, interpreters and compilers, such as Rust [55], Julia [56] and LLVM [57], will create new concepts and data structures, and determine their viability. I don't disagree with the statement, but there's not a lot of meat there either.