4 ms·

" I mean, faster than r or Python (for which I could write fast code but that'd mean I'd have to change the way it is written)" You might benefit from numba. I

" I mean, faster than r or Python (for which I could write fast code but that'd mean I'd have to change the way it is written)" You might benefit from numba. I've used it to speed my Python up enormously and completely painlessly just by adding decorators to critical functions. It's why I'm not considering moving to Julia.

CornCobs 5y ago
Is it actually completely painless? YMMV, but for me I found that there were many unsupported parts of Numpy I had to work around which meant I was effectively doing a full rewrite. Especially assignments using boolean masks and working with multi-dimensional arrays in general is really tough
- garyrob 5y ago
  It just happened wasn't using Numpy except in the most basic possible ways. I had hand-written algorithms in python. That's where it shines. So, yes YMMV! It worked really well for me.
- nextos 5y ago
  I think Numba has some pain points. I have also encountered issues with multi-dimensional arrays. The beauty of Julia is that relatively naive Ruby-like code is already quite quick. And if you implement inner loops in an imperative way, with an eye towards not generating excessive allocations, it can approach C++ speed while still being nice high-level code that is close to mathematics or business logic. Besides, the other strong point of Julia is composability. The ecosystem is made up by lots of small libraries that can interact in ways the original designers did not expect or plan for. In contrast, Python has exceptional libraries, but they tend to be big monoliths. The problem with Julia right now is that some libraries are not sufficiently mature. For example, there's no mature native replacement for XLA or PyTorch. I know about Flux, but it's nowhere close if you wanna create, say, a large transformer. Or say you are working with GLMs. GLM.jl is nowhere close to R. Some other Julia libraries represent the state of the art, though. I just can't wait to get all foundations complete! It's a really promising space for probabilistic and differentiable programming.
baldfat 5y ago
There are so many ways to write R that can actually be very fast. R is a very different language as a whole and it has addressed a lot of problems these last 10 years. I might be biased but I really do like the functional side of R and how logical the libraries from Hadley Wickham have been designed. Python still doesn't feel like a natural fix for data science work. I am guessing it is more bias opinion but why based on 0 for this domain????
- sidkshatriya 5y ago
  I actually prefer the 0 indexing of python. Systems code (c/c++ etc.) already uses 0 based indexing. So it is nice that when you do data science the convention stays the same.
  - Mikeb85 5y ago
    But Fortran, R, Matlab and other tools in the domain use 1 indexing...
  - runarberg 5y ago
    Does it actually matter at this point? I write off by one errors all the time in JavaScript and I usually spot them immediately. I’ve written Julia and R in the past and I found it really easy to switch between contexts, and if I forget, the errors are such that it takes only a few seconds to spot and fix them.
- wiz21c 5y ago
  > There are so many ways to write R that can actually be very fast. But it means that you have to work with arrays. For me it often means breaking the flow of my (code) explanation to migrate to other data structures. Sure it is then fast, but it gets less readable and harder to update. Now, I'm a programmer at heart, so I think in the "functional" paradigm, not the array/signal one. There's some "impedance" I guess :-)
tkuraku 5y ago
Numba is awesome. It solves a lot of problems. Julia is like numba but you are able to use any libraries inside the loops. Think FFT, scipy, etc
aqme28 5y ago
I’ve become a big fan of Numba. No more awkward, human-unreadable vectorized code.
- adgjlsfhk1 5y ago
  IMO, numba loses pretty much all of the advantages of python. The code is fast, but you lose the ability to organize your data the way you would in regular python, and most of the python ecosystem can't be called from within numba functions. If I wanted that developer experience, I would just use C.
  - aqme28 5y ago
    I don't think that's true. You lose some functionality, but since you can call Numba on select critical methods, it's not so bad. You can often sequester your performance-critical logic into some Numba-decorated methods, and your business elements that call out to the rest of your ecosystem are not decorated with Numba. Or to put it another way-- if I'm using vectorized code in Numpy I can't deal with the external ecosystem from within my vectorized code either. > The code is fast, but you lose the ability to organize your data the way you would in regular python, Can you clarify what you mean by this? I lose the ability to organize my data the way I'd like if I'm vectorizing my code too.
    - adgjlsfhk1 5y ago
      I wasn't comparing numba to numpy. I was comparing to python code where you don't care about performance. The main reason I don't find numba appealing is that Julia gives you numba like performance while allowing you to use structs (think classes) to organize stuff.
      - aqme28 5y ago
        This doesn't really make sense. If you don't care about performance then you have no use for Numba. Also, the comment you replied to was explicitly comparing Numba to vectorized Python, so you should not abandon that comparison in your reply without saying so.
KolenCh 5y ago
For simple things only. For example, the jit class in Numba is still experimental, and quite limited. So if you need to write non-trivial class, even if you jit each method, dropping to Python can be a problem (say with lots of instances.) I have tried to write an application targeting HPC and in itself is not very complicated (probably ~2000 lines or in that order.) But I did things like using the Python language as the metaprogramming language for Numba (basically higher-order function where you jit inside.) All in all my experience of Numba tells me that if I am designing the same package now I'd write it in Julia where jit is "first class" and you don't need to constantly thing about the boundary between Numba and Python.

CornCobs 5y ago

Is it actually completely painless? YMMV, but for me I found that there were many unsupported parts of Numpy I had to work around which meant I was effectively doing a full rewrite. Especially assignments using boolean masks and working with multi-dimensional arrays in general is really tough

garyrob 5y ago
It just happened wasn't using Numpy except in the most basic possible ways. I had hand-written algorithms in python. That's where it shines. So, yes YMMV! It worked really well for me.
nextos 5y ago
I think Numba has some pain points. I have also encountered issues with multi-dimensional arrays. The beauty of Julia is that relatively naive Ruby-like code is already quite quick. And if you implement inner loops in an imperative way, with an eye towards not generating excessive allocations, it can approach C++ speed while still being nice high-level code that is close to mathematics or business logic. Besides, the other strong point of Julia is composability. The ecosystem is made up by lots of small libraries that can interact in ways the original designers did not expect or plan for. In contrast, Python has exceptional libraries, but they tend to be big monoliths. The problem with Julia right now is that some libraries are not sufficiently mature. For example, there's no mature native replacement for XLA or PyTorch. I know about Flux, but it's nowhere close if you wanna create, say, a large transformer. Or say you are working with GLMs. GLM.jl is nowhere close to R. Some other Julia libraries represent the state of the art, though. I just can't wait to get all foundations complete! It's a really promising space for probabilistic and differentiable programming.

baldfat 5y ago

There are so many ways to write R that can actually be very fast. R is a very different language as a whole and it has addressed a lot of problems these last 10 years. I might be biased but I really do like the functional side of R and how logical the libraries from Hadley Wickham have been designed. Python still doesn't feel like a natural fix for data science work. I am guessing it is more bias opinion but why based on 0 for this domain????

sidkshatriya 5y ago
I actually prefer the 0 indexing of python. Systems code (c/c++ etc.) already uses 0 based indexing. So it is nice that when you do data science the convention stays the same.
- Mikeb85 5y ago
  But Fortran, R, Matlab and other tools in the domain use 1 indexing...
- runarberg 5y ago
  Does it actually matter at this point? I write off by one errors all the time in JavaScript and I usually spot them immediately. I’ve written Julia and R in the past and I found it really easy to switch between contexts, and if I forget, the errors are such that it takes only a few seconds to spot and fix them.
wiz21c 5y ago
> There are so many ways to write R that can actually be very fast. But it means that you have to work with arrays. For me it often means breaking the flow of my (code) explanation to migrate to other data structures. Sure it is then fast, but it gets less readable and harder to update. Now, I'm a programmer at heart, so I think in the "functional" paradigm, not the array/signal one. There's some "impedance" I guess :-)

tkuraku 5y ago

Numba is awesome. It solves a lot of problems. Julia is like numba but you are able to use any libraries inside the loops. Think FFT, scipy, etc

aqme28 5y ago

I’ve become a big fan of Numba. No more awkward, human-unreadable vectorized code.

adgjlsfhk1 5y ago
IMO, numba loses pretty much all of the advantages of python. The code is fast, but you lose the ability to organize your data the way you would in regular python, and most of the python ecosystem can't be called from within numba functions. If I wanted that developer experience, I would just use C.
- aqme28 5y ago
  I don't think that's true. You lose some functionality, but since you can call Numba on select critical methods, it's not so bad. You can often sequester your performance-critical logic into some Numba-decorated methods, and your business elements that call out to the rest of your ecosystem are not decorated with Numba. Or to put it another way-- if I'm using vectorized code in Numpy I can't deal with the external ecosystem from within my vectorized code either. > The code is fast, but you lose the ability to organize your data the way you would in regular python, Can you clarify what you mean by this? I lose the ability to organize my data the way I'd like if I'm vectorizing my code too.
  - adgjlsfhk1 5y ago
    I wasn't comparing numba to numpy. I was comparing to python code where you don't care about performance. The main reason I don't find numba appealing is that Julia gives you numba like performance while allowing you to use structs (think classes) to organize stuff.
    - aqme28 5y ago
      This doesn't really make sense. If you don't care about performance then you have no use for Numba. Also, the comment you replied to was explicitly comparing Numba to vectorized Python, so you should not abandon that comparison in your reply without saying so.

KolenCh 5y ago

For simple things only. For example, the jit class in Numba is still experimental, and quite limited. So if you need to write non-trivial class, even if you jit each method, dropping to Python can be a problem (say with lots of instances.) I have tried to write an application targeting HPC and in itself is not very complicated (probably ~2000 lines or in that order.) But I did things like using the Python language as the metaprogramming language for Numba (basically higher-order function where you jit inside.) All in all my experience of Numba tells me that if I am designing the same package now I'd write it in Julia where jit is "first class" and you don't need to constantly thing about the boundary between Numba and Python.