5 ms·

What python solutions are scatterbrained? Have you tried Numba?

by Nrpf 11y ago

oiuswv 11y ago
Here, I'll use logical indexing to pull out all the positive even numbers from this array of random integers: Numpy: a = np.random.random_integers(low = -100, high = 100, size = (100,)) a[np.logical_and(a % 2 == 0, a > 0) (or a[(a % 2 == 0) * (a > 0)]) Logical indexing is used everywhere in numerical computing. Using functions like logical_and() or boolean multiplication or addition is more difficult to follow than Matlab or Julia. Julia: a = rand(-100:100, 100) a[(a % 2 .== 0) & (a .> 0)] Well, that's beautiful. Element-wise operations are prefixed with "."
- jdreaver 11y ago
  You have made a very verbose numpy version :) Here is a slightly better one: a = np.random.randint(-100, 100, size=100) a[(a & 2 == 0) & (a > 0)] Python does indeed have logical boolean operators. I think this is clearer than your Julia example because: - The numpy version makes it clear that you are using random integers, not floating point values. - The keyword argument for "size" makes it clear what that second 100 is for. (The use of the first two numbers, -100 and 100, is pretty clear from context.) - In numpy, most operations are element-wise by default, because the result would be ambiguous or not useful otherwise. This removes the line noise of the extra "." before operations. Don't get me wrong, I think Julia is awesome. I just think you've constructed a very poor example for numpy.
  - timholy 11y ago
    > The numpy version makes it clear that you are using random integers, not floating point values. That's also completely clear in the Julia version, if you learn a little Julia. > In numpy, most operations are element-wise by default, because the result would be ambiguous or not useful otherwise. This is why I think the Julia approach is better. If I write `a == 7`, am I testing whether `a` is 7 or whether any of the elements of `a` are 7?
    - jdreaver 11y ago
      > That's also completely clear in the Julia version, if you learn a little Julia. In every language I've used, the default is for a "rand" function to return random floats between 0 and 1, and given arguments it returns floats between the arguments. I don't think it has to do with learning Julia, it is just that including "integer" in the function name makes it clear the function returns integers. > This is why I think the Julia approach is better. If I write `a == 7`, am I testing whether `a` is 7 or whether any of the elements of `a` are 7? I think this is more of a comment about mixing arrays and scalars in a dynamic language. I made my comment assuming you are performing operations on arrays. If you are comparing two arrays, I think the default of element-wise operations makes more sense.
      - KenoFischer 11y ago
        In julia the rand function is more general in that it samples from a distribution, which you can pass as the first argument (defaulting to uniform on [0,1]). Since the first argument is an integer range, you get an integer value.
Tarrosion 11y ago
I'm not the parent author, but why I haven't tried the Python ecosystem: I've heard of NumPy, PyPy, SciPy, Pandas, matplotlib, and now Numba. I don't particularly know what these do or how they overlap or interact. Which is kind of the point: to a complete outsider, the world of Python scientific computing feels like a wild west where everybody is happily proclaiming that their setup is just right. Additionally, Python is slow; Julia is fast. I've heard things like "well, Python is only slow at certain things, and makes it easy to write C code when needed." I don't want to write any C code. Same goes for "typically only a small portion of your Python code is a bottleneck, and it's easy to port that to C." A huge part of the appeal of Julia is that you don't have to worry about language interoperability, calling C, etc. Everything can be written efficiently and readably in Julia, and because the base language is designed around scientific computing, there's little worry about add-on scientific computing packages not playing nice together. Now, I fully believe that I could get the right Python environment and set of packages set up be productive. But to get going with Julia, I just download Julia and go. Also, JuMP [1] is just amazing. [1] http://www.juliaopt.org http://www.juliaopt.org
- ngoldbaum 11y ago
  I usually use cython for optimizing python, which is a lot easier than working directly with the CPython C API. You do need to know C to get the most out of it, but you can go a long way just adding type declarations to python to speed it up.
- Redoubts 11y ago
  > NumPy, PyPy, SciPy, Pandas, matplotlib, and now Numba I'm not sure I buy this argument. Except for pypy, these are all complementary packages and work together. If you breakup anything into its constituent parts, you can make it seem complicated if you want to.
- stdbrouw 11y ago
  > But to get going with Julia, I just download Julia and go. Until you need to do something very straightforward like web scraping or interacting with AWS and find out that nobody's released a package for that yet, so you're going to have to reinvent the wheel before you can get to the scientific question that is of actual interest. Julia looks very interesting as a language, let's just hope it doesn't end up a ghetto like R (lots of awesome statistical tools, a dearth of libraries and tools for everything else).

What python solutions are scatterbrained? Have you tried Numba?

by Nrpf 11y ago

What python solutions are scatterbrained? Have you tried Numba?

Here, I'll use logical indexing to pull out all the positive even numbers from this array of random integers: Numpy: a = np.random.random_integers(low = -100, high = 100, size = (100,)) a[np.logical_and(a % 2 == 0, a > 0) (or a[(a % 2 == 0) * (a > 0)]) Logical indexing is used everywhere in numerical computing. Using functions like logical_and() or boolean multiplication or addition is more difficult to follow than Matlab or Julia. Julia: a = rand(-100:100, 100) a[(a % 2 .== 0) & (a .> 0)] Well, that's beautiful. Element-wise operations are prefixed with "."

jdreaver 11y ago
You have made a very verbose numpy version :) Here is a slightly better one: a = np.random.randint(-100, 100, size=100) a[(a & 2 == 0) & (a > 0)] Python does indeed have logical boolean operators. I think this is clearer than your Julia example because: - The numpy version makes it clear that you are using random integers, not floating point values. - The keyword argument for "size" makes it clear what that second 100 is for. (The use of the first two numbers, -100 and 100, is pretty clear from context.) - In numpy, most operations are element-wise by default, because the result would be ambiguous or not useful otherwise. This removes the line noise of the extra "." before operations. Don't get me wrong, I think Julia is awesome. I just think you've constructed a very poor example for numpy.
- timholy 11y ago
  > The numpy version makes it clear that you are using random integers, not floating point values. That's also completely clear in the Julia version, if you learn a little Julia. > In numpy, most operations are element-wise by default, because the result would be ambiguous or not useful otherwise. This is why I think the Julia approach is better. If I write `a == 7`, am I testing whether `a` is 7 or whether any of the elements of `a` are 7?
  - jdreaver 11y ago
    > That's also completely clear in the Julia version, if you learn a little Julia. In every language I've used, the default is for a "rand" function to return random floats between 0 and 1, and given arguments it returns floats between the arguments. I don't think it has to do with learning Julia, it is just that including "integer" in the function name makes it clear the function returns integers. > This is why I think the Julia approach is better. If I write `a == 7`, am I testing whether `a` is 7 or whether any of the elements of `a` are 7? I think this is more of a comment about mixing arrays and scalars in a dynamic language. I made my comment assuming you are performing operations on arrays. If you are comparing two arrays, I think the default of element-wise operations makes more sense.
    - KenoFischer 11y ago
      In julia the rand function is more general in that it samples from a distribution, which you can pass as the first argument (defaulting to uniform on [0,1]). Since the first argument is an integer range, you get an integer value.

Tarrosion 11y ago

I'm not the parent author, but why I haven't tried the Python ecosystem: I've heard of NumPy, PyPy, SciPy, Pandas, matplotlib, and now Numba. I don't particularly know what these do or how they overlap or interact. Which is kind of the point: to a complete outsider, the world of Python scientific computing feels like a wild west where everybody is happily proclaiming that their setup is just right. Additionally, Python is slow; Julia is fast. I've heard things like "well, Python is only slow at certain things, and makes it easy to write C code when needed." I don't want to write any C code. Same goes for "typically only a small portion of your Python code is a bottleneck, and it's easy to port that to C." A huge part of the appeal of Julia is that you don't have to worry about language interoperability, calling C, etc. Everything can be written efficiently and readably in Julia, and because the base language is designed around scientific computing, there's little worry about add-on scientific computing packages not playing nice together. Now, I fully believe that I could get the right Python environment and set of packages set up be productive. But to get going with Julia, I just download Julia and go. Also, JuMP [1] is just amazing. [1] http://www.juliaopt.org http://www.juliaopt.org

ngoldbaum 11y ago
I usually use cython for optimizing python, which is a lot easier than working directly with the CPython C API. You do need to know C to get the most out of it, but you can go a long way just adding type declarations to python to speed it up.
Redoubts 11y ago
> NumPy, PyPy, SciPy, Pandas, matplotlib, and now Numba I'm not sure I buy this argument. Except for pypy, these are all complementary packages and work together. If you breakup anything into its constituent parts, you can make it seem complicated if you want to.
stdbrouw 11y ago
> But to get going with Julia, I just download Julia and go. Until you need to do something very straightforward like web scraping or interacting with AWS and find out that nobody's released a package for that yet, so you're going to have to reinvent the wheel before you can get to the scientific question that is of actual interest. Julia looks very interesting as a language, let's just hope it doesn't end up a ghetto like R (lots of awesome statistical tools, a dearth of libraries and tools for everything else).