4 ms·

Not to sound dismissive but I remember when that was true of Python (vis-a-vis Perl and some other things). My recollection of discussions on forums when it was

Not to sound dismissive but I remember when that was true of Python (vis-a-vis Perl and some other things). My recollection of discussions on forums when it was starting to gain mindshare were arguments largely about the aesthetics of the syntax. Numerical computing is something where there's always been something that works. Before Python it was C and Fortran. The problem people ran into is that when everything is wrapped around low-level libraries for speed, you eventually run into the catch 22 of using the slower language that you prefer for clarity, or the faster one that makes it acceptable performance-wise. In other words, with Python, to get the performance you would have got in C or Fortran, you have to code in C or Fortran. Then you're not using Python anymore. The idea (in theory, and a lot in practice) with Julia (or Nim, or other LLVM-targeting languages) is that you don't have this penalty. So in that sense Julia is providing something that isn't working in Python or Matlab. I think Julia's not quite what it's cracked up to be, but mostly it is, and I'd probably prefer working with it over Python or Matlab for numerical stuff.

bachmeier 6y ago
> I remember when that was true of Python But Python didn't take off because of scientific computing. Google was a heavy user of Python from the start. Python seriously sucked for scientific computing in 2005. I was looking for a new language (the old one wasn't cutting it) and I'd been looking for a use for Python for years. It had some projects but it just was not in a useful state so I went with R, which was quite mature by that time, and it focused on statistical analysis. Years later Python became popular for scientific computing, likely due to the userbase it had built up in other areas.
- teorema 6y ago
  Yeah I don't disagree re: scientific computing and Python (although I think from the beginning some of the advocacy for Python was coming from more "math-oriented" communities). I still think that when it started accelerating in use (which was mainly among scripting and then web applications, some other stuff too) there were existing solutions. A lot of discussions about established languages versus new ones are very similar to one another; the particular languages are just swapped out for different ones.
- cb321 6y ago
  While it's true in 2004 there was still a Numeric/numarray schism, at the time I was using Pyrex (which soon evolved into Cython) to have a gradually strongly typed Python-like thing cross-compatible with the rest of the ecosystem. That made it quite easy to write code which ran circles around R performance-wise without leaving the Python syntax domain and with a very simple FFI to call C to boot. I even had a tiny "pycc" script to create "executables" instead of "importable modules". Yes, those executables did depend on the installed base of Python stuff. These days, Nim is a better Cython but with less dynamic temptations and more powerful metaprogramming (and, yes, a much smaller ecosystem..maybe not that much smaller than Python in the late 90s, though). { Not that this is all Nim is...It's actually a really good everything-language that's tricky to summarize in just a few words. }
marmaduke 6y ago
I think even this is overstated: Python via Numba and MATLAB both have excellent JIT compilers now, and for the maximum performance you need to hit SIMD. Latter is not reliable in Julia and the former, well, you already have Python and MATLAB.
- stabbles 6y ago
  "SIMD not reliable" is an overstatement itself for a couple reasons: 1. It's very easy to inspect generated code of your kernels where you really need SIMD. For instance: julia> function my_kernel(xs) total = zero(eltype(xs)) for x in xs @fastmath total += x end total end julia> @code_native debuginfo=:none my_kernel(rand(10)) ... L96: vaddpd 8(%rcx,%rax,8), %ymm0, %ymm0 vaddpd 40(%rcx,%rax,8), %ymm1, %ymm1 vaddpd 72(%rcx,%rax,8), %ymm2, %ymm2 vaddpd 104(%rcx,%rax,8), %ymm3, %ymm3 addq $16, %rax cmpq %rax, %rdi jne L96 ... 2. There's a Julia package that does the code gen for vectorization exactly because LLVM does not always get it right: https://chriselrod.github.io/LoopVectorization.jl/stable/examples/matrix_multiplication/ https://chriselrod.github.io/LoopVectorization.jl/stable/exa... 3. You can make LLVM explain why it did not vectorize certain loops just like in clang.
  - marmaduke 6y ago
    SIMD convergence is a big issue here: how do you keep those lanes running together? If you have trivial kernels sure, but ISPC guarantees convergence across control flow which isn’t even available in CUDA (according to ISPC docs at least). Just to be clear: I want to use Julia but without guessing about SIMD use in kernels. Until this is available for complex control flow in Julia, it’s not a game changer in my opinion. Edit: that kernel is trivial. I have nested control flow to vectorize.
    - stabbles 6y ago
      Right, I see what you're getting at. Just taking a slightly less trivial example from Intel's docs, and superficially comparing the assembly they generate to what LoopVectorization.jl can do, I guess there is some hope at least: julia> function simple!(ys, xs) @avx for i = eachindex(xs) ys[i] = if xs[i] < 3. xs[i]^2 else sqrt(xs[i]) end end end This generates vbroadcastsd, vpcmpgtq, vmulpd, vsqrtpd, vblendvpd and vmaskmovpd AVX2 instructions (I don't have AVX512). But more complex control flow does indeed not work; the `@avx` macro errors it can't handle it at the moment.
marmaduke 6y ago
> Before Python it was C and Fortran And before that, analog circuits. Hodgkin and Huxley computed their first results for neuronal activity without a computer.
- gnufx 6y ago
  Although you might have needed the distinction between an electronic computer and a human one then.
- pasttense01 6y ago
  No. Before C and Fortran it was assembly language.
gnufx 6y ago
> My recollection of discussions on forums when it was starting to gain mindshare were arguments largely about the aesthetics of the syntax. I remember dynamic languages people picking up on three things in particular: Python apparently being designed to be inherently inefficient, not having proper garbage collection, and having weird scoping. At least Python was something to point people at who rejected Lisp as an alternative to Tcl and Perl for scientific computing.

bachmeier 6y ago

> I remember when that was true of Python But Python didn't take off because of scientific computing. Google was a heavy user of Python from the start. Python seriously sucked for scientific computing in 2005. I was looking for a new language (the old one wasn't cutting it) and I'd been looking for a use for Python for years. It had some projects but it just was not in a useful state so I went with R, which was quite mature by that time, and it focused on statistical analysis. Years later Python became popular for scientific computing, likely due to the userbase it had built up in other areas.

teorema 6y ago
Yeah I don't disagree re: scientific computing and Python (although I think from the beginning some of the advocacy for Python was coming from more "math-oriented" communities). I still think that when it started accelerating in use (which was mainly among scripting and then web applications, some other stuff too) there were existing solutions. A lot of discussions about established languages versus new ones are very similar to one another; the particular languages are just swapped out for different ones.
cb321 6y ago
While it's true in 2004 there was still a Numeric/numarray schism, at the time I was using Pyrex (which soon evolved into Cython) to have a gradually strongly typed Python-like thing cross-compatible with the rest of the ecosystem. That made it quite easy to write code which ran circles around R performance-wise without leaving the Python syntax domain and with a very simple FFI to call C to boot. I even had a tiny "pycc" script to create "executables" instead of "importable modules". Yes, those executables did depend on the installed base of Python stuff. These days, Nim is a better Cython but with less dynamic temptations and more powerful metaprogramming (and, yes, a much smaller ecosystem..maybe not that much smaller than Python in the late 90s, though). { Not that this is all Nim is...It's actually a really good everything-language that's tricky to summarize in just a few words. }

marmaduke 6y ago

I think even this is overstated: Python via Numba and MATLAB both have excellent JIT compilers now, and for the maximum performance you need to hit SIMD. Latter is not reliable in Julia and the former, well, you already have Python and MATLAB.

stabbles 6y ago
"SIMD not reliable" is an overstatement itself for a couple reasons: 1. It's very easy to inspect generated code of your kernels where you really need SIMD. For instance: julia> function my_kernel(xs) total = zero(eltype(xs)) for x in xs @fastmath total += x end total end julia> @code_native debuginfo=:none my_kernel(rand(10)) ... L96: vaddpd 8(%rcx,%rax,8), %ymm0, %ymm0 vaddpd 40(%rcx,%rax,8), %ymm1, %ymm1 vaddpd 72(%rcx,%rax,8), %ymm2, %ymm2 vaddpd 104(%rcx,%rax,8), %ymm3, %ymm3 addq $16, %rax cmpq %rax, %rdi jne L96 ... 2. There's a Julia package that does the code gen for vectorization exactly because LLVM does not always get it right: https://chriselrod.github.io/LoopVectorization.jl/stable/examples/matrix_multiplication/ https://chriselrod.github.io/LoopVectorization.jl/stable/exa... 3. You can make LLVM explain why it did not vectorize certain loops just like in clang.
- marmaduke 6y ago
  SIMD convergence is a big issue here: how do you keep those lanes running together? If you have trivial kernels sure, but ISPC guarantees convergence across control flow which isn’t even available in CUDA (according to ISPC docs at least). Just to be clear: I want to use Julia but without guessing about SIMD use in kernels. Until this is available for complex control flow in Julia, it’s not a game changer in my opinion. Edit: that kernel is trivial. I have nested control flow to vectorize.
  - stabbles 6y ago
    Right, I see what you're getting at. Just taking a slightly less trivial example from Intel's docs, and superficially comparing the assembly they generate to what LoopVectorization.jl can do, I guess there is some hope at least: julia> function simple!(ys, xs) @avx for i = eachindex(xs) ys[i] = if xs[i] < 3. xs[i]^2 else sqrt(xs[i]) end end end This generates vbroadcastsd, vpcmpgtq, vmulpd, vsqrtpd, vblendvpd and vmaskmovpd AVX2 instructions (I don't have AVX512). But more complex control flow does indeed not work; the `@avx` macro errors it can't handle it at the moment.

marmaduke 6y ago

> Before Python it was C and Fortran And before that, analog circuits. Hodgkin and Huxley computed their first results for neuronal activity without a computer.

gnufx 6y ago
Although you might have needed the distinction between an electronic computer and a human one then.
pasttense01 6y ago
No. Before C and Fortran it was assembly language.

gnufx 6y ago

> My recollection of discussions on forums when it was starting to gain mindshare were arguments largely about the aesthetics of the syntax. I remember dynamic languages people picking up on three things in particular: Python apparently being designed to be inherently inefficient, not having proper garbage collection, and having weird scoping. At least Python was something to point people at who rejected Lisp as an alternative to Tcl and Perl for scientific computing.