4 ms·

Classing for example ISPC as low level invalidates this comment for me; ISPC among others provides a great interface to SIMD model for CPUs that simply isn’t av

Classing for example ISPC as low level invalidates this comment for me; ISPC among others provides a great interface to SIMD model for CPUs that simply isn’t available in Julia.

dragontamer 6y ago
ISPC is higher level than its peers, but its still new/delete based manual memory allocation. So in the great scheme of programming languages, its still rather low level (since you're manually handling memory). Indeed: ISPC provides constructs to provide structure-of-arrays, and other low-level memory layout details. This is a good thing: these details have significant implications on the speed of your program. Nonetheless, any language which wrangles with manual details of memory layout, or new/delete based memory allocation, is inevitably going to be classified as low level in my books. > ISPC among others provides a great interface to SIMD model for CPUs that simply isn’t available in Julia. If Julia can compile into GPU-assembly (which is innately SIMD), I'm sure an AVX-based compile could work eventually. They may have to target AVX512 (since most GPU-assembly requires per-thread exec-masks), but the general concept is being proven as Julia can now compile down into PTX or AMDGPU assembly. Julia's compile down to GPU-assembly / SIMD code is not supported in the "general case", only in select circumstances. But that's still an incredible boon for a high-level language.
- marmaduke 6y ago
  > I'm sure an AVX-based compile could work eventually. Eventually is nice but it’s also easy to imagine that it never gets done. ISPC works today.
  - dragontamer 6y ago
    The CUDA portion of Julia works today, and you can get 10Tflop or 20Tflop GPUs under $1000 to play with it. It's a different language for a different computer. But the fundamentals are still there, laying the groundwork for the future. Sure, ispc can win on latency. But the raw compute girth of GPU SIMD cannot be underestimated. In any case, Julia has proven itself capable of compiling down to a SIMD instruction set and achieving nearly the full performance of those GPUs. Even if it's not a computer you prefer, the programming model and technology demonstrated here is clear.
    - marmaduke 6y ago
      I agree with this and have the hardware but simply prefer to write CUDA kernels directly instead of in Julia syntax. My comment was more geared to the CPU side of things since I write code whose users don’t usually have GPUs available. There are also problem sizes which don’t fit the GPU’s more stark memory system separation: tens of CPU cores with tens of MBs of cache can move past GPUs in terms of memory bandwidth for such cases so banking on CUDA just doesn’t work for everyone.
- marmaduke 6y ago
  > but its still new/delete based manual memory allocation I guess we aren’t solving the same problems: memory allocation is trivial is my domain, mapping complex nested control flow in SIMD is the hard part. > Julia can now compile down into PTX or AMDGPU assembly. Sure but Julia-as-syntax is nothing special now, Numba does this for a Python as well.
  - dnautics 6y ago
    you can and you can't. Julia is composable -- Suppose I want to write a library to find compression polynomials for reed-solomon encoding system (see mary wootters' talks on youtube) for my storage product. I need a LU decomposition algorithm that operates on GF256 (which is just an 8-bit int). But the +/-/x/divide operations are all messed up. I'd have to rewrite LU decomposition. How confident are you that you can get even that right? I'm pretty good at implementing algorithms, but I've messed up LU decomposition before. Then suppose I rewrite the LU decomposition algorithm, using python. Now I want to accelerate the search by running the search on GPUs. I have to re-rewrite the code from scratch. Each GF256 encoding has to have rejiggered operators, and so I need to rewrite custom GPU kernels, then figure out how to resequence the operations (* looks different for each GF256 encoding), etc. This is all SUPER easy in julia.
    - marmaduke 6y ago
      Then use Julia, that's great. My comment was that the SIMD support in Julia isn't sufficient for my problem, not that you can't do your GF256 linear alg on a GPU..?
    - fnord123 6y ago
      > (see mary wootters' talks on youtube) https://www.youtube.com/watch?v=Gh578e98qAk https://www.youtube.com/watch?v=Gh578e98qAk > Suppose I want to write a library to find compression polynomials for reed-solomon encoding system (see mary wootters' talks on youtube) for my storage product. I need a LU decomposition algorithm that operates on GF256 Surely you use isa-l[1]. [1] https://github.com/intel/isa-l https://github.com/intel/isa-l > Now I want to accelerate the search by running the search on GPUs. GPUs are float oriented so I don't think you'll get the performance you hope for out of 8 bit integer operations. If you have interesting results to share I'd like to read them.
staticfloat 6y ago
Julia's SIMD programming model is still very much a work in progress; I think we have a way to go in providing the kind of flexibility and control that languages such as ISPC, Halide, TVM, etc... provide. That being said, packages such as SIMD.jl [0], and LoopVectorization.jl [1] are making fantastic progress, to the point that LoopVectorization forms the basis of a legitimate BLAS contender, in pure Julia [2]. It's not totally there yet, but it's close enough that real work is being done in LV at OpenBLAS-like speeds. As an aside, I find it incredible that these kinds of extensions can be built in packages thanks to the fact that Julia's compiler is extensible enough to allow for direct manipulation of the LLVM intrinsics being emitted by user code. [0] https://github.com/eschnett/SIMD.jl https://github.com/eschnett/SIMD.jl [1] https://github.com/chriselrod/LoopVectorization.jl https://github.com/chriselrod/LoopVectorization.jl [2] https://github.com/MasonProtter/Gaius.jl https://github.com/MasonProtter/Gaius.jl
- marmaduke 6y ago
  It’s not a jab at Julia, really rather that ISPC provides a workable model that would later on be nice to see elsewhere. > find it incredible that these kinds of extensions can be built in packages thanks to the fact that Julia's compiler is extensible Come on, jeez.. Julia’s compiler is a Lisp-based LLVM driver: of course it can do these things.
  - staticfloat 6y ago
    > It’s not a jab at Julia I didn't take it as such; there are legitimate shortcomings to any tool, I just wanted to provide pointers to other readers that the devs are aware of it, and that there is ongoing development to address it. :) > Come on, jeez.. Julia’s compiler is a Lisp-based LLVM driver: of course it can do these things. As someone who, before Julia, was firmly entrenched in C/C++/Python land, I suppose I am discovering many of these "obvious" things for the first time. :)
    - marmaduke 6y ago
      > I suppose I am discovering many of these "obvious" things sorry for the flippant remark then! it's great to be in discover mode, enjoy ;)

Classing for example ISPC as low level invalidates this comment for me; ISPC among others provides a great interface to SIMD model for CPUs that simply isn’t available in Julia.

dragontamer 6y ago

ISPC is higher level than its peers, but its still new/delete based manual memory allocation. So in the great scheme of programming languages, its still rather low level (since you're manually handling memory). Indeed: ISPC provides constructs to provide structure-of-arrays, and other low-level memory layout details. This is a good thing: these details have significant implications on the speed of your program. Nonetheless, any language which wrangles with manual details of memory layout, or new/delete based memory allocation, is inevitably going to be classified as low level in my books. > ISPC among others provides a great interface to SIMD model for CPUs that simply isn’t available in Julia. If Julia can compile into GPU-assembly (which is innately SIMD), I'm sure an AVX-based compile could work eventually. They may have to target AVX512 (since most GPU-assembly requires per-thread exec-masks), but the general concept is being proven as Julia can now compile down into PTX or AMDGPU assembly. Julia's compile down to GPU-assembly / SIMD code is not supported in the "general case", only in select circumstances. But that's still an incredible boon for a high-level language.

marmaduke 6y ago
> I'm sure an AVX-based compile could work eventually. Eventually is nice but it’s also easy to imagine that it never gets done. ISPC works today.
- dragontamer 6y ago
  The CUDA portion of Julia works today, and you can get 10Tflop or 20Tflop GPUs under $1000 to play with it. It's a different language for a different computer. But the fundamentals are still there, laying the groundwork for the future. Sure, ispc can win on latency. But the raw compute girth of GPU SIMD cannot be underestimated. In any case, Julia has proven itself capable of compiling down to a SIMD instruction set and achieving nearly the full performance of those GPUs. Even if it's not a computer you prefer, the programming model and technology demonstrated here is clear.
  - marmaduke 6y ago
    I agree with this and have the hardware but simply prefer to write CUDA kernels directly instead of in Julia syntax. My comment was more geared to the CPU side of things since I write code whose users don’t usually have GPUs available. There are also problem sizes which don’t fit the GPU’s more stark memory system separation: tens of CPU cores with tens of MBs of cache can move past GPUs in terms of memory bandwidth for such cases so banking on CUDA just doesn’t work for everyone.
marmaduke 6y ago
> but its still new/delete based manual memory allocation I guess we aren’t solving the same problems: memory allocation is trivial is my domain, mapping complex nested control flow in SIMD is the hard part. > Julia can now compile down into PTX or AMDGPU assembly. Sure but Julia-as-syntax is nothing special now, Numba does this for a Python as well.
- dnautics 6y ago
  you can and you can't. Julia is composable -- Suppose I want to write a library to find compression polynomials for reed-solomon encoding system (see mary wootters' talks on youtube) for my storage product. I need a LU decomposition algorithm that operates on GF256 (which is just an 8-bit int). But the +/-/x/divide operations are all messed up. I'd have to rewrite LU decomposition. How confident are you that you can get even that right? I'm pretty good at implementing algorithms, but I've messed up LU decomposition before. Then suppose I rewrite the LU decomposition algorithm, using python. Now I want to accelerate the search by running the search on GPUs. I have to re-rewrite the code from scratch. Each GF256 encoding has to have rejiggered operators, and so I need to rewrite custom GPU kernels, then figure out how to resequence the operations (* looks different for each GF256 encoding), etc. This is all SUPER easy in julia.
  - marmaduke 6y ago
    Then use Julia, that's great. My comment was that the SIMD support in Julia isn't sufficient for my problem, not that you can't do your GF256 linear alg on a GPU..?
  - fnord123 6y ago
    > (see mary wootters' talks on youtube) https://www.youtube.com/watch?v=Gh578e98qAk https://www.youtube.com/watch?v=Gh578e98qAk > Suppose I want to write a library to find compression polynomials for reed-solomon encoding system (see mary wootters' talks on youtube) for my storage product. I need a LU decomposition algorithm that operates on GF256 Surely you use isa-l[1]. [1] https://github.com/intel/isa-l https://github.com/intel/isa-l > Now I want to accelerate the search by running the search on GPUs. GPUs are float oriented so I don't think you'll get the performance you hope for out of 8 bit integer operations. If you have interesting results to share I'd like to read them.

staticfloat 6y ago

Julia's SIMD programming model is still very much a work in progress; I think we have a way to go in providing the kind of flexibility and control that languages such as ISPC, Halide, TVM, etc... provide. That being said, packages such as SIMD.jl [0], and LoopVectorization.jl [1] are making fantastic progress, to the point that LoopVectorization forms the basis of a legitimate BLAS contender, in pure Julia [2]. It's not totally there yet, but it's close enough that real work is being done in LV at OpenBLAS-like speeds. As an aside, I find it incredible that these kinds of extensions can be built in packages thanks to the fact that Julia's compiler is extensible enough to allow for direct manipulation of the LLVM intrinsics being emitted by user code. [0] https://github.com/eschnett/SIMD.jl https://github.com/eschnett/SIMD.jl [1] https://github.com/chriselrod/LoopVectorization.jl https://github.com/chriselrod/LoopVectorization.jl [2] https://github.com/MasonProtter/Gaius.jl https://github.com/MasonProtter/Gaius.jl

marmaduke 6y ago
It’s not a jab at Julia, really rather that ISPC provides a workable model that would later on be nice to see elsewhere. > find it incredible that these kinds of extensions can be built in packages thanks to the fact that Julia's compiler is extensible Come on, jeez.. Julia’s compiler is a Lisp-based LLVM driver: of course it can do these things.
- staticfloat 6y ago
  > It’s not a jab at Julia I didn't take it as such; there are legitimate shortcomings to any tool, I just wanted to provide pointers to other readers that the devs are aware of it, and that there is ongoing development to address it. :) > Come on, jeez.. Julia’s compiler is a Lisp-based LLVM driver: of course it can do these things. As someone who, before Julia, was firmly entrenched in C/C++/Python land, I suppose I am discovering many of these "obvious" things for the first time. :)
  - marmaduke 6y ago
    > I suppose I am discovering many of these "obvious" things sorry for the flippant remark then! it's great to be in discover mode, enjoy ;)