5 ms·

Shameless plug for Futhark[0], which you can use to write your computational kernels. The idea is to use functional constructs like map and reduce to express p

Shameless plug for Futhark[0], which you can use to write your computational kernels. The idea is to use functional constructs like map and reduce to express parallel code and let the compiler handle the generation of low-level OpenCL or CUDA. Disclaimer: I've recently finished a PhD focused on memory optimizations in Futhark. [0]: https://futhark-lang.org/ https://futhark-lang.org/ Edit: They do actually mention stuff like Julia and NumPy.

sspiff 3y ago
I used Futhark in the past. It's probably been 5 years or more, so a lot may have changed since then. It's absolutely a much lower barrier to entry, especially if you are familiar with functional programming. However, it also leaves a lot of performance on the table, and I found it to be hard or cumbersome to integrate with languages other than Python at the time. And you're still stuck with CUDA (which is single vendor) or OpenCL (which is a pain to set up on most consumer systems, and has very lackluster driver quality for a lot of vendors). I would have liked that by now support for Vulkan compute or OpenGL compute shaders would have been added, but alas.
- Munksgaard 3y ago
  Great to hear that you liked the language! Regarding the performance, I'd be interested to hear more about your experiences. Because we compile to CUDA or OpenCL in the end, we cannot claim to be faster than what you could (in principle) write in hand. However, most of our benchmarks compare favorably with handwritten reference implementations, and the heavily optimizing compiler is able to write code that is tough to write by hand. However, we're always looking for instances where we can do better. The goal is to be comparable to CUDA/OpenCL in as many cases as possible.
  - sspiff 3y ago
    I'd love to tell you what it did suboptimally in my usecase at the time, but I no longer have the code and it's been so long I don't remember the specifics. Like I said in my earlier comment, it was so long ago the performance remarks may be outdated and the project may have resolved them. I would like to give Futhark another try, but I need easy and portable integration in either Go or Rust, and as far as I can tell, that's not the case today? For my current project, I ended up selecting webgpu for my GPU compute needs.
    - fulafel 3y ago
      Go and Rust should be able to call Futhark via C, no? Link: https://futhark-book.readthedocs.io/en/latest/interoperability.html#calling-futhark-from-c https://futhark-book.readthedocs.io/en/latest/interoperabili...
  - Ambix 3y ago
    Is it possible to write llama.cpp in Futhark? Like do effective math manipulations on 4-bit vectors within GPU?
truckerbill 3y ago
What's your stance on WebGPU - from my understanding it could be perfect thing for Futhark to target, as it would allow for truly portable GPU code both hardware and platform-wise.
- Munksgaard 3y ago
  We actually have a long-standing issue regarding WebGPU[0]. Long story short, we want to support it, and Athas did some exploratory work a couple of years ago and decided that WebGPU wasn't mature enough yet. We already have a WebAssembly backend, so as soon as it is possible to access WebGPU from WebAssembly it should be relatively straightforward. [0]: https://github.com/diku-dk/futhark/issues/1403 https://github.com/diku-dk/futhark/issues/1403
  - skitter 3y ago
    Do you need to directly access it – isn't JS glue code (such as provided via Emscripten) enough?
    - Athas 3y ago
      I work on Futhark. We hope that WebGPU-through-Emscripten is viable, but I haven't yet had the spare cycles to make sure. I did find some examples that looked promising.
messe 3y ago
Do you know if there is any work being done to support a Metal backend for Futhark? That would be interesting for local prototyping on macOS. I've found a fork[1], but it doesn't seem to have been updated since February. [1]: https://github.com/MilesLitteral/futhark-metal https://github.com/MilesLitteral/futhark-metal
- Munksgaard 3y ago
  There is no on-going work to support Metal apart from the work done by Miles. There's an old issue about it: https://github.com/diku-dk/futhark/issues/853#issuecomment-586944330 https://github.com/diku-dk/futhark/issues/853#issuecomment-5...
  - messe 3y ago
    That’s a shame. I think since M1, macOS is very slowly starting to become a more popular platform for ML thanks to the unified memory. A mac is probably the cheapest and easiest way to get access to 64GB+ of memory available to a GPU on a local machine.
pzo 3y ago
I was always wondering how does it compare to halide or sycl or are those completely unrelated projects with different goals?
- Athas 3y ago
  Compared to Halide: * Futhark does not expose a scheduling language that gives you precise control over code generation. This is probably the main selling point of Halide. * Futhark has a much broader focus than Halide, which is mainly oriented towards image processing. Futhark wants to support arbitrary data parallel computation. E.g. see this compiler written in Futhark: https://github.com/Snektron/pareas https://github.com/Snektron/pareas Compared to Sycl: * Futhark is a non-embedded language that is more high level than Sycl. The goals are similar in the sense that both systems to try make (data) parallel programming more accessible. The vision behind Futhark is that the conventional functional programming vocabulary is actually a pretty good fit for parallelism, and that an aggressively optimising compiler can reduce or eliminate the overhead of abstraction. I don't think Sycl is as focused on high levels of abstraction, but rather focuses on being a relatively low-level portable programming interface.
pjmlp 3y ago
I love approaches like these, as they broaden the tooling to everyone instead of the usual C and C++ folks.

sspiff 3y ago

I used Futhark in the past. It's probably been 5 years or more, so a lot may have changed since then. It's absolutely a much lower barrier to entry, especially if you are familiar with functional programming. However, it also leaves a lot of performance on the table, and I found it to be hard or cumbersome to integrate with languages other than Python at the time. And you're still stuck with CUDA (which is single vendor) or OpenCL (which is a pain to set up on most consumer systems, and has very lackluster driver quality for a lot of vendors). I would have liked that by now support for Vulkan compute or OpenGL compute shaders would have been added, but alas.

Munksgaard 3y ago
Great to hear that you liked the language! Regarding the performance, I'd be interested to hear more about your experiences. Because we compile to CUDA or OpenCL in the end, we cannot claim to be faster than what you could (in principle) write in hand. However, most of our benchmarks compare favorably with handwritten reference implementations, and the heavily optimizing compiler is able to write code that is tough to write by hand. However, we're always looking for instances where we can do better. The goal is to be comparable to CUDA/OpenCL in as many cases as possible.
- sspiff 3y ago
  I'd love to tell you what it did suboptimally in my usecase at the time, but I no longer have the code and it's been so long I don't remember the specifics. Like I said in my earlier comment, it was so long ago the performance remarks may be outdated and the project may have resolved them. I would like to give Futhark another try, but I need easy and portable integration in either Go or Rust, and as far as I can tell, that's not the case today? For my current project, I ended up selecting webgpu for my GPU compute needs.
  - fulafel 3y ago
    Go and Rust should be able to call Futhark via C, no? Link: https://futhark-book.readthedocs.io/en/latest/interoperability.html#calling-futhark-from-c https://futhark-book.readthedocs.io/en/latest/interoperabili...
- Ambix 3y ago
  Is it possible to write llama.cpp in Futhark? Like do effective math manipulations on 4-bit vectors within GPU?

truckerbill 3y ago

What's your stance on WebGPU - from my understanding it could be perfect thing for Futhark to target, as it would allow for truly portable GPU code both hardware and platform-wise.

Munksgaard 3y ago
We actually have a long-standing issue regarding WebGPU[0]. Long story short, we want to support it, and Athas did some exploratory work a couple of years ago and decided that WebGPU wasn't mature enough yet. We already have a WebAssembly backend, so as soon as it is possible to access WebGPU from WebAssembly it should be relatively straightforward. [0]: https://github.com/diku-dk/futhark/issues/1403 https://github.com/diku-dk/futhark/issues/1403
- skitter 3y ago
  Do you need to directly access it – isn't JS glue code (such as provided via Emscripten) enough?
  - Athas 3y ago
    I work on Futhark. We hope that WebGPU-through-Emscripten is viable, but I haven't yet had the spare cycles to make sure. I did find some examples that looked promising.

messe 3y ago

Do you know if there is any work being done to support a Metal backend for Futhark? That would be interesting for local prototyping on macOS. I've found a fork[1], but it doesn't seem to have been updated since February. [1]: https://github.com/MilesLitteral/futhark-metal https://github.com/MilesLitteral/futhark-metal

Munksgaard 3y ago
There is no on-going work to support Metal apart from the work done by Miles. There's an old issue about it: https://github.com/diku-dk/futhark/issues/853#issuecomment-586944330 https://github.com/diku-dk/futhark/issues/853#issuecomment-5...
- messe 3y ago
  That’s a shame. I think since M1, macOS is very slowly starting to become a more popular platform for ML thanks to the unified memory. A mac is probably the cheapest and easiest way to get access to 64GB+ of memory available to a GPU on a local machine.

pzo 3y ago

I was always wondering how does it compare to halide or sycl or are those completely unrelated projects with different goals?

Athas 3y ago
Compared to Halide: * Futhark does not expose a scheduling language that gives you precise control over code generation. This is probably the main selling point of Halide. * Futhark has a much broader focus than Halide, which is mainly oriented towards image processing. Futhark wants to support arbitrary data parallel computation. E.g. see this compiler written in Futhark: https://github.com/Snektron/pareas https://github.com/Snektron/pareas Compared to Sycl: * Futhark is a non-embedded language that is more high level than Sycl. The goals are similar in the sense that both systems to try make (data) parallel programming more accessible. The vision behind Futhark is that the conventional functional programming vocabulary is actually a pretty good fit for parallelism, and that an aggressively optimising compiler can reduce or eliminate the overhead of abstraction. I don't think Sycl is as focused on high levels of abstraction, but rather focuses on being a relatively low-level portable programming interface.

pjmlp 3y ago

I love approaches like these, as they broaden the tooling to everyone instead of the usual C and C++ folks.