6 ms·

Tinygrad and rusticl and aco: why not?

46 points by pantalaimon 3y ago

rhn_mk1 3y ago
Rusticl seems a bit hidden for all the (positive) noise it creates. I'd like to try it out, but: - are there any distributions shipping it? - what hardware does it support? I can guess that distros are not shipping it due to some stability concerns (which I could not see so far). I'm also guessing that despite being a Gallium driver, some features will depend on the capabilities of the underlying hardware. Can I hope that my AMD APU will finally get some OpenCL support?
- pantalaimon 3y ago
  > what hardware does it support? That’s the nice thing about it, it supports most hardware supported by Mesa going back even to r600 GPUs [0]. So anything Intel or AMD should work. My guess for as to why distributions are not enabling it yet might be due to the Rust build dependency that might require a newer Rust version than what’s provided by the distribution. [0] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20252 https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20...
- brucethemoose2 3y ago
  > Can I hope that my AMD APU will finally get some OpenCL support? What APU is it? OpenCL support goes back pretty far.
  - rhn_mk1 3y ago
    I haven't found anything for the Vega 8 that would run on Linux.
    - brucethemoose2 3y ago
      ? I have had OpenCL on my 4900HS for awhile, just through the Mesa OpenCL driver. I believe my really old Llano (2011) APU even had OpenCL support. It was a selling point at the time.
      - sspiff 3y ago
        Same, it just works on my Debian powered home server with 5800H with just the Mesa ICD installed. Not that it really matters for me, I only run wgpu compute stuff over Vulkan for my experiments.
      - rhn_mk1 3y ago
        Hm, now I have to start asking questions of my OS. For what it's worth, my Llano kept hanging up when I tried.
        brucethemoose2 3y ago
        You are probsbly just missing the driver, some linux distros dont ship it by default. If you are on Windows... shrug.
imtringued 3y ago
No mention what ACO is. The benchmark is just a dump of the console output with no explanation what the individual phases mean. If I am going to be honest, this is a terrible blog post.
- mathisfun123 3y ago
  > No mention what ACO is. Griping about initialisms has gotta be hn's favorite pass-time by far. It's as if we don't live in a world where things can be looked up with the flick of wrist. It's as if we're not discussing this on a website entirely premised on making information access/retrieval more efficient. > Mesa ACO compiler More than enough disambiguation that the first hit tells me exactly what it is. > If I am going to be honest, this is a terrible blog post. Actually, I was wrong: cheap disparagement is hn's favorite pass-time.
- sva_ 3y ago
  AMD COmpiler
jchw 3y ago
> large GPU stacks usually make me want to wipe the machine after I test them Core value proposition for NixOS/GuixSD right here. That said, the amount of time you have to dedicate to figuring out how to make everything work well for your use cases might negate the benefits a bit :)
- fbdab103 3y ago
  Cannot vouch for it, but the nixified.ai project is precisely geared towards this idea. Having Nix builds to run machine learning environments[0] [0] https://nixified.ai/ https://nixified.ai/
- colordrops 3y ago
  Once you figure it out you never have to figure it out again. And there is likely someone else who already figured out something as common as this so you could just import their config.
  - jchw 3y ago
    I love NixOS, but if you're a heavy Linux user the odds that you do something fairly unique in your personal setup gets very high. For me, to give you an example of just one little microcosm, I have probably the only combination of SwayWM, gnome-keyring and NixOS in the world that works the exact way mine does, and it took quite a while to land on it (and sometimes time to maintain it.) It's actually pretty simple and elegant now, but getting to where I am today felt anything but simple. Back when I first started (5 years ago,) drivers for my label printer were not available, and I stumbled around trying to figure out how to make a derivation for them. Ultimately I succeeded, but for a n00b it did feel pretty ridiculously hard to try to figure it out while not even having much of a background in pure FP languages less the oddity that is Nix. I never got around to adding it to Nixpkgs, but thankfully somebody else did. There is indeed solutions for a LOT of things. Is home manager worth the time? Not sure, I just use global configs instead of home dir configs for most things. There's multiple solutions for secrets but I have a few different Nix configuration repositories (for different sets of machines, not one for each machine; I only do this when it feels like it makes sense, like to keep servers for some org separate from my home machines) and not all of them have SOPS setup because while it works, bootstrapping it always has me guessing a little to make sure the way I'm setting it up actually makes sense. Flakes. They're good, but I started before everyone recommended them for everything. The switching cost of flakes was not huge, but nor was it trivial, as I had to adjust a lot of workflows for it. This kind of negates the "you never have to figure it out again" thing. On the contrary, NixOS is a moving and vastly complicated machine, and it indeed feels like things need to be re-figured out quite often. Rust dependency management in Nixpkgs is sure to change in the future: right now it can't handle all of the situations that Cargo can and often requires including a copy of the lockfile in the Nixpkgs tree which seems unideal. Worth it still? Yes, because I can throw my boot drive out the window and have a machine back up and running in like 10 minutes. All of my desktop machines are very consistent, because any configuration that matters to me goes into declarative and versioned configuration. The concept of "hosing" my OS doesn't exist anymore, and even if a workflow does break, I can trivially bisect it to figure out why. Nix tools are super complicated, but they are super powerful too. NixOS is Linux with superpowers. All of the cool stuff that open source could theoretically enable you to do, NixOS makes practical. But is it perfect? No. To this day, you will not find a derivation for plenty of very useful closed source software. No VMWare Workstation, IDA Pro, etc. AppImage binaries that run other AppImages don't work with appimage-run; though I have a workaround with Nix-LD that does work, I've never seen anyone else doing it, so I suspect not too many people do this. Most open source software is there, but it can be a bit hit or miss at times. You'll occasionally find places where missing dependencies or hardcoded paths in the upstream sources lead to subtly broken or suboptimal things. So if you only manage one machine with NixOS or you don't have a lot of time to invest upfront, or you have some use cases that are pathologically difficult to deal with, it's not always practical for everyone. I think if NixOS immediately sells someone, they're in. They'll get it. If not? Maybe in a couple more years. For me, it was on first sight.
brucethemoose2 3y ago
I have griped about this before, but I don't really get the hype around tinygrad... GitHub is littered with ML frameworks, some of them (Like MLIR/TVM) with very good working demos right now, and other smaller scale ones (like GGML) are SOTA in very specific niches.
- pseg134 3y ago
  And did you gain any knowledge the last time you “griped” about it? If not then who are these “gripe” sessions even serving?
  - brucethemoose2 3y ago
    > And did you gain any knowledge ...No. I am not against tinygrad or anything, but I can't find any projects using it as a backend. And I can't find benchmarks for, say, the stable diffusion or llama demos. But I am frustrated that it seems to be attracting lots of attention on social media, while other ML frameworks/projects targeting wide hardware support, and already being used as backends in places, are flying under the radar. And Hotz's very brief spat with AMD that I saw on his YouTube channel also raised my eyebrows.
    - vegarab 3y ago
      comma.ai uses Tinygrad as its backend
    - tinco 3y ago
      What was his spat with AMD about? I think he complained about AMD's drivers being shit on the Lex Fridman podcast but that's not eye brow raising right?
      - brucethemoose2 3y ago
        Because of unfixed rocm bugs, Hotz stated that Tinygrad was going to abandon AMD support on GitHub, and was very upset about it in the following YouTube video. This is when Lisa Su apparently stepped in and patched things up.
sitkack 3y ago
XDC 2022 | Rusticl: An OpenCL implementation written in Rust | Karol Herbst https://www.youtube.com/watch?v=Es7UmvNAEvk https://www.youtube.com/watch?v=Es7UmvNAEvk

46 points by pantalaimon 3y ago

Rusticl seems a bit hidden for all the (positive) noise it creates. I'd like to try it out, but: - are there any distributions shipping it? - what hardware does it support? I can guess that distros are not shipping it due to some stability concerns (which I could not see so far). I'm also guessing that despite being a Gallium driver, some features will depend on the capabilities of the underlying hardware. Can I hope that my AMD APU will finally get some OpenCL support?

pantalaimon 3y ago
> what hardware does it support? That’s the nice thing about it, it supports most hardware supported by Mesa going back even to r600 GPUs [0]. So anything Intel or AMD should work. My guess for as to why distributions are not enabling it yet might be due to the Rust build dependency that might require a newer Rust version than what’s provided by the distribution. [0] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20252 https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20...
brucethemoose2 3y ago
> Can I hope that my AMD APU will finally get some OpenCL support? What APU is it? OpenCL support goes back pretty far.
- rhn_mk1 3y ago
  I haven't found anything for the Vega 8 that would run on Linux.
  - brucethemoose2 3y ago
    ? I have had OpenCL on my 4900HS for awhile, just through the Mesa OpenCL driver. I believe my really old Llano (2011) APU even had OpenCL support. It was a selling point at the time.
    - sspiff 3y ago
      Same, it just works on my Debian powered home server with 5800H with just the Mesa ICD installed. Not that it really matters for me, I only run wgpu compute stuff over Vulkan for my experiments.
    - rhn_mk1 3y ago
      Hm, now I have to start asking questions of my OS. For what it's worth, my Llano kept hanging up when I tried.
      - brucethemoose2 3y ago
        You are probsbly just missing the driver, some linux distros dont ship it by default. If you are on Windows... shrug.

imtringued 3y ago

No mention what ACO is. The benchmark is just a dump of the console output with no explanation what the individual phases mean. If I am going to be honest, this is a terrible blog post.

mathisfun123 3y ago
> No mention what ACO is. Griping about initialisms has gotta be hn's favorite pass-time by far. It's as if we don't live in a world where things can be looked up with the flick of wrist. It's as if we're not discussing this on a website entirely premised on making information access/retrieval more efficient. > Mesa ACO compiler More than enough disambiguation that the first hit tells me exactly what it is. > If I am going to be honest, this is a terrible blog post. Actually, I was wrong: cheap disparagement is hn's favorite pass-time.
sva_ 3y ago
AMD COmpiler

jchw 3y ago

> large GPU stacks usually make me want to wipe the machine after I test them Core value proposition for NixOS/GuixSD right here. That said, the amount of time you have to dedicate to figuring out how to make everything work well for your use cases might negate the benefits a bit :)

fbdab103 3y ago
Cannot vouch for it, but the nixified.ai project is precisely geared towards this idea. Having Nix builds to run machine learning environments[0] [0] https://nixified.ai/ https://nixified.ai/
colordrops 3y ago
Once you figure it out you never have to figure it out again. And there is likely someone else who already figured out something as common as this so you could just import their config.
- jchw 3y ago
  I love NixOS, but if you're a heavy Linux user the odds that you do something fairly unique in your personal setup gets very high. For me, to give you an example of just one little microcosm, I have probably the only combination of SwayWM, gnome-keyring and NixOS in the world that works the exact way mine does, and it took quite a while to land on it (and sometimes time to maintain it.) It's actually pretty simple and elegant now, but getting to where I am today felt anything but simple. Back when I first started (5 years ago,) drivers for my label printer were not available, and I stumbled around trying to figure out how to make a derivation for them. Ultimately I succeeded, but for a n00b it did feel pretty ridiculously hard to try to figure it out while not even having much of a background in pure FP languages less the oddity that is Nix. I never got around to adding it to Nixpkgs, but thankfully somebody else did. There is indeed solutions for a LOT of things. Is home manager worth the time? Not sure, I just use global configs instead of home dir configs for most things. There's multiple solutions for secrets but I have a few different Nix configuration repositories (for different sets of machines, not one for each machine; I only do this when it feels like it makes sense, like to keep servers for some org separate from my home machines) and not all of them have SOPS setup because while it works, bootstrapping it always has me guessing a little to make sure the way I'm setting it up actually makes sense. Flakes. They're good, but I started before everyone recommended them for everything. The switching cost of flakes was not huge, but nor was it trivial, as I had to adjust a lot of workflows for it. This kind of negates the "you never have to figure it out again" thing. On the contrary, NixOS is a moving and vastly complicated machine, and it indeed feels like things need to be re-figured out quite often. Rust dependency management in Nixpkgs is sure to change in the future: right now it can't handle all of the situations that Cargo can and often requires including a copy of the lockfile in the Nixpkgs tree which seems unideal. Worth it still? Yes, because I can throw my boot drive out the window and have a machine back up and running in like 10 minutes. All of my desktop machines are very consistent, because any configuration that matters to me goes into declarative and versioned configuration. The concept of "hosing" my OS doesn't exist anymore, and even if a workflow does break, I can trivially bisect it to figure out why. Nix tools are super complicated, but they are super powerful too. NixOS is Linux with superpowers. All of the cool stuff that open source could theoretically enable you to do, NixOS makes practical. But is it perfect? No. To this day, you will not find a derivation for plenty of very useful closed source software. No VMWare Workstation, IDA Pro, etc. AppImage binaries that run other AppImages don't work with appimage-run; though I have a workaround with Nix-LD that does work, I've never seen anyone else doing it, so I suspect not too many people do this. Most open source software is there, but it can be a bit hit or miss at times. You'll occasionally find places where missing dependencies or hardcoded paths in the upstream sources lead to subtly broken or suboptimal things. So if you only manage one machine with NixOS or you don't have a lot of time to invest upfront, or you have some use cases that are pathologically difficult to deal with, it's not always practical for everyone. I think if NixOS immediately sells someone, they're in. They'll get it. If not? Maybe in a couple more years. For me, it was on first sight.

brucethemoose2 3y ago

I have griped about this before, but I don't really get the hype around tinygrad... GitHub is littered with ML frameworks, some of them (Like MLIR/TVM) with very good working demos right now, and other smaller scale ones (like GGML) are SOTA in very specific niches.

pseg134 3y ago
And did you gain any knowledge the last time you “griped” about it? If not then who are these “gripe” sessions even serving?
- brucethemoose2 3y ago
  > And did you gain any knowledge ...No. I am not against tinygrad or anything, but I can't find any projects using it as a backend. And I can't find benchmarks for, say, the stable diffusion or llama demos. But I am frustrated that it seems to be attracting lots of attention on social media, while other ML frameworks/projects targeting wide hardware support, and already being used as backends in places, are flying under the radar. And Hotz's very brief spat with AMD that I saw on his YouTube channel also raised my eyebrows.
  - vegarab 3y ago
    comma.ai uses Tinygrad as its backend
  - tinco 3y ago
    What was his spat with AMD about? I think he complained about AMD's drivers being shit on the Lex Fridman podcast but that's not eye brow raising right?
    - brucethemoose2 3y ago
      Because of unfixed rocm bugs, Hotz stated that Tinygrad was going to abandon AMD support on GitHub, and was very upset about it in the following YouTube video. This is when Lisa Su apparently stepped in and patched things up.

sitkack 3y ago

XDC 2022 | Rusticl: An OpenCL implementation written in Rust | Karol Herbst https://www.youtube.com/watch?v=Es7UmvNAEvk https://www.youtube.com/watch?v=Es7UmvNAEvk