7 ms·

Open-source drivers according to Habana

80 points by Aissen 5y ago

AshamedCaptain 5y ago
Kernel maintainers tend to refuse drivers that only work with proprietary user-space, so I guess this is just one way to workaround that.
- 10000truths 5y ago
  Or they could pull an NVidia, and dedicate a whole in-house kernel team to maintaining an out-of-tree kernel module.
  - my123 5y ago
    For NVIDIA? More than one. The Tegra stack uses a totally separate out-of-tree but GPLv2 kernel module (which also works on some dGPU SKUs). It's available at https://nv-tegra.nvidia.com/r/gitweb?p=linux-nvgpu.git;a=summary https://nv-tegra.nvidia.com/r/gitweb?p=linux-nvgpu.git;a=sum... And then there's the partially closed kernel module stack, which is a different code base...
- hansendc 5y ago
  It's not just drivers. It's really about ensuring that the folks that maintain the kernel have a way to test the code they maintain. The reasons that we (the kernel maintainers) have for this requirement are varied. But, for me, it's really nice to have at least one open source implementation that can test the kernel code. Without that, the kernel code can bit rot too easily. Even better is if an open source implementation is in the kernel tree, like in tools/testing/selftests. That makes it even less likely that the kernel code gets broken. Disclaimer: I work on Linux at Intel, although not on drivers like this Habana one.
- eikenberry 5y ago
  One of the points of having the drivers in kernel is that means they kernel can actually run on that hardware. In addition to allowing for testing as others have pointed out, it is also a way to make sure that drivers aren't used to restrict access to the hardware. It ensures the freedom of the platform.
j16sdiz 5y ago
Background story: https://lwn.net/Articles/867168/ https://lwn.net/Articles/867168/ TLDR: Intel want some huge DRI-related in linux kernel. The DRI maintainer insist there need at least one open user mode user. So we have this Proof-of-Concept driver
- drewg123 5y ago
  Thanks for this, without this I had no clue what the article was referring to. It would be nice if there was a fairly stable kernel API (or ABI) so drivers like this didn't have to be in the kernel. Out of tree drivers are a nightmare to maintain. I maintained some out-of-tree drivers for years at Myricom (version of myri10ge with valuable features nak'ed by netdev, MX HPC drivers). Doing this was a massive PITA. Pretty much every minor release brought with it some critical function changing the number of arguments, changing names, etc. RHEL updates were my own special version of hell, since their kernel X.Y.Z in no way resembled the upstream X.Y.Z It got so bad to support 2.6.9 through 3.x that the shim layer for the Linux driver was almost as big as the entire FreeBSD driver (where nobody cared that I implemented LRO).
  - 10000truths 5y ago
    This is by design. Linux doesn't want to pay the maintenance and performance costs of guaranteeing a stable in-kernel API/ABI: https://www.kernel.org/doc/Documentation/process/stable-api-nonsense.rst https://www.kernel.org/doc/Documentation/process/stable-api-...
    - charcircuit 5y ago
      You can make the same argument about user space compatibility. It's extra work and may prevent some improvements, but it's nice to have.
  - josephcsible 5y ago
    > It would be nice if there was a fairly stable kernel API (or ABI) so drivers like this didn't have to be in the kernel. Why? We want as many drivers as possible to be in the kernel.
naoqj 5y ago
...and then they'll complain that corporations would rather maintain their own forks of the kernel.
- Aissen 5y ago
  …and their customers will simply not buy their product if it needs a custom kernel.
yboris 5y ago
Do I understand this right? It seems like Habana has some super-efficient and fast ML hardware, but you can't just drop in an ML project and start using it? For example, only a subset of TensorFlow or PyTorch is supported? Is that right? If you want to use their hardware, you need to jump through some hoops? https://docs.habana.ai/en/latest/Tensorflow_User_Guide/Tensorflow_User_Guide.html https://docs.habana.ai/en/latest/Tensorflow_User_Guide/Tenso... https://docs.habana.ai/en/latest/PyTorch_User_Guide/PyTorch_User_Guide.html https://docs.habana.ai/en/latest/PyTorch_User_Guide/PyTorch_...
- j16sdiz 5y ago
  If you want to use that, you need the closed-source driver. The open-source driver is the minimal to make linux kernel developer happy.
  - yboris 5y ago
    But my point is that even if you use some closed-source driver or whatever is required to "set up" your project - you still can't re-use your regular code wholesale, but will have to modify to fit within whatever features they support. Right?
    - mirker 5y ago
      The API of Tensorflow and PyTorch is quite large. Even Tensorflow has a dialect specifically for TPU, so the notion of accelerator independence (especially when considering performance) has not been realized yet (though this is still a work in progress). Anyway, how could you reuse your code wholesale when one of the most common operations is calling “.cuda()” on a tensor?
    - my123 5y ago
      Unsupported layers will transparently fall back to CPU execution. You can then choose to implement those who can be as TPC kernels. It's fundamentally much less flexible than a GPU. Efficiency of Habana hardware isn't that great either, but that's another story... (they're still using TSMC 16nm in 2022 notably). Where Habana has an advantage in some workloads is cost.
sjmm1989 5y ago
Okay, I clearly don't understand everything going on in this one. Why? There must be something going on that explains why we are letting people get away with breaking the rules the rest of us are supposed to be follow. Must be. Why? Because if I understand anything at all about open source, it's that you don't get to post closed source as if it is open source; even if you try to resort to shenanigans and trickery to get around the rules. So my question is this. Why are we letting Intel get away with loopholing the rules the rest of us would have to follow? Seems to me the best thing to do here is punish them like the petulant child they are being. Erase their code, and tell them to politely fuck off. Or at least more politely than Linus Torvalds did with Nvidia. Also, I don't see why letting them put their code up is of any use in the first place if they are just going to do things like this to essentially break it. Like seriously folks, what in the ever living fuck? If it were up to me, I'd have people hacking them just to show them their place, and putting EVERYTHING up online for all to use. And that would be after doing something like sending evidence of wrong doings to the justice departments of every single nation who wants to take a chew out of them. (of which there are likely many) Why? Because the fact we allow companies like Microsoft and Intel to continuously get away with all their bullshit that they keep on trying to pull more bullshit. It's not rocket science folks. Time to apply the brake to their bullshitmobile and hard.
- Topgamer7 5y ago
  > If it were up to me, I'd have people hacking them Cut this garbage shit out. Hack on the driver implementation instead. > Because if I understand anything at all about open source, it's that you don't get to post closed source as if it is open source If it was upstreamed into the linux tree, then they have open sourced, with the appropriate license. So the driver is not fully functional, but what they have included could be the building blocks for someone to add the additional functionality. It is a shitty practice, and if you want to drive adoption, this isn't going to do it. At least they didn't make a marketing statement that they're open source purveyors and software saviors.
throw_a_grenade 5y ago
Oh, and that's what happens when there's no viable userspace: https://lore.kernel.org/lkml/CAFCwf10UrVGwGRqwPAfGUsyqqdp8Gxjo1nva3hf8dH8uZszPEg@mail.gmail.com/ https://lore.kernel.org/lkml/CAFCwf10UrVGwGRqwPAfGUsyqqdp8Gx...

80 points by Aissen 5y ago

Kernel maintainers tend to refuse drivers that only work with proprietary user-space, so I guess this is just one way to workaround that.

10000truths 5y ago
Or they could pull an NVidia, and dedicate a whole in-house kernel team to maintaining an out-of-tree kernel module.
- my123 5y ago
  For NVIDIA? More than one. The Tegra stack uses a totally separate out-of-tree but GPLv2 kernel module (which also works on some dGPU SKUs). It's available at https://nv-tegra.nvidia.com/r/gitweb?p=linux-nvgpu.git;a=summary https://nv-tegra.nvidia.com/r/gitweb?p=linux-nvgpu.git;a=sum... And then there's the partially closed kernel module stack, which is a different code base...
hansendc 5y ago
It's not just drivers. It's really about ensuring that the folks that maintain the kernel have a way to test the code they maintain. The reasons that we (the kernel maintainers) have for this requirement are varied. But, for me, it's really nice to have at least one open source implementation that can test the kernel code. Without that, the kernel code can bit rot too easily. Even better is if an open source implementation is in the kernel tree, like in tools/testing/selftests. That makes it even less likely that the kernel code gets broken. Disclaimer: I work on Linux at Intel, although not on drivers like this Habana one.
eikenberry 5y ago
One of the points of having the drivers in kernel is that means they kernel can actually run on that hardware. In addition to allowing for testing as others have pointed out, it is also a way to make sure that drivers aren't used to restrict access to the hardware. It ensures the freedom of the platform.

j16sdiz 5y ago

Background story: https://lwn.net/Articles/867168/ https://lwn.net/Articles/867168/ TLDR: Intel want some huge DRI-related in linux kernel. The DRI maintainer insist there need at least one open user mode user. So we have this Proof-of-Concept driver

drewg123 5y ago
Thanks for this, without this I had no clue what the article was referring to. It would be nice if there was a fairly stable kernel API (or ABI) so drivers like this didn't have to be in the kernel. Out of tree drivers are a nightmare to maintain. I maintained some out-of-tree drivers for years at Myricom (version of myri10ge with valuable features nak'ed by netdev, MX HPC drivers). Doing this was a massive PITA. Pretty much every minor release brought with it some critical function changing the number of arguments, changing names, etc. RHEL updates were my own special version of hell, since their kernel X.Y.Z in no way resembled the upstream X.Y.Z It got so bad to support 2.6.9 through 3.x that the shim layer for the Linux driver was almost as big as the entire FreeBSD driver (where nobody cared that I implemented LRO).
- 10000truths 5y ago
  This is by design. Linux doesn't want to pay the maintenance and performance costs of guaranteeing a stable in-kernel API/ABI: https://www.kernel.org/doc/Documentation/process/stable-api-nonsense.rst https://www.kernel.org/doc/Documentation/process/stable-api-...
  - charcircuit 5y ago
    You can make the same argument about user space compatibility. It's extra work and may prevent some improvements, but it's nice to have.
- josephcsible 5y ago
  > It would be nice if there was a fairly stable kernel API (or ABI) so drivers like this didn't have to be in the kernel. Why? We want as many drivers as possible to be in the kernel.

naoqj 5y ago

...and then they'll complain that corporations would rather maintain their own forks of the kernel.

Aissen 5y ago
…and their customers will simply not buy their product if it needs a custom kernel.

yboris 5y ago

Do I understand this right? It seems like Habana has some super-efficient and fast ML hardware, but you can't just drop in an ML project and start using it? For example, only a subset of TensorFlow or PyTorch is supported? Is that right? If you want to use their hardware, you need to jump through some hoops? https://docs.habana.ai/en/latest/Tensorflow_User_Guide/Tensorflow_User_Guide.html https://docs.habana.ai/en/latest/Tensorflow_User_Guide/Tenso... https://docs.habana.ai/en/latest/PyTorch_User_Guide/PyTorch_User_Guide.html https://docs.habana.ai/en/latest/PyTorch_User_Guide/PyTorch_...

j16sdiz 5y ago
If you want to use that, you need the closed-source driver. The open-source driver is the minimal to make linux kernel developer happy.
- yboris 5y ago
  But my point is that even if you use some closed-source driver or whatever is required to "set up" your project - you still can't re-use your regular code wholesale, but will have to modify to fit within whatever features they support. Right?
  - mirker 5y ago
    The API of Tensorflow and PyTorch is quite large. Even Tensorflow has a dialect specifically for TPU, so the notion of accelerator independence (especially when considering performance) has not been realized yet (though this is still a work in progress). Anyway, how could you reuse your code wholesale when one of the most common operations is calling “.cuda()” on a tensor?
  - my123 5y ago
    Unsupported layers will transparently fall back to CPU execution. You can then choose to implement those who can be as TPC kernels. It's fundamentally much less flexible than a GPU. Efficiency of Habana hardware isn't that great either, but that's another story... (they're still using TSMC 16nm in 2022 notably). Where Habana has an advantage in some workloads is cost.

sjmm1989 5y ago

Okay, I clearly don't understand everything going on in this one. Why? There must be something going on that explains why we are letting people get away with breaking the rules the rest of us are supposed to be follow. Must be. Why? Because if I understand anything at all about open source, it's that you don't get to post closed source as if it is open source; even if you try to resort to shenanigans and trickery to get around the rules. So my question is this. Why are we letting Intel get away with loopholing the rules the rest of us would have to follow? Seems to me the best thing to do here is punish them like the petulant child they are being. Erase their code, and tell them to politely fuck off. Or at least more politely than Linus Torvalds did with Nvidia. Also, I don't see why letting them put their code up is of any use in the first place if they are just going to do things like this to essentially break it. Like seriously folks, what in the ever living fuck? If it were up to me, I'd have people hacking them just to show them their place, and putting EVERYTHING up online for all to use. And that would be after doing something like sending evidence of wrong doings to the justice departments of every single nation who wants to take a chew out of them. (of which there are likely many) Why? Because the fact we allow companies like Microsoft and Intel to continuously get away with all their bullshit that they keep on trying to pull more bullshit. It's not rocket science folks. Time to apply the brake to their bullshitmobile and hard.

Topgamer7 5y ago
> If it were up to me, I'd have people hacking them Cut this garbage shit out. Hack on the driver implementation instead. > Because if I understand anything at all about open source, it's that you don't get to post closed source as if it is open source If it was upstreamed into the linux tree, then they have open sourced, with the appropriate license. So the driver is not fully functional, but what they have included could be the building blocks for someone to add the additional functionality. It is a shitty practice, and if you want to drive adoption, this isn't going to do it. At least they didn't make a marketing statement that they're open source purveyors and software saviors.

throw_a_grenade 5y ago

Oh, and that's what happens when there's no viable userspace: https://lore.kernel.org/lkml/CAFCwf10UrVGwGRqwPAfGUsyqqdp8Gxjo1nva3hf8dH8uZszPEg@mail.gmail.com/ https://lore.kernel.org/lkml/CAFCwf10UrVGwGRqwPAfGUsyqqdp8Gx...