4 ms·

Do I understand this right? It seems like Habana has some super-efficient and fast ML hardware, but you can't just drop in an ML project and start using it? For

by yboris 5y ago

Do I understand this right? It seems like Habana has some super-efficient and fast ML hardware, but you can't just drop in an ML project and start using it? For example, only a subset of TensorFlow or PyTorch is supported? Is that right? If you want to use their hardware, you need to jump through some hoops? https://docs.habana.ai/en/latest/Tensorflow_User_Guide/Tensorflow_User_Guide.html https://docs.habana.ai/en/latest/Tensorflow_User_Guide/Tenso... https://docs.habana.ai/en/latest/PyTorch_User_Guide/PyTorch_User_Guide.html https://docs.habana.ai/en/latest/PyTorch_User_Guide/PyTorch_...

j16sdiz 5y ago
If you want to use that, you need the closed-source driver. The open-source driver is the minimal to make linux kernel developer happy.
- yboris 5y ago
  But my point is that even if you use some closed-source driver or whatever is required to "set up" your project - you still can't re-use your regular code wholesale, but will have to modify to fit within whatever features they support. Right?
  - mirker 5y ago
    The API of Tensorflow and PyTorch is quite large. Even Tensorflow has a dialect specifically for TPU, so the notion of accelerator independence (especially when considering performance) has not been realized yet (though this is still a work in progress). Anyway, how could you reuse your code wholesale when one of the most common operations is calling “.cuda()” on a tensor?
  - my123 5y ago
    Unsupported layers will transparently fall back to CPU execution. You can then choose to implement those who can be as TPC kernels. It's fundamentally much less flexible than a GPU. Efficiency of Habana hardware isn't that great either, but that's another story... (they're still using TSMC 16nm in 2022 notably). Where Habana has an advantage in some workloads is cost.

4 ms·

Do I understand this right? It seems like Habana has some super-efficient and fast ML hardware, but you can't just drop in an ML project and start using it? For

by yboris 5y ago

j16sdiz 5y ago
If you want to use that, you need the closed-source driver. The open-source driver is the minimal to make linux kernel developer happy.
- yboris 5y ago
  But my point is that even if you use some closed-source driver or whatever is required to "set up" your project - you still can't re-use your regular code wholesale, but will have to modify to fit within whatever features they support. Right?
  - mirker 5y ago
    The API of Tensorflow and PyTorch is quite large. Even Tensorflow has a dialect specifically for TPU, so the notion of accelerator independence (especially when considering performance) has not been realized yet (though this is still a work in progress). Anyway, how could you reuse your code wholesale when one of the most common operations is calling “.cuda()” on a tensor?
  - my123 5y ago
    Unsupported layers will transparently fall back to CPU execution. You can then choose to implement those who can be as TPC kernels. It's fundamentally much less flexible than a GPU. Efficiency of Habana hardware isn't that great either, but that's another story... (they're still using TSMC 16nm in 2022 notably). Where Habana has an advantage in some workloads is cost.