5 ms·

I'm not a python expert, but this feels very odd to me (both the init construction and the return [tgemm.mm](http://tgemm.mm/ http://tgemm.mm/)(input, self.we

I'm not a python expert, but this feels very odd to me (both the *init* construction and the return [tgemm.mm](http://tgemm.mm/ http://tgemm.mm/)(input, self.weight, self.bias, None, None) call, which looks like markdown to me: from aiter.tuned_gemm import tgemm import torch class LinearLayer(torch.nn.Module): def **init**(self, in_features, out_features): super(LinearLayer, self).**init**() self.weight = torch.nn.Parameter(torch.randn(out_features, in_features).cuda()) self.bias = torch.nn.Parameter(torch.randn(out_features).cuda()) def forward(self, input): input = input.cuda() return [tgemm.mm](http://tgemm.mm/)(input, self.weight, self.bias, None, None)

evertedsphere 2y ago
yep the syntax highlighting / doc hyperlinking clearly broke there (or, less charitably, whatever llm produced that prose had a moment) it's __init__ of course
- deleted 2y ago
  [deleted]
fock 2y ago
also why is it calling .cuda() to move tensors to a cuda driver? I suppose this is because this is based on HIP - which comes with it's own set of problems, but that's ROCm for the masses I guess. Also the tgemm.mm has to be a torch module (at first I thought this was some lowlevel library which they now have a preview of, because there is a ROCm-torch already ...) which is evident from the table just before the summary. That table also smells like they are mostly focused on inference... EDIT: seems official ROCm-torch is also based on HIP.
Lerc 2y ago
I was puzzling over the code wondering why they .cuda() everything like that when I realised that that was only the beginning of the weirdness. I'm assuming the scrambled annotations were due to some odd chain of things the code went through on the way to becoming a post. Maybe they did it as a parable about the problems of having many layers of abstraction causing processes with unintended consequences?
cavisne 2y ago
Yeah this is AMD in a nutshell. A bunch of fluffy descriptions and then the only concrete example would clearly never run. EDIT: They fixed the code pretty quickly

I'm not a python expert, but this feels very odd to me (both the *init* construction and the return [tgemm.mm](http://tgemm.mm/ http://tgemm.mm/)(input, self.we

evertedsphere 2y ago

yep the syntax highlighting / doc hyperlinking clearly broke there (or, less charitably, whatever llm produced that prose had a moment) it's __init__ of course

deleted 2y ago
[deleted]

fock 2y ago

also why is it calling .cuda() to move tensors to a cuda driver? I suppose this is because this is based on HIP - which comes with it's own set of problems, but that's ROCm for the masses I guess. Also the tgemm.mm has to be a torch module (at first I thought this was some lowlevel library which they now have a preview of, because there is a ROCm-torch already ...) which is evident from the table just before the summary. That table also smells like they are mostly focused on inference... EDIT: seems official ROCm-torch is also based on HIP.

Lerc 2y ago

I was puzzling over the code wondering why they .cuda() everything like that when I realised that that was only the beginning of the weirdness. I'm assuming the scrambled annotations were due to some odd chain of things the code went through on the way to becoming a post. Maybe they did it as a parable about the problems of having many layers of abstraction causing processes with unintended consequences?

cavisne 2y ago

Yeah this is AMD in a nutshell. A bunch of fluffy descriptions and then the only concrete example would clearly never run. EDIT: They fixed the code pretty quickly