4 ms·

Great work! I am curious that how much effort it would take to support LoRAs with different ranks?

• zhye 3y ago
  It will take some effort to implement operators but not too much (cutlass's group gemm already support different mnk's), however the performance benefit is marginal compared to padding all LoRA ranks to the same rank because all these kernels are not compute bound.