3 ms·
Oh, nice to finally bump into someone who has experience with CSR! bucketMul has few uncoalesced reads, and it uses a different data structure than the regular
by kolinko 3y ago
Oh, nice to finally bump into someone who has experience with CSR!
bucketMul has few uncoalesced reads, and it uses a different data structure than the regular CSR - it's decribed here: https://kolinko.github.io/effort/bucketmul.html https://kolinko.github.io/effort/bucketmul.html It splits each Matrix row into 16 parts, and chooses which ones are necessary to read. The writes are fully linear.
Not sure if I speak sense though, it's getting a bit late today, and it's been a long day ;)
- marmaduke 3y agoYeah, nice, you could also consider a low rank factorization as well.
- kolinko 3y agoI looked into it at the beginning, but as far as I understand, the modern models like Mistral are difficult to do LoRA on - you can use it to finetune, but the model itself doesn't lend itself to such an operation. I'm still quite new to the field, so I'd appreciate some more insights into this, and a correction.