2 ms·
I'm sitting here with "only" 12GB of VRAM and I can't train even the bigger RWKV models, and that's a freaking RNN, it's not even a transformer! And that's high
by LoganDark 3y ago
I'm sitting here with "only" 12GB of VRAM and I can't train even the bigger RWKV models, and that's a freaking RNN, it's not even a transformer! And that's highly disappointing for someone who's been working as a contributor to one of the inference libraries, because the fruits of my labor aren't even going to benefit me that much. Bleh~
I (somewhat) lie, all my experience working on rwkv.cpp does gain me knowledge and also contribute to the community. I'm just sort of sad that I don't have enough VRAM to finetune a bigger model to fit something like a chat format, or learn a certain vocabulary or whatever.