3 ms·
I gave this a try and it seems to max out using about 12GB of VRAM on a RTX 3090 Ti.
by capableweb 4y ago
I gave this a try and it seems to max out using about 12GB of VRAM on a RTX 3090 Ti.
- capableweb 4y agoTried the 30b-hf set too, but was too much (24GB available). 13b-hf works fine, maxing out at 17GB.