4 ms·
It sounds like there's forks that are able to work with <=8GB cards. And I'm not sure but I think the weights are using f32, so switching to half might make it
by wyldfire 4y ago
It sounds like there's forks that are able to work with <=8GB cards. And I'm not sure but I think the weights are using f32, so switching to half might make it yet easier still to get this to work w/less memory.
But yeah the next generation of models would probably capitalize on more memory somehow.
- filiphorvat 4y agoPeople have reported that this repo even works with 2gb cards if you run it with --lowvram and --opt-split-attention.
- password4321 4y agoYes, the amount of VRAM doesn't seem to be as much of a limitation anymore. However, processing power is still important.