4 ms·
This might be silly because of how few people it benefits, but could it be broken up on to multiple 8GB cards on the same system?
by joshspankit 3y ago
This might be silly because of how few people it benefits, but could it be broken up on to multiple 8GB cards on the same system?
- modeless 3y agoYes, it absolutely could. You're right that this configuration is rare. Although people have been putting together machines with multiple 24GB cards in order to split and run larger models like llama2-70B.
- wahnfrieden 3y agoThe latest large models are 120B and 100k context such as Goliath and Tess XL