3 ms·
To a comment that suggested I try the 70b model, I replied "my card can't run that model". Someone replies back with "you may as well throw the card out if you'
by LoganDark 3y ago
To a comment that suggested I try the 70b model, I replied "my card can't run that model". Someone replies back with "you may as well throw the card out if you're going to be trying to run that model". My point exactly.
More seriously, using all-CPU is not much faster as my computer only has 16GB of actual memory, which I'm aware is also hugely underspecced for a 70b model, even with memory mapping.
I have a nice NVMe SSD, so there's not much else for me to do here except upgrade my memory or graphics card.
- freedomben 3y agothat would make sense the downvotes, thank you!