3 ms·
With only 8GB of VRAM, it can't be that good ...
by amelius 3y ago
With only 8GB of VRAM, it can't be that good ...
- speed_spread 3y agoNewer models such as Phi2 run comfortably with 4GB and are good enough to be useful for casual interaction. Sticking with local inference, multiple small models tuned for specific usage scenarios is where it's at.