4 ms·
Seems like 6B would still be useful if I want to run it on my GPU without exiting firefox.
by aidenn0 3y ago
Seems like 6B would still be useful if I want to run it on my GPU without exiting firefox.
- londons_explore 3y agoKinda lame that applications can't be told "yo, your gpu buffer has now been moved back to RAM".
- selcuka 3y agoWith Apple Silicon there is no separation between GPU RAM and CPU RAM, so the whole model can be loaded into RAM and processed by GPU (Metal).