2 ms·
the 7b distilled version works great on my laptop cpu and iGPU with vulkan. You can use llama.cpp (for iGPU with vulkan) or ollama (for cpu). My laptop is a c
by buyucu 2y ago
the 7b distilled version works great on my laptop cpu and iGPU with vulkan. You can use llama.cpp (for iGPU with vulkan) or ollama (for cpu).
My laptop is a cheap laptop from 5 years ago. Not cutting edge hardware.