3 ms·
Any plans to support smaller models? I have a M4 Mac Mini with 16GB unified memory and an RTX 3060 (Laptop) with 6GB VRAM. My own product experiments all revolv
by brainless 2mo ago
Any plans to support smaller models? I have a M4 Mac Mini with 16GB unified memory and an RTX 3060 (Laptop) with 6GB VRAM. My own product experiments all revolve around small models and harness around them. Happy to contribute.
- mmastrac 2mo agoI've been pondering a smaller quantization and experts swapping for this! Happy to take on PRs if you want to experiment as well.