3 ms·

CPU moe offloading. see e.g. https://www.reddit.com/r/LocalLLaMA/comments/1t9eo83/running_qwen36_35b_a3b_on_8gb_vram_and_32gb_ram/ https://www.reddit.com/r/Loc

by mswphd 3mo ago

CPU moe offloading. see e.g. https://www.reddit.com/r/LocalLLaMA/comments/1t9eo83/running_qwen36_35b_a3b_on_8gb_vram_and_32gb_ram/ https://www.reddit.com/r/LocalLLaMA/comments/1t9eo83/running...