3 ms·
for unified memory, the dense models are way too slow and for local GPU-based setups, large MoE are too large but they're fine on unified memory systems essent
by muyuu 5mo ago
for unified memory, the dense models are way too slow and for local GPU-based setups, large MoE are too large but they're fine on unified memory systems
essentially, hardware is the main reason you may choose one or the other locally
i have a Strix Halo system so I will be trying this Dwarf Star 4 thingie eventually when i have some free time