40 ms·
Yeah for that dense models are not great better to pick some MoE to get benefits of large vram pool while keeping reasonable speed.
by mirekrusin 2mo ago
Yeah for that dense models are not great better to pick some MoE to get benefits of large vram pool while keeping reasonable speed.