4 ms·
LLM noob here. Would this optimization work with any MoE model or is it specific for this one?
by GTP 1y ago
LLM noob here. Would this optimization work with any MoE model or is it specific for this one?
- magicalhippo 1y agoIt's just doing a regex on the layer names, so should work with other models as long as they have the expert layers named similarly. It worked with Qwen 3 for me, for example. The option is just a shortcut, you can provide your own regex to move specific layers to specific devices.