3 ms·
Demo: Selfhost Mixtral 8x7B MoE on Mac+cross devices, PORTABLE 2MB inference app
- 3Sophons 3y agoSteps: https://www.secondstate.io/articles/mixtral-8-7b/ https://www.secondstate.io/articles/mixtral-8-7b/
- Kelteseth 3y ago> The portable Wasm app automatically takes advantage of the hardware accelerators So this should work with AMD cards? And it would need at least 32 GB of RAM, right?