3 ms·
what tokens/s are you getting with a 122B MoE model in this setup? I didn't see any benchmarks in the benchmarks section on the readme.md
by altruios 6mo ago
what tokens/s are you getting with a 122B MoE model in this setup? I didn't see any benchmarks in the benchmarks section on the readme.md
- gigatexal 6mo agoyeah this I'd like to see added to teh readme.
- aegis_camera 6mo agoI'll add more details. We just wired up the pipeline on both MAC and IOS.
- aegis_camera 6mo agohttps://www.sharpai.org/benchmark/ https://www.sharpai.org/benchmark/ The MLX part is what we've done with SwiftLM, the local result is still being verified more details are on-going.