4 ms·
It seems to be comparable to other top models. Good, but nothing ground breaking.
by redox99 2y ago
It seems to be comparable to other top models. Good, but nothing ground breaking.
- jasonjmcghee 2y agoScout outperforms llama 3.1 405b and Gemini Flash 2.0 lite and it's MoE so as fast as a 17B model. That's pretty crazy. It means you can run it on a high-ram apple silicon and it's going to be insanely fast on groq (thousands of tokens per second). Time to first token will bottleneck the generation.