3 ms·
Oddly, they are only charging slightly more for their hosted version: open-mistral-7b is 25c/m tokens open-mistral-nemo-2407 is 30c/m tokens https://mistral.a
by voiper1 2y ago
Oddly, they are only charging slightly more for their hosted version:
open-mistral-7b is 25c/m tokens
open-mistral-nemo-2407 is 30c/m tokens
https://mistral.ai/technology/#pricing https://mistral.ai/technology/#pricing
- dannyw 2y agoPossibly a NVIDIA subsidy. You run NEMO models, you get cheaper GPUs.
- Palmik 2y agoThey specifically call out fp8 aware training and TensoRT LLM is really good (efficient) with fp8 inference on H100 and other hopper cards. It's possible that they run the 7b natively in fp16 as smaller models suffer more from even "modest" quantization like this.