3 ms·

InferenceMAX: Open-Source Inference Benchmarking

4 points by pella 1y ago

deleted 1y ago
[deleted]
pella 1y ago
- https://github.com/InferenceMAX/InferenceMAX https://github.com/InferenceMAX/InferenceMAX - "NVIDIA GB200 NVL72, AMD MI355X, Throughput Token per GPU, Latency Tok/s/user, Perf per Dollar, Cost per Million Tokes, Tokens per Provisioned Megawatt, DeepSeek R1 670B, GPTOSS 120B, Llama3 70B"