4 ms·
Looks like he bases the benchmarks off of https://github.com/ggml-org/llama.cpp/discussions/4167 https://github.com/ggml-org/llama.cpp/discussions/4167 I get w
by FloatArtifact 2y ago
Looks like he bases the benchmarks off of https://github.com/ggml-org/llama.cpp/discussions/4167 https://github.com/ggml-org/llama.cpp/discussions/4167
I get why he calls it a simulator, as it can simulate token output. It's an important aspect for evaluating use case if you need to get a sense of how much token output is relevant beyond the simple tokens per second text.