3 ms·
Am I missing it or is there no information about performance? Looking for a tokens/sec
by hamdingers 1y ago
Am I missing it or is there no information about performance? Looking for a tokens/sec
- simlevesque 1y agoHe didn't give that info but the transcript linked at the end shows how much time was spent for each query.
- aseipp 1y agoRight now I get 59 tok/sec on GPT-OSS 120B using Unsloth's dynamic 4-bit quants, via llama.cpp https://news.ycombinator.com/item?id=45881049 https://news.ycombinator.com/item?id=45881049