5 ms·
It’s interesting that they kept the price the same while doing inference on Cerebras is much more expensive.
by simianwords 9mo ago
It’s interesting that they kept the price the same while doing inference on Cerebras is much more expensive.
- chillee 9mo agothis is almost certainly not being done on cerebras
- diwank 9mo agoI dont think this is Cerebras. Running on cerebras would change model behavior a bit and it could potentially get a ~10x speedup and it'd be more expensive. So most likely this is them writing new more optimized kernels for Blackwell series maybe?
- simianwords 9mo agoFair point but it remains to answer - why isn’t this speed up available in ChatGPT and only in the api?