4 ms·

Thanks for the kind words, we’ve improved performance now actually, follow the instructions on the core repo. Same model should run 3x faster on the same phone

Thanks for the kind words, we’ve improved performance now actually, follow the instructions on the core repo. Same model should run 3x faster on the same phone. These improvements are still being pushed to the SDKs though.

cco 1y ago
Wow! 3x is huge. I've had great experiences with gpt-oss20b on my laptop, a genuinely useful local model. 3x probably doesn't get my Pixel Pro 9 to being able to run 20b models, but its getting close!
- HenryNdubuaku 1y ago
  Although GPT OSS 20B has 1.7B activated parameters which will be fast, 20B weights is a lot for developers to bundle or consumers to download. That’s the actual problem.