4 ms·
Until it thermally throttles 40 seconds later. But yeah, it's really cool how many platforms the vanilla code in llama.cpp can be easily compiled on. And someho
by superkuh 4y ago
Until it thermally throttles 40 seconds later. But yeah, it's really cool how many platforms the vanilla code in llama.cpp can be easily compiled on. And somehow I doubt they did the quantization step on the Pixel itself. My favorite was the person who did it on the rpi4. I know a guy working on getting it going on rpi3 but the ARM7/8 mixing , NEON support, and 64 bit ARM intrinsics are apparently non-trivial to convert.
- sebzim4500 4y ago>And somehow I doubt they did the quantization step on the Pixel itself You're probably right (because why would they?) but I don't see any reason they couldn't have done this if they wanted to.
- aquajet 4y agoI would have tried it but I didn't have enough storage on my phone to hold both the original and quantized weights.
- aquajet 4y agoI'm the original tweet author. Currently typing this from my Pixel after running it countless times :)
- mdrzn 4y agoI would need a step by step guide, but I would love to test it on my Galaxy S21 Ultra 5G. It has 16gb ram and I have about 350 GB available.