4 ms·

I did the same thing on my Pixel Fold. Tried two different images with two different prompts: "What can you see?" and "Describe this image" First image ('Descr

I did the same thing on my Pixel Fold. Tried two different images with two different prompts: "What can you see?" and "Describe this image" First image ('Describe', photo of my desk) - 15.6 seconds to first token - 2.6 tokens/second - Total 180 seconds Second image ('What can you see?', photo of a bowl of pasta) - 10.3 seconds to first token - 3.1 tokens/second - Total 26 seconds The Edge Gallery app defaults to CPU as the accelerator. Switched to GPU. Pasta / what can you see: - It actually takes a full 1-2 minutes to start printing tokens. But the stats say 4.2 seconds to first token... - 5.8 tokens/second - 12 seconds total Desk / describe: - The output is: while True: print("[toxicity=0]") - Bugged? I stopped it after 80 seconds of output. 1st token after 4.1 seconds, then 5.7 tokens/second.

the_pwner224 1y ago
Pixel 4a release date = August 2020 Pixel Fold was in the Pixel 8 generation but uses the Tensor G2 from the 7s. Pixel 7 release date = October 2022 That's a 26 month difference, yet a full order of magnitude difference in token generation rate on the CPU. Who said Moore's Law is dead? ;)
- deleted 1y ago
  [deleted]
- sujayk_33 1y ago
  8 has G3 chip
- z2 1y ago
  As a another data point, on E4B, my Pixel 6 Pro (Tensor v1, Oct 2021) is getting about 4.4 t/s decode on a picture of a glass of milk, and over 6 t/s on text chat. It's amazing, I never dreamed I'd be viably running an 8 billion param model when I got it 4 years ago. And kudos to the Pixel team for including 12 GB of RAM when even today PC makers think they can get away with selling 8.