5 ms·

Quote from Google: > Our work with Tensor has never been about speeds and feeds, or traditional performance metrics. It’s about pushing the mobile computing ex

Quote from Google: > Our work with Tensor has never been about speeds and feeds, or traditional performance metrics. It’s about pushing the mobile computing experience forward. And in our new Tensor G3 chip, every major subsystem has been upgraded, paving the way for on-device generative AI. It includes the latest generation of Arm CPUs, an upgraded GPU, new ISP and Imaging DSP and our next-gen TPU, which was custom-designed to run Google’s AI models. Then why the internet requirement? Were they trying something that failed last minute forcing them to ship it the way it is? Or was this always just advertisement material?

ForkMeOnTinder 3y ago
> Were they trying something that failed last minute Seems likely. Your chip underperforms? Just block people from using benchmarking apps, and also tell the press you never cared about speed. Problem solved.
- josephcsible 3y ago
  See https://www.notebookcheck.net/Google-blocked-Pixel-8-Pixel-8-Pro-reviewers-from-using-popular-benchmarks-to-test-the-Tensor-G3-chip-new-owners-too.759260.0.html https://www.notebookcheck.net/Google-blocked-Pixel-8-Pixel-8... for details about "block people from using benchmarking apps".
  - AlexandrB 3y ago
    Wtf? How is this not an abuse of market power?
    - canucker2016 3y ago
      the notebookcheck.net article states: "In a highly unusual move, it has been revealed that Google has blocked reviewers of the Pixel 8 and Pixel 8 Pro from installing popular benchmarks including Geekbench and 3D Mark." there's an embedded URL linking to a YouTube video, which shows that the Google Play store won't allow the user to install Geekbench or 3dMark on the pre-released Pixel 8 review devices. see https://www.youtube.com/watch?v=bk4ZUmKqRm0&t=330 https://www.youtube.com/watch?v=bk4ZUmKqRm0&t=330
      - charcircuit 3y ago
        Those app are notorious for leaking performance. At least with geekbench you have to buy a license to even have the choice to not upload results.
    - fidotron 3y ago
      It's rife across the whole benchmarking industry. Lies, damn lies, and benchmarks. The massive problem, which has particularly afflicted Qualcomm and Intel in the past, is the difference between peak and sustained performance being absolutely huge, since running at peak would trigger thermal conditions necessitating dynamic down clocking to below normal speeds, which would then persist for a surprisingly long time before normal performance resumes. Certain groups would detect known benchmarking code and alter the acceptable thermal parameters for the execution of the benchmark tests. (i.e. allow the device to become unusually hot). As others have mentioned this has all the hallmarks of an execution failure, either by Google, Samsung, both, or some unknown third party. Google have a cultural problem of believing way too much in theoretical untested potential solutions instead of aiming for the boring known to work but only 95% as theoretically good, only to find when building it reality is different enough to more than nullify the advantages. It has bitten them in this area before, and it more than likely will again.
      - hypercube33 3y ago
        Now that you mention it...I have only seen one review about workstations that benchmarked machines for 24 hours to make sure they didn't thermally saturate and throttle and I'm wondering why that isn't more of a metric for non mobile stuff. Heck even mobile workstations may be put under heavy load for durations and this would show if their cooling keeps up or falls short
      - fomine3 3y ago
        Apple too. iPhone is known for state of the art SoC and moderate heat design, that causes unstable peak performance for heavy gaming or benchmark.
        fidotron 3y ago
        No, Apple is not the same for this. Their devices do not suffer from anything like the same downtime after running at peak - but they can do nothing to stop developers draining the battery, and that's what is going on. If they were to do that developers would have a giant hissy fit about it. I have worked with prototypes from other manufacturers that had to be recalled after burning people. I have also found bugs in SoCs that their manufacturers were lying to OEMs about, and had to explain to different OEMs what was going on, after they had shipped millions of units. The simple truth is modern developers are irresponsible and so the manufacturers get to play everyone off against each other to spread as much self serving nonsense as possible.
- intelVISA 3y ago
  they can't keep getting away with this (Copy)(2)
- numpad0 3y ago
  saw some phone geeks suspecting last minute addition for thermal dispersion under LCD as well as frequency limiter to manage heat. not sure where the suspicion came from but sounds plausible
nunez 3y ago
Maybe Google intends on pushing those activities entirely on device later?
- IAmGraydon 3y ago
  I was thinking this. It may not be ready yet and they had to launch before it was done.
barkingcat 3y ago
All of that text you quoted is pure advertising nonsense. It's like reading buzzword after buzzword.
dataflow 3y ago
> Our work with Tensor has never been about speeds and feeds, or traditional performance metrics Meanwhile, on [1] they say: > Pixel 8 and Pixel 8 Pro are equipped with Google Tensor G3, Google’s fastest, most efficient, and most secure chip yet. Every major component of the chip has been upgraded for enhanced performance and efficiency. Not only has the number of machine learning models on device more than doubled since 2021, but their complexity and sophistication have increased as well. And Tensor is running many of these complex models at the same time. Sounds to me like it was very much about performance... [1] https://store.google.com/intl/en/ideas/articles/google-tensor-pixel-smartphone/ https://store.google.com/intl/en/ideas/articles/google-tenso...
qwertox 3y ago
paving the way for on-device generative AI seems to indicate that they're not there yet.
- troupo 3y ago
  I wonder if they can get there. Apple did bet on the on-device ML, with varied results. I doubt Google can get around their "we run everything in the cloud" mindset. As an anecdotal data point: they recently silently defaulted storing photos in Google Photos instead of on device (tweet in Russian: https://twitter.com/igrekde/status/1715456025594134690 https://twitter.com/igrekde/status/1715456025594134690)
  - jpalawaga 3y ago
    google has tons of on-device ml. live audio transcriptions, for example. always-on shazam. generative ai is very computationally expensive, even for inference. I too wonder if they can get there, but nobody has really done it.
    - kortilla 3y ago
      Is Shazam considered AI? I thought fingerprinting songs was a relatively straight-forward algo that didn’t require any training.
      - ShamelessC 3y ago
        Sort of. They were early to use vector databases for semantic search. Now we use embeddings from a neural net as features for semantic search, I think they used lots of labels and analyzed features like beats per minute, chord progression, etc. happy to be corrected of course.
- antifa 3y ago
  To me, that choice of words indicates Google is intending to claim the hardware is ready.
IshKebab 3y ago
It's pretty obvious - some ML tasks are easy enough to do on the phone (e.g. speech recognition); some are not (e.g. inpainting). Yeah they are being a bit slippery about it, but I don't think it's really a surprise that ML models that require a GTX 4090 to run aren't going to work on a phone.
- user_7832 3y ago
  While I mostly agree with this, apparently some AI features run fine offline on the Pixel 7 Pro but not on the 8/Pro (as per a comment on the article). And I'm not sure if the computation for AI wallpapers is that high to need to be offloaded too. But then again, I think this was more of a bug than an intended feature.
- izacus 3y ago
  The fact that those tasks are running in cloud was called out in Pixel 8 keynote and mentioned in their blog post as well - https://blog.google/products/pixel/google-pixel-8-pro-camera/ https://blog.google/products/pixel/google-pixel-8-pro-camera... This seems yet another nothingburger news to generate clicks and HN rants.
  - IshKebab 3y ago
    It's not mentioned in that blog post except for Video Boost.
  - canucker2016 3y ago
    From the keynote, the only thing I remember being called out as offloaded to the cloud was the HDR video - a feature that will show up eventually on the phone. All of the AI photo-related tasks that the Google employees demoed, i.e. inpainting et al, happened in realtime. Rewatching the keynote just now, I see some of the Magic Editor-related scenes have the following disclaimer in tiny white text, "Features simulated and sequences shortened throughout ad. Magic Editor coming soon" - see https://www.youtube.com/watch?v=pxlaUCJZ27E&t=2416s https://www.youtube.com/watch?v=pxlaUCJZ27E&t=2416s
wccrawford 3y ago
That's exactly how it went with Stadia and "negative latency", and that was never actually implemented before they shuttered Stadia.
SoftTalker 3y ago
> Then why the internet requirement? So they can include it in their advertising profile about you, and later insert ads into the generated content.
- hef19898 3y ago
  Later?
  - l33t7332273 3y ago
    Do they currently?
danpalmer 3y ago
There's a bit of a difference between an on-device chip for some AI tasks (like all modern smartphones and some computers have), and high end server-class graphics cards/GPUs/accelerators than take hundreds of watts to run for generative AI. State of the art for low end generative AI is still running on some sort of desktop class GPU with optimised models and getting not-so-great results. A phone can't do that. Lots of ML happens on device, more is moving to the device. Generative AI is not ready for that yet from what I can tell (based on external experience, not Google specific).
- canucker2016 3y ago
  from Feb 2023 blog post, https://www.qualcomm.com/news/onq/2023/02/worlds-first-on-device-demonstration-of-stable-diffusion-on-android https://www.qualcomm.com/news/onq/2023/02/worlds-first-on-de...: "...The result of this full-stack optimization is running Stable Diffusion on a smartphone under 15 seconds for 20 inference steps to generate a 512x512 pixel image — this is the fastest inference on a smartphone and comparable to cloud latency. User text input is completely unconstrained." Hasn't shown up in a shipping cellphone yet AFAIK.
  - danpalmer 3y ago
    There's a big difference between tech demos and production software for a general audience. Generative AI is only barely production software at the best of times, so almost any compromises mean it's not shippable. This blog post is interesting, but there are so many caveats to it. It's a 1B parameter model which is tiny. Inference takes 20 seconds, which sucks for UX, but also that will be 20s of sustained extremely high load on the device, which means battery drain that would probably make this unshippable as well. It's also worth noting that the images in the blog post were not generated by this process as far as I can tell, they're just stable diffusion examples. It's good this research is being done of course, but I think we're a few years away from this being shipped in any real form on device.
    - canucker2016 3y ago
      You stated, "State of the art for low end generative AI is still running on some sort of desktop class GPU with optimised models and getting not-so-great results. A phone can't do that." The Qualcomm article states, "Read on to learn how Qualcomm AI Research performed full-stack AI optimizations using the Qualcomm AI Stack to deploy Stable Diffusion on an Android smartphone for the very first time. "

ForkMeOnTinder 3y ago

> Were they trying something that failed last minute Seems likely. Your chip underperforms? Just block people from using benchmarking apps, and also tell the press you never cared about speed. Problem solved.

josephcsible 3y ago
See https://www.notebookcheck.net/Google-blocked-Pixel-8-Pixel-8-Pro-reviewers-from-using-popular-benchmarks-to-test-the-Tensor-G3-chip-new-owners-too.759260.0.html https://www.notebookcheck.net/Google-blocked-Pixel-8-Pixel-8... for details about "block people from using benchmarking apps".
- AlexandrB 3y ago
  Wtf? How is this not an abuse of market power?
  - canucker2016 3y ago
    the notebookcheck.net article states: "In a highly unusual move, it has been revealed that Google has blocked reviewers of the Pixel 8 and Pixel 8 Pro from installing popular benchmarks including Geekbench and 3D Mark." there's an embedded URL linking to a YouTube video, which shows that the Google Play store won't allow the user to install Geekbench or 3dMark on the pre-released Pixel 8 review devices. see https://www.youtube.com/watch?v=bk4ZUmKqRm0&t=330 https://www.youtube.com/watch?v=bk4ZUmKqRm0&t=330
    - charcircuit 3y ago
      Those app are notorious for leaking performance. At least with geekbench you have to buy a license to even have the choice to not upload results.
  - fidotron 3y ago
    It's rife across the whole benchmarking industry. Lies, damn lies, and benchmarks. The massive problem, which has particularly afflicted Qualcomm and Intel in the past, is the difference between peak and sustained performance being absolutely huge, since running at peak would trigger thermal conditions necessitating dynamic down clocking to below normal speeds, which would then persist for a surprisingly long time before normal performance resumes. Certain groups would detect known benchmarking code and alter the acceptable thermal parameters for the execution of the benchmark tests. (i.e. allow the device to become unusually hot). As others have mentioned this has all the hallmarks of an execution failure, either by Google, Samsung, both, or some unknown third party. Google have a cultural problem of believing way too much in theoretical untested potential solutions instead of aiming for the boring known to work but only 95% as theoretically good, only to find when building it reality is different enough to more than nullify the advantages. It has bitten them in this area before, and it more than likely will again.
    - hypercube33 3y ago
      Now that you mention it...I have only seen one review about workstations that benchmarked machines for 24 hours to make sure they didn't thermally saturate and throttle and I'm wondering why that isn't more of a metric for non mobile stuff. Heck even mobile workstations may be put under heavy load for durations and this would show if their cooling keeps up or falls short
    - fomine3 3y ago
      Apple too. iPhone is known for state of the art SoC and moderate heat design, that causes unstable peak performance for heavy gaming or benchmark.
      - fidotron 3y ago
        No, Apple is not the same for this. Their devices do not suffer from anything like the same downtime after running at peak - but they can do nothing to stop developers draining the battery, and that's what is going on. If they were to do that developers would have a giant hissy fit about it. I have worked with prototypes from other manufacturers that had to be recalled after burning people. I have also found bugs in SoCs that their manufacturers were lying to OEMs about, and had to explain to different OEMs what was going on, after they had shipped millions of units. The simple truth is modern developers are irresponsible and so the manufacturers get to play everyone off against each other to spread as much self serving nonsense as possible.
intelVISA 3y ago
they can't keep getting away with this (Copy)(2)
numpad0 3y ago
saw some phone geeks suspecting last minute addition for thermal dispersion under LCD as well as frequency limiter to manage heat. not sure where the suspicion came from but sounds plausible

nunez 3y ago

Maybe Google intends on pushing those activities entirely on device later?

IAmGraydon 3y ago
I was thinking this. It may not be ready yet and they had to launch before it was done.

barkingcat 3y ago

All of that text you quoted is pure advertising nonsense. It's like reading buzzword after buzzword.

dataflow 3y ago

> Our work with Tensor has never been about speeds and feeds, or traditional performance metrics Meanwhile, on [1] they say: > Pixel 8 and Pixel 8 Pro are equipped with Google Tensor G3, Google’s fastest, most efficient, and most secure chip yet. Every major component of the chip has been upgraded for enhanced performance and efficiency. Not only has the number of machine learning models on device more than doubled since 2021, but their complexity and sophistication have increased as well. And Tensor is running many of these complex models at the same time. Sounds to me like it was very much about performance... [1] https://store.google.com/intl/en/ideas/articles/google-tensor-pixel-smartphone/ https://store.google.com/intl/en/ideas/articles/google-tenso...

qwertox 3y ago

paving the way for on-device generative AI seems to indicate that they're not there yet.

troupo 3y ago
I wonder if they can get there. Apple did bet on the on-device ML, with varied results. I doubt Google can get around their "we run everything in the cloud" mindset. As an anecdotal data point: they recently silently defaulted storing photos in Google Photos instead of on device (tweet in Russian: https://twitter.com/igrekde/status/1715456025594134690 https://twitter.com/igrekde/status/1715456025594134690)
- jpalawaga 3y ago
  google has tons of on-device ml. live audio transcriptions, for example. always-on shazam. generative ai is very computationally expensive, even for inference. I too wonder if they can get there, but nobody has really done it.
  - kortilla 3y ago
    Is Shazam considered AI? I thought fingerprinting songs was a relatively straight-forward algo that didn’t require any training.
    - ShamelessC 3y ago
      Sort of. They were early to use vector databases for semantic search. Now we use embeddings from a neural net as features for semantic search, I think they used lots of labels and analyzed features like beats per minute, chord progression, etc. happy to be corrected of course.
antifa 3y ago
To me, that choice of words indicates Google is intending to claim the hardware is ready.

IshKebab 3y ago

It's pretty obvious - some ML tasks are easy enough to do on the phone (e.g. speech recognition); some are not (e.g. inpainting). Yeah they are being a bit slippery about it, but I don't think it's really a surprise that ML models that require a GTX 4090 to run aren't going to work on a phone.

user_7832 3y ago
While I mostly agree with this, apparently some AI features run fine offline on the Pixel 7 Pro but not on the 8/Pro (as per a comment on the article). And I'm not sure if the computation for AI wallpapers is that high to need to be offloaded too. But then again, I think this was more of a bug than an intended feature.
izacus 3y ago
The fact that those tasks are running in cloud was called out in Pixel 8 keynote and mentioned in their blog post as well - https://blog.google/products/pixel/google-pixel-8-pro-camera/ https://blog.google/products/pixel/google-pixel-8-pro-camera... This seems yet another nothingburger news to generate clicks and HN rants.
- IshKebab 3y ago
  It's not mentioned in that blog post except for Video Boost.
- canucker2016 3y ago
  From the keynote, the only thing I remember being called out as offloaded to the cloud was the HDR video - a feature that will show up eventually on the phone. All of the AI photo-related tasks that the Google employees demoed, i.e. inpainting et al, happened in realtime. Rewatching the keynote just now, I see some of the Magic Editor-related scenes have the following disclaimer in tiny white text, "Features simulated and sequences shortened throughout ad. Magic Editor coming soon" - see https://www.youtube.com/watch?v=pxlaUCJZ27E&t=2416s https://www.youtube.com/watch?v=pxlaUCJZ27E&t=2416s

wccrawford 3y ago

That's exactly how it went with Stadia and "negative latency", and that was never actually implemented before they shuttered Stadia.

SoftTalker 3y ago

> Then why the internet requirement? So they can include it in their advertising profile about you, and later insert ads into the generated content.

hef19898 3y ago
Later?
- l33t7332273 3y ago
  Do they currently?

danpalmer 3y ago

There's a bit of a difference between an on-device chip for some AI tasks (like all modern smartphones and some computers have), and high end server-class graphics cards/GPUs/accelerators than take hundreds of watts to run for generative AI. State of the art for low end generative AI is still running on some sort of desktop class GPU with optimised models and getting not-so-great results. A phone can't do that. Lots of ML happens on device, more is moving to the device. Generative AI is not ready for that yet from what I can tell (based on external experience, not Google specific).

canucker2016 3y ago
from Feb 2023 blog post, https://www.qualcomm.com/news/onq/2023/02/worlds-first-on-device-demonstration-of-stable-diffusion-on-android https://www.qualcomm.com/news/onq/2023/02/worlds-first-on-de...: "...The result of this full-stack optimization is running Stable Diffusion on a smartphone under 15 seconds for 20 inference steps to generate a 512x512 pixel image — this is the fastest inference on a smartphone and comparable to cloud latency. User text input is completely unconstrained." Hasn't shown up in a shipping cellphone yet AFAIK.
- danpalmer 3y ago
  There's a big difference between tech demos and production software for a general audience. Generative AI is only barely production software at the best of times, so almost any compromises mean it's not shippable. This blog post is interesting, but there are so many caveats to it. It's a 1B parameter model which is tiny. Inference takes 20 seconds, which sucks for UX, but also that will be 20s of sustained extremely high load on the device, which means battery drain that would probably make this unshippable as well. It's also worth noting that the images in the blog post were not generated by this process as far as I can tell, they're just stable diffusion examples. It's good this research is being done of course, but I think we're a few years away from this being shipped in any real form on device.
  - canucker2016 3y ago
    You stated, "State of the art for low end generative AI is still running on some sort of desktop class GPU with optimised models and getting not-so-great results. A phone can't do that." The Qualcomm article states, "Read on to learn how Qualcomm AI Research performed full-stack AI optimizations using the Qualcomm AI Stack to deploy Stable Diffusion on an Android smartphone for the very first time. "