4 ms·

Its own CPU, and a PCIe endpoint? Sounds like a good persistent exploit vector. I wonder what tasks it will be used to accelerate.

by bri3d 9y ago

twic 9y ago
> Google says the Pixel Visual Core is designed "to handle the most challenging imaging and machine learning applications" I suspect they hope it will be the latter rather than the former. Design models at HQ, then distribute them to be trained on each user's data on their own phone. That means you need to use less bandwidth to transfer training data and models, which leaves more available to serve ads!
- amiramir 9y ago
  I suspect you are being slightly tongue in cheek about the bandwidth for ads (yay 5G ;-).) With that said, I think a key benefit of on-device learning is the privacy angle. The Pixel 2's always-on song identification is said to be done on the phone without your audio data being sent to Google. Similarly the Google Clips camera apparently does its magic without sending image data to Google. With devices having less to differentiate on privacy is increasingly visible in the marketing arena particularly for devices that seem to be watching or listening at all times.
zitterbewegung 9y ago
Also, what if you replace the ML model with something more nefarious? Like have the model classify things incorrectly. Or if a target face is in the picture then send a request to a server of your choosing.
bdonlan 9y ago
It's only useful as a persistent exploit vector if it has persistence. If I was designing this, it'd be simpler to have it boot off an image downloaded from the main SoC rather than giving it its own flash.
- bri3d 9y ago
  Good point - if it's just got firmware uploaded from the SoC, it's more of an escalation / memory protection bypass vector versus a persistence vector. I also wonder what gets sent across to it - for example, if a malicious video from the web could land on the coprocessor. Mostly it's just fun seeing more and more co-processing devices arrive in phones as they get broken :)

Its own CPU, and a PCIe endpoint? Sounds like a good persistent exploit vector. I wonder what tasks it will be used to accelerate.

by bri3d 9y ago

Its own CPU, and a PCIe endpoint? Sounds like a good persistent exploit vector. I wonder what tasks it will be used to accelerate.

twic 9y ago

> Google says the Pixel Visual Core is designed "to handle the most challenging imaging and machine learning applications" I suspect they hope it will be the latter rather than the former. Design models at HQ, then distribute them to be trained on each user's data on their own phone. That means you need to use less bandwidth to transfer training data and models, which leaves more available to serve ads!

amiramir 9y ago
I suspect you are being slightly tongue in cheek about the bandwidth for ads (yay 5G ;-).) With that said, I think a key benefit of on-device learning is the privacy angle. The Pixel 2's always-on song identification is said to be done on the phone without your audio data being sent to Google. Similarly the Google Clips camera apparently does its magic without sending image data to Google. With devices having less to differentiate on privacy is increasingly visible in the marketing arena particularly for devices that seem to be watching or listening at all times.

zitterbewegung 9y ago

Also, what if you replace the ML model with something more nefarious? Like have the model classify things incorrectly. Or if a target face is in the picture then send a request to a server of your choosing.

bdonlan 9y ago

It's only useful as a persistent exploit vector if it has persistence. If I was designing this, it'd be simpler to have it boot off an image downloaded from the main SoC rather than giving it its own flash.

bri3d 9y ago
Good point - if it's just got firmware uploaded from the SoC, it's more of an escalation / memory protection bypass vector versus a persistence vector. I also wonder what gets sent across to it - for example, if a malicious video from the web could land on the coprocessor. Mostly it's just fun seeing more and more co-processing devices arrive in phones as they get broken :)