5 ms·

Using an alt-account for this as I don't normally post on non-technical or potentially controversial topics. I'm working on a product today which includes a PP

Using an alt-account for this as I don't normally post on non-technical or potentially controversial topics. I'm working on a product today which includes a PPG sensor, on an innocuous part of the body (e.g. wrist, etc.). While we expose many of the sensors to developers via an SDK API, the PPG sensor is heavily processed (downsampled, in a sense) to minimal information such as heart rate. This is due to concerns about application developers being able to use the raw PPG to create health inferences and/or use it as a way to identify/track individual users of our device. One of our demos was to a defense company and an engineer there was appalled at all the sensors we were trying to utilize. He said something to the effect of "No way I'm wearing that. You could do a lot of fucked-up shit with that. I would know, because I'm a fucked-up guy and my job is making that fucked-up shit." He followed up with: "Could you use that PPG sensor to see if I have an erection?" and the question was drowned away by laughter across the room, One of our data scientists who works closely with the PPG sensor models told me in private later: "I actually thought that was a really good question. I thought about it for awhile, and I think the answer is yes -- we could." I don't know if he's right -- the data we get from the PPG sensor is pretty dirty especially whenever the wearer is moving around. But our signal processing pales in comparison to what Apple is able to achieve, so maybe someone better could. Or maybe not, it's not something we're spending engineering time to determine!

DonHopkins 5y ago
Just to be clear, the first "P" in "PPG" stands for "Photo", not "Penile", right? https://en.wikipedia.org/wiki/Photoplethysmogram https://en.wikipedia.org/wiki/Photoplethysmogram But to be fair, there's also VPPG and CPPG: https://en.wikipedia.org/wiki/Vaginal_photoplethysmograph https://en.wikipedia.org/wiki/Vaginal_photoplethysmograph https://en.wikipedia.org/wiki/Clitoral_photoplethysmograph https://en.wikipedia.org/wiki/Clitoral_photoplethysmograph So maybe Apple should call their touch sensing and haptic feedback technology the "Faptic Engine", since the Apple Watch can detect when and how vigorously you're masturbating? (Apple Watch users: Do you take it off, or use the other hand, or is there "an app for that"?) https://www.webopedia.com/definitions/taptic-engine/ https://www.webopedia.com/definitions/taptic-engine/ >A combination of “tap” and “haptic feedback,” taptic engine is a name Apple created for its technology that provides tactile sensations in the form of vibrations to users of Apple devices like the Apple Watch, iPhones, iPads, and MacBook laptops. >Apple debuted its taptic engine technology along with the Force Touch feature in the 2015 editions of Apple MacBook notebooks and the initial Apple Watch, and the two features work together to provide a user with more input control and feedback. https://appleinsider.com/articles/16/09/27/inside-the-iphone-7-apples-taptic-engine-explained https://appleinsider.com/articles/16/09/27/inside-the-iphone... >Apple has moved the Taptic Engine to its third device — the iPhone 7. The new technology replaces the older linear actuator, and will ultimately bring a world of force feedback sensations to the user, as developers implement the technology in their own apps. >The Taptic Engine is Apple's implementation of haptic user interface feedback. Using a linear actuator, devices like the iPhone 7 can reproduce the sensation of motion or generate new and distinct tactile experiences. In some cases, audio feedback from onboard speakers completes the illusion. https://www.patentlyapple.com/patently-apple/2020/01/apple-won-39-patents-today-covering-an-enhanced-ppg-sensor-system-for-apple-watch-3d-haptic-touch-more.html https://www.patentlyapple.com/patently-apple/2020/01/apple-w... >Apple Won 39 Patents Today covering an Enhanced PPG Sensor System for Apple Watch, 3D-Haptic Touch & more Maybe there's another application (and nickname) for the Tap Strap 2: https://www.tapwithus.com/ https://www.tapwithus.com/ Tap Strap ASL (tracking the raw Tap Strap 2 device data and recognizing American Sign Language with machine learning): https://www.youtube.com/watch?v=sqjacp5Pqs0 https://www.youtube.com/watch?v=sqjacp5Pqs0
- runnerup 5y ago
  Yes, in our case -- and in the general case, PPG can be assumed to stand for Photoplethysmogram. Also I'm not aware of any examples where "Photoplethysmogram" was used as a "Penile (PHOTO)-plethysmography", generally what I've read on "Penile plethysmography" are more direct measurements of the penis itself, not optical measurements of blood flow elsewhere on the body. Because of that, I don't think it's reasonable to call Apples "touch sensing and haptic feedback technology the 'Faptic Engine' " at this time! but it is a funny pun! We merely entertained the idea that it might be possible to use machine learning to infer erection status from a ppg sensor somewhere else.
jamal-kumar 5y ago
I mean I did have a good chuckle at your story but... I gotta ask, what are the actual useful applications for what you are building? I have to step back from a ton of opportunities (Like I said, Clearview AI rang some alarm bells) because it wasn't the right thing to do. Some guy yesterday tried to rope me into one of those land banking scams and I actually sent him goatse and told him to delete my number, that's how far I want people with less than good ideas around me. Can't even pay me enough to violate my principles.
- runnerup 5y ago
  In this case the PPG sensor is pretty peripheral to the value proposition, which is why we can reduce it to just heart rate and still satisfy the needs of our customers (application developers). There are a number of other biometrics used which are more salient for what we're looking at, which is generally related to training people on new skills or relearning existing skills differently than they were originally learned. Most of the applications so far are non-defense and pretty non controversial. I'd genuinely be happy to talk about it with you offline but would rather not publically associate the product with that story because the product is pretty cool overall, and I'm a major technology skeptic in general so it takes me a lot to decide something is potentially valuable and not evil. Also the main misunderstanding there was the point: "No, our API does not expose that raw PPG sensor data so no application developer could make that inference". In terms of stepping back...I agree. I've done that on previous projects, left companies or forced gross changes in architecture to satisfy my conscience. It really helps this time that I'm working with a lot of people who aren't afraid to share stories like this one and talk about them candidly, as well as share links on public internal Teams groups about all kinds of privacy concerns over biometrics (theres no fear of creating a paper trail which shows "we knew or should have known"). As we are building the "platform" rather than the applications, everyone genuinely seems to feel it's important to limit the harm that third party developers can cause. And for now at least, the product is actually quite hobbled by erring towards safety. That said when you ask "what are the useful applications", that's a very difficult question for me to answer right now. Mainly I think the platform has a ton of potential, but each third party application development probably needs a lot more psychology doctoral level research to properly apply the platforms capabilities for each third party applications specific situation. And in lieu of that, developers have to either get lucky or have incredibly good intuition about the domains and people that they're creating applications for.

DonHopkins 5y ago

Just to be clear, the first "P" in "PPG" stands for "Photo", not "Penile", right? https://en.wikipedia.org/wiki/Photoplethysmogram https://en.wikipedia.org/wiki/Photoplethysmogram But to be fair, there's also VPPG and CPPG: https://en.wikipedia.org/wiki/Vaginal_photoplethysmograph https://en.wikipedia.org/wiki/Vaginal_photoplethysmograph https://en.wikipedia.org/wiki/Clitoral_photoplethysmograph https://en.wikipedia.org/wiki/Clitoral_photoplethysmograph So maybe Apple should call their touch sensing and haptic feedback technology the "Faptic Engine", since the Apple Watch can detect when and how vigorously you're masturbating? (Apple Watch users: Do you take it off, or use the other hand, or is there "an app for that"?) https://www.webopedia.com/definitions/taptic-engine/ https://www.webopedia.com/definitions/taptic-engine/ >A combination of “tap” and “haptic feedback,” taptic engine is a name Apple created for its technology that provides tactile sensations in the form of vibrations to users of Apple devices like the Apple Watch, iPhones, iPads, and MacBook laptops. >Apple debuted its taptic engine technology along with the Force Touch feature in the 2015 editions of Apple MacBook notebooks and the initial Apple Watch, and the two features work together to provide a user with more input control and feedback. https://appleinsider.com/articles/16/09/27/inside-the-iphone-7-apples-taptic-engine-explained https://appleinsider.com/articles/16/09/27/inside-the-iphone... >Apple has moved the Taptic Engine to its third device — the iPhone 7. The new technology replaces the older linear actuator, and will ultimately bring a world of force feedback sensations to the user, as developers implement the technology in their own apps. >The Taptic Engine is Apple's implementation of haptic user interface feedback. Using a linear actuator, devices like the iPhone 7 can reproduce the sensation of motion or generate new and distinct tactile experiences. In some cases, audio feedback from onboard speakers completes the illusion. https://www.patentlyapple.com/patently-apple/2020/01/apple-won-39-patents-today-covering-an-enhanced-ppg-sensor-system-for-apple-watch-3d-haptic-touch-more.html https://www.patentlyapple.com/patently-apple/2020/01/apple-w... >Apple Won 39 Patents Today covering an Enhanced PPG Sensor System for Apple Watch, 3D-Haptic Touch & more Maybe there's another application (and nickname) for the Tap Strap 2: https://www.tapwithus.com/ https://www.tapwithus.com/ Tap Strap ASL (tracking the raw Tap Strap 2 device data and recognizing American Sign Language with machine learning): https://www.youtube.com/watch?v=sqjacp5Pqs0 https://www.youtube.com/watch?v=sqjacp5Pqs0

runnerup 5y ago
Yes, in our case -- and in the general case, PPG can be assumed to stand for Photoplethysmogram. Also I'm not aware of any examples where "Photoplethysmogram" was used as a "Penile (PHOTO)-plethysmography", generally what I've read on "Penile plethysmography" are more direct measurements of the penis itself, not optical measurements of blood flow elsewhere on the body. Because of that, I don't think it's reasonable to call Apples "touch sensing and haptic feedback technology the 'Faptic Engine' " at this time! but it is a funny pun! We merely entertained the idea that it might be possible to use machine learning to infer erection status from a ppg sensor somewhere else.

jamal-kumar 5y ago

I mean I did have a good chuckle at your story but... I gotta ask, what are the actual useful applications for what you are building? I have to step back from a ton of opportunities (Like I said, Clearview AI rang some alarm bells) because it wasn't the right thing to do. Some guy yesterday tried to rope me into one of those land banking scams and I actually sent him goatse and told him to delete my number, that's how far I want people with less than good ideas around me. Can't even pay me enough to violate my principles.

runnerup 5y ago
In this case the PPG sensor is pretty peripheral to the value proposition, which is why we can reduce it to just heart rate and still satisfy the needs of our customers (application developers). There are a number of other biometrics used which are more salient for what we're looking at, which is generally related to training people on new skills or relearning existing skills differently than they were originally learned. Most of the applications so far are non-defense and pretty non controversial. I'd genuinely be happy to talk about it with you offline but would rather not publically associate the product with that story because the product is pretty cool overall, and I'm a major technology skeptic in general so it takes me a lot to decide something is potentially valuable and not evil. Also the main misunderstanding there was the point: "No, our API does not expose that raw PPG sensor data so no application developer could make that inference". In terms of stepping back...I agree. I've done that on previous projects, left companies or forced gross changes in architecture to satisfy my conscience. It really helps this time that I'm working with a lot of people who aren't afraid to share stories like this one and talk about them candidly, as well as share links on public internal Teams groups about all kinds of privacy concerns over biometrics (theres no fear of creating a paper trail which shows "we knew or should have known"). As we are building the "platform" rather than the applications, everyone genuinely seems to feel it's important to limit the harm that third party developers can cause. And for now at least, the product is actually quite hobbled by erring towards safety. That said when you ask "what are the useful applications", that's a very difficult question for me to answer right now. Mainly I think the platform has a ton of potential, but each third party application development probably needs a lot more psychology doctoral level research to properly apply the platforms capabilities for each third party applications specific situation. And in lieu of that, developers have to either get lucky or have incredibly good intuition about the domains and people that they're creating applications for.