12 ms·
Social network ad targeting can “listen into” conversations using “live photos”
- blakesterz 3y agoI can see how that apps COULD do this, but is there any evidence any apps actually do "listen" to live photos? I don't see any evidence presented. I actually don't doubt it, I'd just like to see some kind of evidence it's being done.
- testHNac 3y agoIf they didn't till today, they will start now.
- anamexis 3y agoWhy would random 1 second snippets of audio from when photos were being taken be useful to advertisers?
- NotYourLawyer 3y agoBecause there are billions of them.
- fauxpause_ 3y agoThere are billions of ants
- bastawhiz 3y agoAll of those individual seconds don't add up to a sum greater than their parts. There are trillions (quadrillions?) of seconds of reality that those same cameras/microphones didn't capture. Capturing a single second of each of a billion people's lives isn't really all that useful, especially for advertisers.
- SketchySeaBeast 3y ago"Based upon our spying people spend a large amount of time saying '...eese'. Can someone please find out what 'eese' is?"
- TeMPOraL 3y agoDo people really even say that?
- SketchySeaBeast 3y agoOnce upon a time people did in fact say "Cheese" when taking pictures.
- TeMPOraL 3y agoYes, but I thought that this was long ago, and since then, saying this got... cheesy.
- NotYourLawyer 3y agoI’m willing to bet that an AI could learn a lot about a person by listening to a large number of short audio clips, together with the photos themselves.
- bastawhiz 3y agoOne second isn't even long enough to hear the full pronunciation of all of most English words. Let's say people take ten photos per day. Let's generously say that captures ten random spoken words. Ten random words per day is hardly enough for a _human_ to learn anything about a person, let alone AI. AI cannot magically conjure data from noise. And when you think about what people take pictures of (their parking spot, selfies, nudes, landmarks, birthday cakes, sunsets, cats), what's heard is likely not even relevant to the picture taker's life or interests. If I look at all of the photos I've taken in the last two weeks, I've got: - Cat (2) - Building (1) - Stuff in my home (6) - Selfie (4) All thirteen photos were taken in ~silence.
- alephxyz 3y agoUltrasonic beacon tracking + being able to tell if someone is indoors, in a vehicle, at a concert, sporting event, watching something on tv, etc
- SketchySeaBeast 3y agoI would expect that you could tell if someone was indoors, in a vehicle, at a concert, or sporting event based upon the content of the photo.
- bastawhiz 3y agoMost photos have a timestamp and geotag. Knowing whether you're in a vehicle, at a concert or sporting event, or really doing just about anything can be gathered from that information as well as whatever the photos is of. One second of audio isn't giving much (additional) useful data.
- godelski 3y agoTraining ML models
- hackernewds 3y ago??? is this a meme
- godelski 3y agoWhy would this be a meme?
- umeshunni 3y agoBecause it's what people who have no clue say when they have run out of reasons for their conspiracy theory. Q: Why would anyone do that? A: It's to train an ML model, maan.. to train an AI, maaan... to target ads, maaaan..
- godelski 3y agoShort voice clips are used to train ML models. But you might also replace ML with ads in your comment and we could make a similar argument.
- soared 3y agoIt’s more feasible that an app would analyze the metadata and image itself than the audio, and I don’t think we’ve seen that happening.
- lancesells 3y agoMeta analyzes the metadata and image. I would be shocked if they aren't analyzing the audio of Live Photos since their release.
- soared 3y agoFor ad targeting?
- a13o 3y agoYes. Facebook runs all photos thru digital recognition. If you look at the alt text for a photo in the feed it will have a string generated by their photo analyzer. But that's just the front. They also use facial recognition to learn which other FB accounts are in your orbit, or make shadow profiles if they don't recognize a facial fingerprint. That way if that person ever signs up, they match the metadata from the shadow profile. This is all in service of scaling ad targeting surface area, which can be sold by the eyeball to generate massive wealth.
- spondylosaurus 3y agoHow would you turn live photo audio into adtech-compliant metadata? AFAIK there's nothing remotely like that in the OpenRTB spec, which is the standard for all real-time bidding ads.
- saagarjha 3y agoText to speech?
- spondylosaurus 3y ago
- thih9 3y agoI guess in some way it doesn’t matter, there shouldn’t be a way to do that in any case.
- altacc 3y agoThere is nothing in that thread that proves it could happen, let alone is happening, beyond "my friend says it's possible" and then everyone jumping on that as if some random person assuming something is possible is the same as it's actually happening. A falsehood only needs to smell like the truth to be believed. Access to photos is definitely happening but the rest is conjecture without some proper research into this. Ultimately this seems like far too much hassle and processing for very little gain, given the 2 seconds of audio in a live photo is bound to be useless for determining product relevance. Advertising algorithms are clever enough to target well without having to go to the expense of analysing real time audio from your mic or your photos & videos.
- MagicMoonlight 3y agoIf you had $10b and had to maximise data collection, would you implement it? Therefore it has been implemented.
- giobox 3y agoThis is actually a wider issue with live photos IMO - people often share them not remembering or noticing snippets of private conversations can be present. I have live photos enabled by default and have to remind myself to check the audio before I forward a picture quite often, or remember to turn it back to a static image. Sending your mother baby pics while your spouse moans about the mother-in-law in background, that kind of thing... I'd actually probably get a lot of use from a feature whereby live photos displays a quick transcript of any words it picked up in the recording over the top of the image, or in a prompt, any time you try to share one with clear button to remove it. Similar to the local voicemail transcription feature, then I wouldn't have to waste time playing the recording. You could probably add the same detected words to the photos search index too.
- benguild 3y agoagreed
- ok_dad 3y agoThere’s a setting in iPhones which will just send the static image when you share images.
- giobox 3y agoRight, but I don't want to always send static images either.
- hackernewds 3y agoLive photos do not even load on Android
- kevviiinn 3y agoWhat does that have to do with this?
- 3y ago
- Calvin02 3y agoIf I were running a social network, I would prioritize using AI to describe photos and using that as part of the targeting data over the 1-1.5 seconds of audio in Live Photos.
- nullsense 3y agoI would presume they're already past that point and just want all they can get.
- umeshunni 3y agoIs this the 2023 Tin Foil hat version of Instagram is listening to your microphone all the time?
- lancesells 3y agohttps://www.businessinsider.com/instagram-camera-on-when-not-taking-photos-ios-14-2020-7?op=1 https://www.businessinsider.com/instagram-camera-on-when-not... "Caused by a bug"
- june_twenty 3y agoI'd imagine it would be leaked if they were actually doing it. The bug story holds up imo.
- altacc 3y agoDifferent thing though. Things like this feed the myth that facebook is always listening to you. It's close enough to the idea to allow the cognitive leap from 'possible' to 'happening' without more than anecdotal evidence. I've never seen conclusive proof, e.g. showing data collection, transmission or usage, that any targeted advertising is based on overheard conversations whilst a phone not in use. Targeted advertising is just really good at segmenting people, combined with confirmation bias and the number of ads people don't register mentally until they are relevant in this context.
- Nextgrid 3y agoA similar risk exists with geotagged photos. Even if you don't mind it accessing locations from individual photos, nothing prevents the app from scanning your entire library and getting all the locations from there too. Same risk with timestamps - the app can get a list of all the timestamps and use that to confirm/refute other fuzzy datapoints collected elsewhere (web tracking, etc).
- chatmasta 3y agoThere are a few separate issues here: 1) photos permission includes access to full file for each photo, including geotagged metadata and any audio from "live photo" This seems desirable and working as expected. You might legitimately want to share the live photo, or the photo with metadata. However, it would be nice if the photo picker asked you whether you wanted to share with metadata. (Note: unlike the photo picker in native apps, the Safari file picker, ie the HTML5 <input type="file" />, does strip EXIF data by default.) 2) it's easy to take a live photo without realizing it, and similarly easy to share it This could be improved when selecting photos within the photo picker. There should be an option to separately select the static or full file (maybe with a long press). 2) granting access to "all photos" doesn't alert you when an app accesses a photo without you selecting it for some reason This used to be much worse, when the default and only option was to grant access to all photos, or no photos. Ever since Apple added the option to select which photos an app can access, I think this is less of a problem. But there is an education issue; people don't realize that "all" means the app can truly read all your photos, including geotagged metadata and audio of live photos, even without you selecting one to "upload" (or "do something with," depending on the purpose of the app). Even as a relative expert, I never understood this until I saw a demo app (to prove the privacy issue) that looped through all your photos and displayed your geotagged locations. But even if people did realize this, there is no indication that an app has accessed a photo, like there is with the icon indicating the microphone or camera was used recently. Perhaps a solution to this could be changing the photo picker to overlay a green dot on any photo the app has accessed.
- version_five 3y agoI guess I'm old. Is "live photo" a new name for video? This is the first time I've heard the term.
- JohnFen 3y agoI'd never heard of them either. Seems to be an iPhone thing. "Basically, Live Photos on iPhone is a regular digital photo and a short 2 seconds video recording the few moments before the photo. Essentially, the iPhone camera captures 2 media files, one photo and one video. When viewing a Live Photo, the operating system, iOS plays the video file first and then it shows the picture." The part that makes me feel old is that I have literally no idea why this sort of thing would be useful or desirable.
- majormajor 3y agoThey can be fun, you get a photo plus a short video "for free." Much easier than filming constantly and editing things down later. You can pick a different key frame to display as well, so it can be a good way to fix an awkwardly-timed yawn or such. I don't use them personally, though, but I'm a curmudgeon about my camera controls ;) I think Google added something similar in their Pixel camera app.
- JohnFen 3y agoEh, I think this is just one of those things I don't "get". A 2 second video seems without value to me.
- majormajor 3y agoFrom the POV of the user it's much more "a photo" than a video. It just gives you a bit of extra context and ability to pick your key frame.
- aendruk 3y agoA two-second video is pretty pointless. But a photo that incidentally moves a little bit is often charming.
- majormajor 3y ago"Ad targeting is listening more often than you think" is a "conspiracy theory" I believe in. I've had too many "mentioned an idea, then got targeted ads for it within a day" incidents to not think some combination of things on the Google Home or iPhones or other smart devices is spying. Yeah, yeah, familiar with all the "someone else on the same wifi probably searched for it" or "Baader-Meinhof" or what have you, but I don't live with a lot of people and Instagram ads in particular are VERY focused these days in a way that Baader-Meinhof doesn't really fit - if I'd seen the ad for [specific thing] a couple days before I mentioned it, it would've been noticeably weird and out of place in a different way. I was in ad-tech for a bit in the last decade, and even at a tiny company there were some data sources we bought or heard about that were pretty spooky, so I would not be shocked if there are some wild ones out there today.
- jvm___ 3y agoI mentioned not being flexible enough on a Facebook comment and instantly my reels feed is full of flexibility videos.
- JohnFen 3y ago> I was in ad-tech for a bit in the last decade I'm glad that you got out! I tend to be shy about calling things "evil", but I really do think the entire ad-tech industry qualifies.
- camdat 3y agoIt would be trivial to confirm yourself, Wireshark can scan for outgoing network packets and audio would require a large, continuous, byte stream. Unless you assume that Facebook et al are running ASR models on-device all the time and somehow making it invisible to the end-user.
- anon223345 3y agoGood call, I’m going to try this actually
- 3y ago
- kristo 3y agoI have been using social media strictly through mobile Safari for a month now and my ads have gotten so insanely irrelevant. It made me wonder what data they were getting from the app that they don’t have access to any more
- chatmasta 3y agoThe divergence between mobile Safari and in-app browsers is really frustrating. As far as I know, it's not possible to install content blockers that work within in-app browsers, which means that every Social Media or Newsreader app with an in-app browser is its own separate cookie sandbox exposing me to tracking scripts that I'd have otherwise avoided with content blockers in Safari. Not to mention the app itself has full access to the DOM of every page in its in-app browser. I know I can use PiHole or similar, but it feels like like a deceptive move from Apple, possibly with the motive of creating an incentive for developers to build for the app store where it's harder to block ads. Some apps are friendly about it. I appreciate that Apollo gives me the option of which browser to use for opening links, and that it distinguishes between "in-app browser" just as it does between Chrome, Firefox and Safari. For a long time I've had an idea for an app that can take advantage of this, but for the user's benefit. Imagine a newsreader app with modular news sources, where each "source" is a client side script that runs within the in-app browser, to navigate to a URL and then "parse the content" by getting rid of ads, paywalls, etc. So unlike a typical RSS feed reader app that makes you rawdog ten articles in an in-app browser before you hit a paywall, it would inject user-defined scripts into the in-app browser to make sure you never see the paywall in the first place. Philosophically, it's still a web browser, but with a more rigid interface for browsing between websites. The client is in control. It would be like Gopher for the modern age.
- scarface74 3y ago> The divergence between mobile Safari and in-app browsers is really frustrating There are two separate web views that apps can use. SFSafariViewController - content blockers will work with this. It’s an out of process web view that the host app has very little control over. WKWebView gives the developers a lot more control over the web view and it runs in process. Content blockers don’t work in these views.
- moomoo11 3y agoHow do these ad tech engineers at google and meta feel when they see posts like this? “Haha I made that xD. Oh well 600k tc lol”
- Gigachad 3y agoSome people have no problems developing literal killing machines, or even using them. I'm not at all surprised some have no problem building advertisement targeting.
- moomoo11 3y agoI guess you’re right. Even the Death Star engineers were probably excited to be working on cutting edge tech.
- version_five 3y agoThere are lots of coherent arguments (whether you agree with them or not) about why defence tech could be considered valuable. Ad-tech has a lot less going for it.
- Gigachad 3y agoMost people don't care all that much how much value their work adds to society. As long as it doesn't cause much harm and lets me live comfy, who cares.
- jonas21 3y agoLive photos are an Apple feature.
- flangola7 3y agoIt's also a setting in Google Camera
- AHOHA 3y ago
- pcdoodle 3y agoAnother thing to consider: A 3d fingerprint of a space can be created with live photos. I'm sure object recognition benefits from the small perspective changes as well.
- Havoc 3y agoLive Photos rarely contain more than a word or two in my experience. Not ideal but hardly end of world
- tedunangst 3y agoWhy isn't there an app that pays me to upload five seconds of audio from whatever bar I'm sitting at?
- hgsgm 3y ago"can". Not evidence that it "does".
- deleted 3y ago[deleted]
- BigBalli 3y agoIf you allow access to all Photos it includes videos... which includes audio. Cool "hack" but not doomsday for privacy.
- evrydayhustling 3y agoThis title is clickbait. A direct title would be: "iOS photo permissions expose audio to applications".
- thih9 3y agoIsn’t it just as easy to forget that audio in videos can be accessed in the same way? Sure, users mostly remember that they’re recording audio when shooting a video; but do they realise that advertisers have access to that as part of their photo library?
- jnsaff2 3y agoAfter experiencing facebook abusing the location metadata when sharing [0] I only use the iOS photos app and share from there to any app I don’t fully trust and each time I have to manually go and disable location sharing, I wish I could just default to never sharing it. [0] by tagging the photo location or suggesting location specific things etc