3 ms·

Every time you see a little banner ad at the bottom of an app, there’s an instantaneous auction to show you that ad space. Google forwards info to bidders, who

Every time you see a little banner ad at the bottom of an app, there’s an instantaneous auction to show you that ad space. Google forwards info to bidders, who calculate how much they will spend to show you the ad. This means even the losers of the auction get a firehose of data. There are companies set up right now whose purpose is to lose that auction but collect the data anyways. It should not surprise you to learn, then, that In-Q-Tel (the non classified investment arm of the CIA) has invested in some of these analytics (read: digital surveillance) companies.

jillyboel 2y ago
How does one sign up for such a firehose? I was naively under the impression google did this bidding internally.
- rvnx 2y ago
  You need to get your ad platform technically certified and a very good reason to join (several millions in marketing budget will do the job), and a very good contact at Google, once this is done, this is what you have access to in real-time: https://developers.google.com/authorized-buyers/rtb/downloads/realtime-bidding-proto https://developers.google.com/authorized-buyers/rtb/download... otherwise you can go through intermediaries like BidSwitch which are essentially resellers, it's much easier but you get less information
add-sub-mul-div 2y ago
> There are companies set up right now whose purpose is to lose that auction but collect the data anyways. Which is against Google's terms, but of course they don't police it because it's their way of selling user data without explicitly selling user data.
- divbzero 2y ago
  If I understand this right, Google isn’t actually selling user data to auction losers because auction losers are collecting the data without spending any money.
  - deleted 2y ago
    [deleted]
  - add-sub-mul-div 2y ago
    They leak data that enables the marketplace they run and profit from. That's the indirection. They count on this obfuscation (lack of an explicit transaction for the bid request data) to keep people from thinking of them as selling data. Collectively the participants use this data to optimize their future bids and other surveillance/marketing efforts outside the Google ecosystem. I'm sure Google would prefer there not be any parties involved that are purely leeches. But they may not be able to or may not care enough to tell the difference between leeches and parties that are still working out their infrastructure and bidding strategy and will begin bidding eventually. Or perhaps making a certain number of low bids you don't intend to win is what keeps you in the game.
    - _njyk 2y ago
      Google doesn't want the leeches, they throttle the buyers who don't win enough, and Google was forced by the law to make lot of privacy improvements. For example, before the GDPR you would receive the full IP address of the user and the visited webpage URL. 123.456.123.456 https://www.youtube.com/watch?v=vNvlZg_zh1s https://www.youtube.com/watch?v=vNvlZg_zh1s Now you receive only: 123.456.123.0 and more limited information about the context Though this better privacy has a cost, because it means less revenue for Google, but also less revenue for the publishers, and less relevant advertising.
  - chenlian 2y ago
    So this is something like Greenland Island, as long as Trump don’t pay, it’s not sold to Trump?
doctorpangloss 2y ago
The ads customers do not see data. A handful of exchanges do. Participating in header bidding gives you data similar to what you would see from operating a popular mobile website. I don't know. It doesn't surprise me that In-Q-Tel makes investments in good arbitrage businesses like exchanges. I'm sure many good investors make good investments. It isn't some kind of cynical surveillance play.
- zoul 2y ago
  It isn't some kind of cynical surveillance play. The overwhelming experience of the contemporary internet says it always is.
- tveita 2y ago
  > Participating in header bidding gives you data similar to what you would see from operating a popular mobile website. A popular mobile website can only geolocate its own visitors based on IP and will have no idea what apps they have installed and what they do whenever they are not visiting that website. The ad exchange gets information any time the users opens any of thousands of apps, without them ever interacting with the exchange. "This device opened Grindr at this exact GPS coordinate, then Candy Crush at the church wifi, then a month later played Yahtzee for three hours near a military base in Afghanistan" Then they package up that historical data and sell it. You can have years of location data for whatever purpose you can think of. A thread on using this for surveillance from about a year ago: https://news.ycombinator.com/item?id=38289337 https://news.ycombinator.com/item?id=38289337 As I commented there, while the antitrust case against Google is well-deserved, one effect of breaking their ad monopoly is opening up for even more actors to receive real-time header bidding data. A look at the resolution of position data you can get: https://nrkbeta.no/2020/12/03/my-phone-was-spying-on-me-so-i-tracked-down-the-surveillants/ https://nrkbeta.no/2020/12/03/my-phone-was-spying-on-me-so-i... This will be correlated and joined with the geoip data from the apps without location data.
  - rvnx 2y ago
    Mostly yes, but there is a detail to know. RTB bidders are actually throtled based on the amount of traffic they purchase. For example, if you win on average 100% of the time, you will get to see 100% of the traffic (for example, third-parties can see 100% of YouTube views, know which IP block has seen what videos, etc). However, if you win on average 1% of the time, you may get only 1% of the requests. Usually it's proportional to the amount you buy, to prevent such passive data collection.
deleted 2y ago
[deleted]
iJohnDoe 2y ago
This link specifically says they get the data even if they don’t win the auction. https://www.ftc.gov/news-events/news/press-releases/2024/12/ftc-takes-action-against-mobilewalla-collecting-selling-sensitive-location-data https://www.ftc.gov/news-events/news/press-releases/2024/12/... > When Mobilewalla bid to place an ad for its clients on a real-time advertising bidding exchange, it unfairly collected and retained the information in the bid request, even when it didn’t have a winning bid, according to the complaint. The FTC’s complaint alleges that from January 2018 to June 2020, Mobilewalla collected more than 500 million unique consumer advertising identifiers paired with consumers’ precise location data. The raw location data Mobilewalla collected was not anonymized and the company doesn’t have policies to remove sensitive locations from the data set, meaning that such data could be used to identify individual consumers’ mobile devices and the sensitive locations they visited. The company sold access to this raw data to third-parties, including advertisers, data brokers and analytic firms.
- janalsncm 2y ago
  I think you meant to reply to someone who replied to me.
  - iJohnDoe 2y ago
    It was meant to reinforce your comment. :)
- rgbrenner 2y ago
  That doesn't mention Google anywhere in it that I can find.
NelsonMinar 2y ago
Is this legal in Europe? Is it common practice?
- rvnx 2y ago
  In Europe it's a big no, but Israel is not politically in Europe, and yes it is a very common and widely accepted practice from companies there. It started with the Download Valley where companies would monetize search ( https://en.wikipedia.org/wiki/Download_Valley https://en.wikipedia.org/wiki/Download_Valley ) and siphon personal data. The same with location information, plenty of shady SDKs offer to pay you if you leak location of your users. Again, mostly based in Israel, but sometimes New York.

jillyboel 2y ago

How does one sign up for such a firehose? I was naively under the impression google did this bidding internally.

rvnx 2y ago
You need to get your ad platform technically certified and a very good reason to join (several millions in marketing budget will do the job), and a very good contact at Google, once this is done, this is what you have access to in real-time: https://developers.google.com/authorized-buyers/rtb/downloads/realtime-bidding-proto https://developers.google.com/authorized-buyers/rtb/download... otherwise you can go through intermediaries like BidSwitch which are essentially resellers, it's much easier but you get less information

add-sub-mul-div 2y ago

> There are companies set up right now whose purpose is to lose that auction but collect the data anyways. Which is against Google's terms, but of course they don't police it because it's their way of selling user data without explicitly selling user data.

divbzero 2y ago
If I understand this right, Google isn’t actually selling user data to auction losers because auction losers are collecting the data without spending any money.
- deleted 2y ago
  [deleted]
- add-sub-mul-div 2y ago
  They leak data that enables the marketplace they run and profit from. That's the indirection. They count on this obfuscation (lack of an explicit transaction for the bid request data) to keep people from thinking of them as selling data. Collectively the participants use this data to optimize their future bids and other surveillance/marketing efforts outside the Google ecosystem. I'm sure Google would prefer there not be any parties involved that are purely leeches. But they may not be able to or may not care enough to tell the difference between leeches and parties that are still working out their infrastructure and bidding strategy and will begin bidding eventually. Or perhaps making a certain number of low bids you don't intend to win is what keeps you in the game.
  - _njyk 2y ago
    Google doesn't want the leeches, they throttle the buyers who don't win enough, and Google was forced by the law to make lot of privacy improvements. For example, before the GDPR you would receive the full IP address of the user and the visited webpage URL. 123.456.123.456 https://www.youtube.com/watch?v=vNvlZg_zh1s https://www.youtube.com/watch?v=vNvlZg_zh1s Now you receive only: 123.456.123.0 and more limited information about the context Though this better privacy has a cost, because it means less revenue for Google, but also less revenue for the publishers, and less relevant advertising.
- chenlian 2y ago
  So this is something like Greenland Island, as long as Trump don’t pay, it’s not sold to Trump?

doctorpangloss 2y ago

The ads customers do not see data. A handful of exchanges do. Participating in header bidding gives you data similar to what you would see from operating a popular mobile website. I don't know. It doesn't surprise me that In-Q-Tel makes investments in good arbitrage businesses like exchanges. I'm sure many good investors make good investments. It isn't some kind of cynical surveillance play.

zoul 2y ago
It isn't some kind of cynical surveillance play. The overwhelming experience of the contemporary internet says it always is.
tveita 2y ago
> Participating in header bidding gives you data similar to what you would see from operating a popular mobile website. A popular mobile website can only geolocate its own visitors based on IP and will have no idea what apps they have installed and what they do whenever they are not visiting that website. The ad exchange gets information any time the users opens any of thousands of apps, without them ever interacting with the exchange. "This device opened Grindr at this exact GPS coordinate, then Candy Crush at the church wifi, then a month later played Yahtzee for three hours near a military base in Afghanistan" Then they package up that historical data and sell it. You can have years of location data for whatever purpose you can think of. A thread on using this for surveillance from about a year ago: https://news.ycombinator.com/item?id=38289337 https://news.ycombinator.com/item?id=38289337 As I commented there, while the antitrust case against Google is well-deserved, one effect of breaking their ad monopoly is opening up for even more actors to receive real-time header bidding data. A look at the resolution of position data you can get: https://nrkbeta.no/2020/12/03/my-phone-was-spying-on-me-so-i-tracked-down-the-surveillants/ https://nrkbeta.no/2020/12/03/my-phone-was-spying-on-me-so-i... This will be correlated and joined with the geoip data from the apps without location data.
- rvnx 2y ago
  Mostly yes, but there is a detail to know. RTB bidders are actually throtled based on the amount of traffic they purchase. For example, if you win on average 100% of the time, you will get to see 100% of the traffic (for example, third-parties can see 100% of YouTube views, know which IP block has seen what videos, etc). However, if you win on average 1% of the time, you may get only 1% of the requests. Usually it's proportional to the amount you buy, to prevent such passive data collection.

deleted 2y ago

[deleted]

iJohnDoe 2y ago

This link specifically says they get the data even if they don’t win the auction. https://www.ftc.gov/news-events/news/press-releases/2024/12/ftc-takes-action-against-mobilewalla-collecting-selling-sensitive-location-data https://www.ftc.gov/news-events/news/press-releases/2024/12/... > When Mobilewalla bid to place an ad for its clients on a real-time advertising bidding exchange, it unfairly collected and retained the information in the bid request, even when it didn’t have a winning bid, according to the complaint. The FTC’s complaint alleges that from January 2018 to June 2020, Mobilewalla collected more than 500 million unique consumer advertising identifiers paired with consumers’ precise location data. The raw location data Mobilewalla collected was not anonymized and the company doesn’t have policies to remove sensitive locations from the data set, meaning that such data could be used to identify individual consumers’ mobile devices and the sensitive locations they visited. The company sold access to this raw data to third-parties, including advertisers, data brokers and analytic firms.

janalsncm 2y ago
I think you meant to reply to someone who replied to me.
- iJohnDoe 2y ago
  It was meant to reinforce your comment. :)
rgbrenner 2y ago
That doesn't mention Google anywhere in it that I can find.

NelsonMinar 2y ago

Is this legal in Europe? Is it common practice?

rvnx 2y ago
In Europe it's a big no, but Israel is not politically in Europe, and yes it is a very common and widely accepted practice from companies there. It started with the Download Valley where companies would monetize search ( https://en.wikipedia.org/wiki/Download_Valley https://en.wikipedia.org/wiki/Download_Valley ) and siphon personal data. The same with location information, plenty of shady SDKs offer to pay you if you leak location of your users. Again, mostly based in Israel, but sometimes New York.