6 ms·

People don't grasp how easy it is to build data models like this even without privileged first-party data access. In 2012 I created a killer prototype that dem

People don't grasp how easy it is to build data models like this even without privileged first-party data access. In 2012 I created a killer prototype that demonstrated that you could accurately reconstruct most people's flight history at scale from social media and/or ad data. Probably the first of its kind. This has been possible for a long time. A quick sketch of how it worked: We filtered out all spatiotemporal edges in the entity graph with an implied speed of <300 kilometers per hour or <200 kilometers distance, IIRC. This was the proxy for "was on a plane". It also implicitly provided the origin and destination. These edges can be correlated with both public flight data and maintenance IoT data from jet engines to put entities on a specific flight. People overlook the extent to which innocuous industrial IoT data can be used as a proxy for relationships in unrelated domains. In rare cases, there was more than one plausible commercial flight. Because we had their flight history, we assumed in these cases that it was the primary airline they had used in the past, either generally or for that specific origin and destination. This almost always resolved perfectly. This was impressively effective and it didn't require first-party data from airlines or particularly sophisticated analytics. Space and time are the primary keys of reality.

wingspar 1y ago
Honestly asking, How did you validate your results?
- fsckboy 1y ago
  i don't have any special knowledge in this area, but just thinking about it idly while sitting here, "robbing their homes while they are away" comes to mind as a good proxy.
  - iterance 1y ago
    That seems like a risk, but not a validation method, unless you are feeling particularly bold.
  - animal_spirits 1y ago
    Reminds me of this news story of footballer John Terry who's house was robbed because he posted a picture of him on holiday. The insurance company tried to use a 'reasonable care' clause of home insurance to deny his insurance claim. - https://www.blakefire-security.co.uk/blog/social-media-and-john-terrys-burglary/ https://www.blakefire-security.co.uk/blog/social-media-and-j...
    - gruez 1y ago
      >The insurance company tried to use a 'reasonable care' clause of home insurance to deny his insurance claim. >- https://www.blakefire-security.co.uk/blog/social-media-and-j https://www.blakefire-security.co.uk/blog/social-media-and-j... FYI the source you posted never claimed that John Terry's insurance tried to deny the claim, only mentioning that "some" insurance companies warn of it. However even that claim is questionable, because it isn't even from an insurance company, it's from a content marketing piece by an insurance comparison website.
    - 0cf8612b2e1e 1y ago
      Wouldn’t that mean all celebrities are uninsurable? If politician/singer/athlete has a public away event, there is little they can do to obscure that fact.
      - lmm 1y ago
        Their policy could require a housesitter or security guard on those occasions, or some other risk countermeasure like an alarm system.
    - astura 1y ago
      Um that's kinda silly, everyone would know a footballer isn't home while he's at a football game?
  - wingspar 1y ago
    Basically a plot line on the show “Black List”. Had an inside guy at the post office who would forward people stopping mail delivery on vacation. Then used homes as safe houses.
- jandrewrogers 1y ago
  In this particular case it was just a proof-of-concept, albeit at scale. We did not run a proper ground-truthing process but people actually running that type of data model in production could have ground-truthed the analytic model if they wanted to. However, it turns out that thousands of people like to talk about their flights on social media, so we scraped that as a spot check and it mostly lined up perfectly. Good enough for a demo and it would have been difficult to come up with an alternative explanation for the patterns in the data. The purpose of the PoC was to sell the data analysis infrastructure that made that type analysis possible at scale, it wasn't about the data per se. It was a compelling demo we invented given the data that happened to be available. Startup life.
  - jcranmer 1y ago
    > Good enough for a demo and it would have been difficult to come up with an alternative explanation for the patterns in the data. For fun edge cases, there's always Antarctica, where you can travel from a US base (which looks like you're in the US) to a NZ base (which looks like you're in NZ) in a couple of minutes: https://brr.fyi/posts/credit-card-shenanigans https://brr.fyi/posts/credit-card-shenanigans
justanything 1y ago
Can you eli5 the implementation and how your prototype worked?
- gleenn 1y ago
  Sounds like if you have a record of a lot of location/timestamp data for people, you look at the distance difference divided by the time difference. Now you have average speed for any pair of points. Now filter where the average speed is as fast as a Boeing jet. That filters out most of the data except for people who are almost certainly on a plane. Et voila, you now look at those data points geolocation and you have people who traveled from one city to another because you already have the location. Compare City1 -> City2 with any public flights in those cities around those times and you know who flew on what flight from where to where and at what time.
  - timeon 1y ago
    I'm more interested in this part: > you have a record of a lot of location/timestamp data for people What is the source of that data?
    - jacobr1 1y ago
      from the parent post: `social media and/or ad data` So if you have ad impression data you have IP geolocation, or maybe better, along with the timestamp. Similarly for socials sometimes you get location metadata, and with image uploads you can can get location metadata (though today these are often stripped, historically they weren't).
      - Macha 1y ago
        Do people post on social media at high enough resolution to do this? Especially since the claim was in 2012, is airplane wifi and roaming data reliable enough for people to view enough ads to do this? Also where are you getting all a users ad impressions across different providers to have this kind of timing information? There's a lot of creepy data available linked to ads and lots of companies doing enrichment with different fields if you have already have a real identifier for a user (e.g. phone number, email, cc, isp account number), but this sounds way more like "we created a load of hypothetical simulated data and pretended we had access to all of it"
noman-land 1y ago
Where do you get "maintenance IoT data from jet engines"?
- supportengineer 1y ago
  Exactly. This does not pass the sniff test.
- jszymborski 1y ago
  Indeed, seems like it's way easier to just got the databroker route.
gruez 1y ago
>We filtered out all spatiotemporal edges in the entity graph with an implied speed of <300 kilometers per hour or <200 kilometers distance, IIRC. This was the proxy for "was on a plane". It also implicitly provided the origin and destination. Sounds like the bigger issue is that you're able to get "spatiotemporal" data in the first place? Otherwise it's like saying "we can figure out all stores you've been to, if we have your credit card transaction history". Sure, it's kinda creepy that you can figure out which stores I went to, but the bigger problem is that you can get the transaction data in the first place. Moreover whatever "spatiotemporal" data needed to reconstruct such flight history is probably more valuable than the flight history itself. Who cares if you know Joe flew on United 8340 when you have hour-by-hour updates on his rough location?
- magicalist 1y ago
  > Sounds like the bigger issue is that you're able to get "spatiotemporal" data in the first place? Yeah, this just sounds like it's written from the perspective of a data broker. Tying particular ad analytics (presumably ip geolocation?) to thousands of particular individuals and having it well populated enough to track them is "privileged first-party data access" by another name.
  - const_cast 1y ago
    Your location is leaked in many, many ways. Even if you have location services off on your phone, the first-party (Google, Apple) has access to your precise location. On Android, this bypasses VPNs, and I believe on iOS/Mac first-party apps also bypass VPNs. You are trusting that this data is not leaked to any third-parties. You cannot verify this, as the data is exfiltrated to servers which you can't verify. Okay, fine, I'll just install another operating system then, like KDE plasma mobile or GrapheneOS. Your location is still leaked 24/7. This is because your cellular modem has it's own operating system, running underneath your phone's operating system, which is triangulating your location at all times. Once again, you are trusting that telecommunications companies aren't misusing this - but please remember they're complied, by law, to make a lot of this information available to numerous third parties. Okay fine, let me just remove the Sim then and use my phone on Wifi only, always through a VPN. Your location is still being leaked potentially, for example, by your car. Your car also has a cellular modem which leaks your location, and you probably signed a contract allowing that data to be given to hundreds of third-parties. Of course, all of this is assuming you don't use any social media. Social media can also leak your location, even without location services. If you review a restaurant - that's your location. Where are your friends? You're probably around them. And on and on.
tqi 1y ago
Presumably ICE is trying to determine what cities / countries a person has visited and when, ie your starting point.
no_wizard 1y ago
What was your accuracy rate for this? I imagine it was quite high, but do you happen to remember what your +/- was?
bsder 1y ago
> People don't grasp how easy it is to build data models like this even without privileged first-party data access. People on this site probably understand this better than 99% of the world. The problem is "What can I, as an individual, do about it?"
- Den_VR 1y ago
  You can also exploit it for personal profit. As for stopping it, good luck. Best case is probably to degrade or poison data sources in a preferably legal way.
- autoexec 1y ago
  block ads, stay off most social media, don't use mobile devices while traveling
  - rs186 1y ago
    > don't use mobile devices while traveling Some airlines don't even allow you to check in without using their app, unless you are willing to pay for a fee.
canadiantim 1y ago
Great info
vpShane 1y ago
> People don't grasp how easy it is to build data models like this even without privileged first-party data access. Correct. But this is about them using first party data access, and often times this data is stolen; then utilized by any body/any thing - in this case, CBP and ICE to do unreasonable searches and building of data models which are often, but in these cases: Probably used against both citizens and non-citizens. It's absolutely amazing that these data models can be built, Meta and Anthropic pirated entire libraries of books to build their AI models at zero consequence. The point I'm making is, why are the people the one that have to be exploited in the building of these models, and then the sale of them?
deleted 1y ago
[deleted]
zahlman 1y ago
The thing that really tickles me is that there's supposedly all this frightening information that can be gathered on people, including by investigating the history of ads they were served; but then in the vast majority of cases the only use the Bad Guys ever seem to come up with for that information is to serve you more ads.
bmitc 1y ago
> People don't grasp how easy it is to build data models like this even without privileged first-party data access. > We filtered out all spatiotemporal edges How were you getting spatiotemporal data?
smcin 1y ago
Can you post a link/self-citation to any 2012 post on that prototype, I couldn't find it. Was this only on US data, or EU or worldwide? And when you say "ad data", do you mean like ADINT? And when you infer "was on a plane" from spatiotemporal edges in the entity graph with an implied speed of >300 km/h and >200 km distance, does that only work if the individual social-media app itself is collecting real-time geolocation? So if you only access social media from the browser not the app, does that partially prevent this privacy exploit?

wingspar 1y ago

Honestly asking, How did you validate your results?

fsckboy 1y ago
i don't have any special knowledge in this area, but just thinking about it idly while sitting here, "robbing their homes while they are away" comes to mind as a good proxy.
- iterance 1y ago
  That seems like a risk, but not a validation method, unless you are feeling particularly bold.
- animal_spirits 1y ago
  Reminds me of this news story of footballer John Terry who's house was robbed because he posted a picture of him on holiday. The insurance company tried to use a 'reasonable care' clause of home insurance to deny his insurance claim. - https://www.blakefire-security.co.uk/blog/social-media-and-john-terrys-burglary/ https://www.blakefire-security.co.uk/blog/social-media-and-j...
  - gruez 1y ago
    >The insurance company tried to use a 'reasonable care' clause of home insurance to deny his insurance claim. >- https://www.blakefire-security.co.uk/blog/social-media-and-j https://www.blakefire-security.co.uk/blog/social-media-and-j... FYI the source you posted never claimed that John Terry's insurance tried to deny the claim, only mentioning that "some" insurance companies warn of it. However even that claim is questionable, because it isn't even from an insurance company, it's from a content marketing piece by an insurance comparison website.
  - 0cf8612b2e1e 1y ago
    Wouldn’t that mean all celebrities are uninsurable? If politician/singer/athlete has a public away event, there is little they can do to obscure that fact.
    - lmm 1y ago
      Their policy could require a housesitter or security guard on those occasions, or some other risk countermeasure like an alarm system.
  - astura 1y ago
    Um that's kinda silly, everyone would know a footballer isn't home while he's at a football game?
- wingspar 1y ago
  Basically a plot line on the show “Black List”. Had an inside guy at the post office who would forward people stopping mail delivery on vacation. Then used homes as safe houses.
jandrewrogers 1y ago
In this particular case it was just a proof-of-concept, albeit at scale. We did not run a proper ground-truthing process but people actually running that type of data model in production could have ground-truthed the analytic model if they wanted to. However, it turns out that thousands of people like to talk about their flights on social media, so we scraped that as a spot check and it mostly lined up perfectly. Good enough for a demo and it would have been difficult to come up with an alternative explanation for the patterns in the data. The purpose of the PoC was to sell the data analysis infrastructure that made that type analysis possible at scale, it wasn't about the data per se. It was a compelling demo we invented given the data that happened to be available. Startup life.
- jcranmer 1y ago
  > Good enough for a demo and it would have been difficult to come up with an alternative explanation for the patterns in the data. For fun edge cases, there's always Antarctica, where you can travel from a US base (which looks like you're in the US) to a NZ base (which looks like you're in NZ) in a couple of minutes: https://brr.fyi/posts/credit-card-shenanigans https://brr.fyi/posts/credit-card-shenanigans

justanything 1y ago

Can you eli5 the implementation and how your prototype worked?

gleenn 1y ago
Sounds like if you have a record of a lot of location/timestamp data for people, you look at the distance difference divided by the time difference. Now you have average speed for any pair of points. Now filter where the average speed is as fast as a Boeing jet. That filters out most of the data except for people who are almost certainly on a plane. Et voila, you now look at those data points geolocation and you have people who traveled from one city to another because you already have the location. Compare City1 -> City2 with any public flights in those cities around those times and you know who flew on what flight from where to where and at what time.
- timeon 1y ago
  I'm more interested in this part: > you have a record of a lot of location/timestamp data for people What is the source of that data?
  - jacobr1 1y ago
    from the parent post: `social media and/or ad data` So if you have ad impression data you have IP geolocation, or maybe better, along with the timestamp. Similarly for socials sometimes you get location metadata, and with image uploads you can can get location metadata (though today these are often stripped, historically they weren't).
    - Macha 1y ago
      Do people post on social media at high enough resolution to do this? Especially since the claim was in 2012, is airplane wifi and roaming data reliable enough for people to view enough ads to do this? Also where are you getting all a users ad impressions across different providers to have this kind of timing information? There's a lot of creepy data available linked to ads and lots of companies doing enrichment with different fields if you have already have a real identifier for a user (e.g. phone number, email, cc, isp account number), but this sounds way more like "we created a load of hypothetical simulated data and pretended we had access to all of it"

noman-land 1y ago

Where do you get "maintenance IoT data from jet engines"?

supportengineer 1y ago
Exactly. This does not pass the sniff test.
jszymborski 1y ago
Indeed, seems like it's way easier to just got the databroker route.

gruez 1y ago

>We filtered out all spatiotemporal edges in the entity graph with an implied speed of <300 kilometers per hour or <200 kilometers distance, IIRC. This was the proxy for "was on a plane". It also implicitly provided the origin and destination. Sounds like the bigger issue is that you're able to get "spatiotemporal" data in the first place? Otherwise it's like saying "we can figure out all stores you've been to, if we have your credit card transaction history". Sure, it's kinda creepy that you can figure out which stores I went to, but the bigger problem is that you can get the transaction data in the first place. Moreover whatever "spatiotemporal" data needed to reconstruct such flight history is probably more valuable than the flight history itself. Who cares if you know Joe flew on United 8340 when you have hour-by-hour updates on his rough location?

magicalist 1y ago
> Sounds like the bigger issue is that you're able to get "spatiotemporal" data in the first place? Yeah, this just sounds like it's written from the perspective of a data broker. Tying particular ad analytics (presumably ip geolocation?) to thousands of particular individuals and having it well populated enough to track them is "privileged first-party data access" by another name.
- const_cast 1y ago
  Your location is leaked in many, many ways. Even if you have location services off on your phone, the first-party (Google, Apple) has access to your precise location. On Android, this bypasses VPNs, and I believe on iOS/Mac first-party apps also bypass VPNs. You are trusting that this data is not leaked to any third-parties. You cannot verify this, as the data is exfiltrated to servers which you can't verify. Okay, fine, I'll just install another operating system then, like KDE plasma mobile or GrapheneOS. Your location is still leaked 24/7. This is because your cellular modem has it's own operating system, running underneath your phone's operating system, which is triangulating your location at all times. Once again, you are trusting that telecommunications companies aren't misusing this - but please remember they're complied, by law, to make a lot of this information available to numerous third parties. Okay fine, let me just remove the Sim then and use my phone on Wifi only, always through a VPN. Your location is still being leaked potentially, for example, by your car. Your car also has a cellular modem which leaks your location, and you probably signed a contract allowing that data to be given to hundreds of third-parties. Of course, all of this is assuming you don't use any social media. Social media can also leak your location, even without location services. If you review a restaurant - that's your location. Where are your friends? You're probably around them. And on and on.

tqi 1y ago

Presumably ICE is trying to determine what cities / countries a person has visited and when, ie your starting point.

no_wizard 1y ago

What was your accuracy rate for this? I imagine it was quite high, but do you happen to remember what your +/- was?

bsder 1y ago

> People don't grasp how easy it is to build data models like this even without privileged first-party data access. People on this site probably understand this better than 99% of the world. The problem is "What can I, as an individual, do about it?"

Den_VR 1y ago
You can also exploit it for personal profit. As for stopping it, good luck. Best case is probably to degrade or poison data sources in a preferably legal way.
autoexec 1y ago
block ads, stay off most social media, don't use mobile devices while traveling
- rs186 1y ago
  > don't use mobile devices while traveling Some airlines don't even allow you to check in without using their app, unless you are willing to pay for a fee.

canadiantim 1y ago

Great info

vpShane 1y ago

> People don't grasp how easy it is to build data models like this even without privileged first-party data access. Correct. But this is about them using first party data access, and often times this data is stolen; then utilized by any body/any thing - in this case, CBP and ICE to do unreasonable searches and building of data models which are often, but in these cases: Probably used against both citizens and non-citizens. It's absolutely amazing that these data models can be built, Meta and Anthropic pirated entire libraries of books to build their AI models at zero consequence. The point I'm making is, why are the people the one that have to be exploited in the building of these models, and then the sale of them?

deleted 1y ago

[deleted]

zahlman 1y ago

The thing that really tickles me is that there's supposedly all this frightening information that can be gathered on people, including by investigating the history of ads they were served; but then in the vast majority of cases the only use the Bad Guys ever seem to come up with for that information is to serve you more ads.

bmitc 1y ago

> People don't grasp how easy it is to build data models like this even without privileged first-party data access. > We filtered out all spatiotemporal edges How were you getting spatiotemporal data?

smcin 1y ago

Can you post a link/self-citation to any 2012 post on that prototype, I couldn't find it. Was this only on US data, or EU or worldwide? And when you say "ad data", do you mean like ADINT? And when you infer "was on a plane" from spatiotemporal edges in the entity graph with an implied speed of >300 km/h and >200 km distance, does that only work if the individual social-media app itself is collecting real-time geolocation? So if you only access social media from the browser not the app, does that partially prevent this privacy exploit?