4 ms·

What about using GPT-3 (et al.) to generate plausible fake data?

dredmorbius 6y ago
Variously: - You're emitting side-channel metadata all the time, most especially location and activity data (especially if you have any mobile devices) which is 1) useful 2) identifying and 3) very expensive to spoof. - Purchase and other commercial / financial activity likewise. These are literally expensive, and hence credible signals. - Unless you're spamming your contacts regularly, your social networks and interactions are also highly reliable and difficult to spoof. - Dang would not be pleased by a sudden influx of GPT-3 posters on HN. Similar reactions likely for other online services. - So long as your legitimate / semantically significant activity is present with minimal obfuscation, that signal exists and is easy to tease out. - If your activities are interactive and responsive, rather than scheduled, poisoning is less viable and more visible. Data fuzzing / poisoning works fairly well (though not even always then) for systems such ss numbers stations: broadcast-mode distribution, regularly-timed transmissions, consistently encrypted (no cleartext), no location or movement signals from recipients, and no transactional activities --- whether financial mor social. For all else, not so much. I could see some value to, say, search-engine spamming (with spurious searches), or ads networks (fake clicks). For the rest, it's just plain hard. Think threat models; what are you hoping to accomplish/protect, from whom, at what cost(s)?
- markus_zhang 6y ago
  Interesting analysis. I guess it's going to pretty expensive and impossible technically to protect oneself from a State player, and also very expensive and near-impossible to protect against a big "infrastructure" provider (Google, Apple, Big telecom, etc.). However, it should be doable if we could protect us from malevenent apps or non-state crackers.
  - dredmorbius 6y ago
    it's going to pretty expensive and impossible technically to protect oneself Pretty much. Ultimately this is a problem of law, markets, and norms rather than code (to draw on Lessig's four laws) or individual choice. Code itself may help with the backing of other factors, the more so if technical protections are legally required. A huge problem is in establishing both the harms and relationships. It struck me a few weeks ago that the major ills of information technology most often discussed --- censorship, propaganda, surveillance, and manipulation --- have one common root: monopoly. https://joindiaspora.com/posts/7bfcf170eefc013863fa002590d8e506 https://joindiaspora.com/posts/7bfcf170eefc013863fa002590d8e... Discussed on HN: https://news.ycombinator.com/item?id=24771470 https://news.ycombinator.com/item?id=24771470 (Not entirely original, though I'm unaware of anyone who's put all four together. Tim Wu nailed surveillance in 2013 https://www.newyorker.com/tech/annals-of-technology/why-monopolies-make-spying-easier https://www.newyorker.com/tech/annals-of-technology/why-mono...) And I'm not willingly accepting the fact, though attempting to avoid tracking and surveillence is hard and carries real costs. As with security generally, you can simply raise the costs of tracking, which is effective against most casual or commercially-motivated actors, though not a determined political or personal antagonist. Ultimately making exploitation sufficiently expensive or painful may be necessary. Friends, and lots of them, with capacity to act.
    - BlueTemplar 6y ago
      > censorship, propaganda, surveillance, and manipulation --- have one common root: monopoly > Not entirely original, though I'm unaware of anyone who's put all four together. Looks like Cory Doctorow beat you by a mere couple of months : https://onezero.medium.com/how-to-destroy-surveillance-capitalism-8135e6744d59 https://onezero.medium.com/how-to-destroy-surveillance-capit... (Well, as I remember it, I don't really have time to re-read this whole book now. And it's great to have a synthetic post about the same issue !) I'm very surprised that nobody seems to have linked that in those discussions, especially you !
      - dredmorbius 6y ago
        I'm reading Doctorow's book now, and he gets mention (and commented) in tthe Mastodon thread: https://toot.cat/@dredmorbius/105023745888827646 https://toot.cat/@dredmorbius/105023745888827646 Doctorow also mentions the monopoly-surveillance link. We're both beat by Wu by years, and arguably Zuboff by decades, on that one element. Though Doctorow does spend a lot of time talking abbout AdTech (mostly dismissing its significance, incorrectly IMO), and while he's certainly discussed both censsorsship and propaganda in his work generally, he doesn't quite bring the monopoly aspect and combined interreationship into focus. Some cursory searching through the literature also seems to find this connection missing, though I'd be happy to be shown wrong. (The Wu piece was found after my initial postings, and is absolutely a piece of this.)

Variously: - You're emitting side-channel metadata all the time, most especially location and activity data (especially if you have any mobile devices) which is 1) useful 2) identifying and 3) very expensive to spoof. - Purchase and other commercial / financial activity likewise. These are literally expensive, and hence credible signals. - Unless you're spamming your contacts regularly, your social networks and interactions are also highly reliable and difficult to spoof. - Dang would not be pleased by a sudden influx of GPT-3 posters on HN. Similar reactions likely for other online services. - So long as your legitimate / semantically significant activity is present with minimal obfuscation, that signal exists and is easy to tease out. - If your activities are interactive and responsive, rather than scheduled, poisoning is less viable and more visible. Data fuzzing / poisoning works fairly well (though not even always then) for systems such ss numbers stations: broadcast-mode distribution, regularly-timed transmissions, consistently encrypted (no cleartext), no location or movement signals from recipients, and no transactional activities --- whether financial mor social. For all else, not so much. I could see some value to, say, search-engine spamming (with spurious searches), or ads networks (fake clicks). For the rest, it's just plain hard. Think threat models; what are you hoping to accomplish/protect, from whom, at what cost(s)?

markus_zhang 6y ago
Interesting analysis. I guess it's going to pretty expensive and impossible technically to protect oneself from a State player, and also very expensive and near-impossible to protect against a big "infrastructure" provider (Google, Apple, Big telecom, etc.). However, it should be doable if we could protect us from malevenent apps or non-state crackers.
- dredmorbius 6y ago
  it's going to pretty expensive and impossible technically to protect oneself Pretty much. Ultimately this is a problem of law, markets, and norms rather than code (to draw on Lessig's four laws) or individual choice. Code itself may help with the backing of other factors, the more so if technical protections are legally required. A huge problem is in establishing both the harms and relationships. It struck me a few weeks ago that the major ills of information technology most often discussed --- censorship, propaganda, surveillance, and manipulation --- have one common root: monopoly. https://joindiaspora.com/posts/7bfcf170eefc013863fa002590d8e506 https://joindiaspora.com/posts/7bfcf170eefc013863fa002590d8e... Discussed on HN: https://news.ycombinator.com/item?id=24771470 https://news.ycombinator.com/item?id=24771470 (Not entirely original, though I'm unaware of anyone who's put all four together. Tim Wu nailed surveillance in 2013 https://www.newyorker.com/tech/annals-of-technology/why-monopolies-make-spying-easier https://www.newyorker.com/tech/annals-of-technology/why-mono...) And I'm not willingly accepting the fact, though attempting to avoid tracking and surveillence is hard and carries real costs. As with security generally, you can simply raise the costs of tracking, which is effective against most casual or commercially-motivated actors, though not a determined political or personal antagonist. Ultimately making exploitation sufficiently expensive or painful may be necessary. Friends, and lots of them, with capacity to act.
  - BlueTemplar 6y ago
    > censorship, propaganda, surveillance, and manipulation --- have one common root: monopoly > Not entirely original, though I'm unaware of anyone who's put all four together. Looks like Cory Doctorow beat you by a mere couple of months : https://onezero.medium.com/how-to-destroy-surveillance-capitalism-8135e6744d59 https://onezero.medium.com/how-to-destroy-surveillance-capit... (Well, as I remember it, I don't really have time to re-read this whole book now. And it's great to have a synthetic post about the same issue !) I'm very surprised that nobody seems to have linked that in those discussions, especially you !
    - dredmorbius 6y ago
      I'm reading Doctorow's book now, and he gets mention (and commented) in tthe Mastodon thread: https://toot.cat/@dredmorbius/105023745888827646 https://toot.cat/@dredmorbius/105023745888827646 Doctorow also mentions the monopoly-surveillance link. We're both beat by Wu by years, and arguably Zuboff by decades, on that one element. Though Doctorow does spend a lot of time talking abbout AdTech (mostly dismissing its significance, incorrectly IMO), and while he's certainly discussed both censsorsship and propaganda in his work generally, he doesn't quite bring the monopoly aspect and combined interreationship into focus. Some cursory searching through the literature also seems to find this connection missing, though I'd be happy to be shown wrong. (The Wu piece was found after my initial postings, and is absolutely a piece of this.)