11 ms·

All these comments about metadata not being useful are missing the point. Metadata is incredible valuable and sometimes just as valuable as the decrypted data i

All these comments about metadata not being useful are missing the point. Metadata is incredible valuable and sometimes just as valuable as the decrypted data itself. Knowing what sites a target visits, access patterns, changes in behavior: all this can be fed into ML algorithms to come up with fingerprints. You don't need to be able decrypt the data in transit if you know the endpoints and can somehow compromise the endpoints at a later date. And that is way easier. Breaking encryption is hard and time consuming. Identifying a site a user regularly visits and exploiting that is more straightforward.

pfortuny 6y ago
People have gone to jail for metadata. That is exactly what you are saying. That is its importance. And that is only speaking of something within the Rule of Law (accessing metadata with a warrant)... Outside of the Rule of Law, people have been killed for metadata.
- deleted 6y ago
  [deleted]
- e12e 6y ago
  Ed: > Outside of the Rule of Law, people have been killed for metadata. Indeed: https://theintercept.com/2014/02/10/the-nsas-secret-role/ https://theintercept.com/2014/02/10/the-nsas-secret-role/ > According to a former drone operator for the military’s Joint Special Operations Command (JSOC) who also worked with the NSA, the agency often identifies targets based on controversial metadata analysis and cell-phone tracking technologies. Rather than confirming a target’s identity with operatives or informants on the ground, the CIA or the U.S. military then orders a strike based on the activity and location of the mobile phone a person is believed to be using.
  - pfortuny 6y ago
    Thanks. Too lazy to search for specific examples. Also, obviously, Mafias and the USSR, PRC...
  - zo1 6y ago
    That's actually pretty good policing. They should apply that domestically for non-terrorist violent criminals. +1 they have my vote. (Obviously not on the "drop random bombs on them" part.)
cblconfederate 6y ago
I will randomly upvote or downvote a link here and there just to confuse them
gaius_baltar 6y ago
> Metadata is incredible valuable and sometimes just as valuable as the decrypted data itself. Just adding an example for the people who don't see the value of metadata: WhatsApp is still a viable revenue source for Facebook even as they have no access to the text of the messages due to E2EE. Knowing who talks to who, at what times, the type and approximate size of messages, the members of groups, and the contents of the phone book of every user gives enough information to keep their business model without exposing them to court orders asking for the plaintext (that's the reason they added E2EE to start with, there is no incentive to improve the service when they have a billion heads of cattle to milk).
- Threeve303 6y ago
  Every so often I like to go back and read Using Metadata to find Paul Revere [0]. [0] https://kieranhealy.org/blog/archives/2013/06/09/using-metadata-to-find-paul-revere/ https://kieranhealy.org/blog/archives/2013/06/09/using-metad...
- just-ok 6y ago
  > they have no access to the text of the messages due to E2EE. Correction: they might not have access to the message text. It's entirely possible (if not plausible: FB doesn't exactly have a good track record) for FB to just self-MitM the E2EE and see everything that passes through their servers. From their site: > The verification process is optional for end-to-end encrypted chats, and only used to confirm that the messages and calls you send are end-to-end encrypted. Even this process--which I'm sure very few people do--is fallible given the lack of authenticity: there's no way to confirm that the given keys are what's actually used for encryption. Yes, this may come across as very "tinfoil-hat-y," but do you really trust FB to not be exploring every possible avenue to increase their data streams?
  - samsonradu 6y ago
    > It's entirely possible (if not plausible: FB doesn't exactly have a good track record) for FB to just self-MitM the E2EE and see everything that passes through their servers. Why would they even need to MitM in transit when they control the endpoints? They can just analyze the raw text locally (in the app) and extract valuable information.
deleted 6y ago
[deleted]
josh2600 6y ago
Just chiming in here: it’s almost all about the graph. If you have the graph, the content is almost irrelevant. This is why Signal hiding the graph as best they can, using SGX, is incredibly important work. Say what you want about Secure Enclaves, we know of no better way to conceal social graphs. Yes there is still potentially some metadata analysis that can be done at the server to coordinate IP addresses but we know signal doesn’t keep those logs because of their response to the sealed subpoena (which they successfully sued with the ACLU to unseal): https://signal.org/bigbrother/eastern-virginia-grand-jury/ https://signal.org/bigbrother/eastern-virginia-grand-jury/ We can only dream of a world where companies are held to this standard of transparency and user privacy.
- upofadown 6y ago
  >...we know signal doesn’t keep those logs because of their response to the sealed subpoena ... That doesn't prove that. If Signal was, say, a NSA project they would have to respond to such things in that way to protect the signal intelligence value of the metadata they were collecting for their primary mission. After Crypto AG we know it is a bad idea to trust any particular entity. Something like Signal can only be trusted as much as can verified.
  - josh2600 6y ago
    Absolutely. You should trust anything as much as you can verify it and no further. I submit that there is no better option right now.
    - upofadown 6y ago
      If you are not trusting the people that are running these things, then Signal is just another siloed messenger where the servers are controlled by a single entity. There are certainly worse but Signal is not special.
      - monocasa 6y ago
        Signal has open clients with reproducible builds. We know that they are keeping their promises wrt what information is communicated with the backends. That's a step above the other options in common use, and in fact does make Signal special.
virtue3 6y ago
I can't believe that anyone that was around here during the snowden stuff hitting the fan would even remotely say metadata isn't useful. "Law enforcement agencies have claimed that metadata helps to eliminate suspects by revealing their networks and contacts. But there is no information regarding the use of metadata by government bodies that are not officially enforcement agencies within the meaning of the data retention laws." https://theconversation.com/think-your-metadata-is-only-visible-to-national-security-agencies-think-again-121253 https://theconversation.com/think-your-metadata-is-only-visi...
godelski 6y ago
It honestly worries me that this is the top comment on hacker news. Not because it is wrong (it isn't) but because of all places that website filled with tech workers and experts in the full software stack, full of people that work on and exploit meta data, it still needs to be discussed how important metadata is. If we can't convince people with their ear to the ground, how does one convince the general public. Especially since it isn't intuitive how metadata is useful. Though the analogy I typically use is a private investigator following you around. Can't hear your conversations, but can see everyone you talk to, where, and for how long.
- rosywoozlechan 6y ago
  > If we can't convince people with their ear to the ground, how does one convince the general public. Convince them of what? Some of us don't believe the NSA are bad actors and and possibly we also believe they're doing their jobs and support them in that.
  - lostcolony 6y ago
    "The NSA" - who do you mean here? The org in its official function doing unofficial things without oversight? Or the individual working for the NSA spying on his ex-lover for blackmail material? I mean, either you're saying "no one within the NSA has ever been a bad actor", or you're saying "the bad actions are acceptable collateral damage; no oversight needs to be applied to ensure the trade off between effectiveness and collateral damage is balanced", or you're saying "not ALL actors are bad" and leaving it at that. And...none of those strikes me as a particular defensible position to take.
    - deleted 6y ago
      [deleted]
  - Shared404 6y ago
    You're one of the people this [0] comment is talking about. Also, how can you possibly believe that the NSA are not bad actors? Between trying to hobble encryption, spying on everything, and enabling bad individual actions, and having a horrible success rate [1], what is left to defend? [0] https://news.ycombinator.com/item?id=24962802 https://news.ycombinator.com/item?id=24962802 [1] https://www.newamerica.org/international-security/policy-papers/do-nsas-bulk-surveillance-programs-stop-terrorists/ https://www.newamerica.org/international-security/policy-pap...
13415 6y ago
Both you and these commenters are missing the point. They're not just collecting metadata. We know from the Snowden leaks that the NSA was able to decrypt most https traffic as well as most SSH and VPN traffic around 2013. Although protocol security has been beefed up a bit and many bugs have been weeded out since then, it's still naive to assume they've lost this capability.
upofadown 6y ago
In general, entities like the NSA need to treat metadata as important because that is often all they have. That is because most everything is encrypted these days. The NSA has known about the "going dark" problem for a long time now and this is the reaction. So this situation can be considered a sort of a triumph. For most people metadata is no real threat to them. Generally it is already publicly known who your friends and family are and those are the people most interact with online. It is mostly valuable that no one else know what those interactions are even if they know when they occurred. For the important instance of businesses the situation is much the same although sometimes there might be value in traffic analysis for larger businesses that have enough traffic to analyze.
sneak 6y ago
Michael Hayden, former director of the NSA and CIA: “We kill people based on metadata.” https://youtu.be/PxwEwwlDM8Q https://youtu.be/PxwEwwlDM8Q (39s clip)
pengstrom 6y ago
Isn't the whole distinction between data and metadata rather arbitrary in this context?
azernik 6y ago
Forget ML; just a queryable database where your analysts can plug in a known surveillance target and see who they're talking to has lots of value. (IIUC, that's Palantir's original core product, not anything in the ML space.)
neolog 6y ago
> All these comments about metadata not being useful I only see one
jmnicolas 6y ago
To reinforce your point: > Ex-NSA Chief: 'We Kill People Based on Metadata' https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-we-kill-people-based-on-metadata https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-...

pfortuny 6y ago

People have gone to jail for metadata. That is exactly what you are saying. That is its importance. And that is only speaking of something within the Rule of Law (accessing metadata with a warrant)... Outside of the Rule of Law, people have been killed for metadata.

deleted 6y ago
[deleted]
e12e 6y ago
Ed: > Outside of the Rule of Law, people have been killed for metadata. Indeed: https://theintercept.com/2014/02/10/the-nsas-secret-role/ https://theintercept.com/2014/02/10/the-nsas-secret-role/ > According to a former drone operator for the military’s Joint Special Operations Command (JSOC) who also worked with the NSA, the agency often identifies targets based on controversial metadata analysis and cell-phone tracking technologies. Rather than confirming a target’s identity with operatives or informants on the ground, the CIA or the U.S. military then orders a strike based on the activity and location of the mobile phone a person is believed to be using.
- pfortuny 6y ago
  Thanks. Too lazy to search for specific examples. Also, obviously, Mafias and the USSR, PRC...
- zo1 6y ago
  That's actually pretty good policing. They should apply that domestically for non-terrorist violent criminals. +1 they have my vote. (Obviously not on the "drop random bombs on them" part.)

cblconfederate 6y ago

I will randomly upvote or downvote a link here and there just to confuse them

gaius_baltar 6y ago

> Metadata is incredible valuable and sometimes just as valuable as the decrypted data itself. Just adding an example for the people who don't see the value of metadata: WhatsApp is still a viable revenue source for Facebook even as they have no access to the text of the messages due to E2EE. Knowing who talks to who, at what times, the type and approximate size of messages, the members of groups, and the contents of the phone book of every user gives enough information to keep their business model without exposing them to court orders asking for the plaintext (that's the reason they added E2EE to start with, there is no incentive to improve the service when they have a billion heads of cattle to milk).

Threeve303 6y ago
Every so often I like to go back and read Using Metadata to find Paul Revere [0]. [0] https://kieranhealy.org/blog/archives/2013/06/09/using-metadata-to-find-paul-revere/ https://kieranhealy.org/blog/archives/2013/06/09/using-metad...
just-ok 6y ago
> they have no access to the text of the messages due to E2EE. Correction: they might not have access to the message text. It's entirely possible (if not plausible: FB doesn't exactly have a good track record) for FB to just self-MitM the E2EE and see everything that passes through their servers. From their site: > The verification process is optional for end-to-end encrypted chats, and only used to confirm that the messages and calls you send are end-to-end encrypted. Even this process--which I'm sure very few people do--is fallible given the lack of authenticity: there's no way to confirm that the given keys are what's actually used for encryption. Yes, this may come across as very "tinfoil-hat-y," but do you really trust FB to not be exploring every possible avenue to increase their data streams?
- samsonradu 6y ago
  > It's entirely possible (if not plausible: FB doesn't exactly have a good track record) for FB to just self-MitM the E2EE and see everything that passes through their servers. Why would they even need to MitM in transit when they control the endpoints? They can just analyze the raw text locally (in the app) and extract valuable information.

deleted 6y ago

[deleted]

josh2600 6y ago

Just chiming in here: it’s almost all about the graph. If you have the graph, the content is almost irrelevant. This is why Signal hiding the graph as best they can, using SGX, is incredibly important work. Say what you want about Secure Enclaves, we know of no better way to conceal social graphs. Yes there is still potentially some metadata analysis that can be done at the server to coordinate IP addresses but we know signal doesn’t keep those logs because of their response to the sealed subpoena (which they successfully sued with the ACLU to unseal): https://signal.org/bigbrother/eastern-virginia-grand-jury/ https://signal.org/bigbrother/eastern-virginia-grand-jury/ We can only dream of a world where companies are held to this standard of transparency and user privacy.

upofadown 6y ago
>...we know signal doesn’t keep those logs because of their response to the sealed subpoena ... That doesn't prove that. If Signal was, say, a NSA project they would have to respond to such things in that way to protect the signal intelligence value of the metadata they were collecting for their primary mission. After Crypto AG we know it is a bad idea to trust any particular entity. Something like Signal can only be trusted as much as can verified.
- josh2600 6y ago
  Absolutely. You should trust anything as much as you can verify it and no further. I submit that there is no better option right now.
  - upofadown 6y ago
    If you are not trusting the people that are running these things, then Signal is just another siloed messenger where the servers are controlled by a single entity. There are certainly worse but Signal is not special.
    - monocasa 6y ago
      Signal has open clients with reproducible builds. We know that they are keeping their promises wrt what information is communicated with the backends. That's a step above the other options in common use, and in fact does make Signal special.

virtue3 6y ago

I can't believe that anyone that was around here during the snowden stuff hitting the fan would even remotely say metadata isn't useful. "Law enforcement agencies have claimed that metadata helps to eliminate suspects by revealing their networks and contacts. But there is no information regarding the use of metadata by government bodies that are not officially enforcement agencies within the meaning of the data retention laws." https://theconversation.com/think-your-metadata-is-only-visible-to-national-security-agencies-think-again-121253 https://theconversation.com/think-your-metadata-is-only-visi...

godelski 6y ago

It honestly worries me that this is the top comment on hacker news. Not because it is wrong (it isn't) but because of all places that website filled with tech workers and experts in the full software stack, full of people that work on and exploit meta data, it still needs to be discussed how important metadata is. If we can't convince people with their ear to the ground, how does one convince the general public. Especially since it isn't intuitive how metadata is useful. Though the analogy I typically use is a private investigator following you around. Can't hear your conversations, but can see everyone you talk to, where, and for how long.

rosywoozlechan 6y ago
> If we can't convince people with their ear to the ground, how does one convince the general public. Convince them of what? Some of us don't believe the NSA are bad actors and and possibly we also believe they're doing their jobs and support them in that.
- lostcolony 6y ago
  "The NSA" - who do you mean here? The org in its official function doing unofficial things without oversight? Or the individual working for the NSA spying on his ex-lover for blackmail material? I mean, either you're saying "no one within the NSA has ever been a bad actor", or you're saying "the bad actions are acceptable collateral damage; no oversight needs to be applied to ensure the trade off between effectiveness and collateral damage is balanced", or you're saying "not ALL actors are bad" and leaving it at that. And...none of those strikes me as a particular defensible position to take.
  - deleted 6y ago
    [deleted]
- Shared404 6y ago
  You're one of the people this [0] comment is talking about. Also, how can you possibly believe that the NSA are not bad actors? Between trying to hobble encryption, spying on everything, and enabling bad individual actions, and having a horrible success rate [1], what is left to defend? [0] https://news.ycombinator.com/item?id=24962802 https://news.ycombinator.com/item?id=24962802 [1] https://www.newamerica.org/international-security/policy-papers/do-nsas-bulk-surveillance-programs-stop-terrorists/ https://www.newamerica.org/international-security/policy-pap...

13415 6y ago

Both you and these commenters are missing the point. They're not just collecting metadata. We know from the Snowden leaks that the NSA was able to decrypt most https traffic as well as most SSH and VPN traffic around 2013. Although protocol security has been beefed up a bit and many bugs have been weeded out since then, it's still naive to assume they've lost this capability.

upofadown 6y ago

In general, entities like the NSA need to treat metadata as important because that is often all they have. That is because most everything is encrypted these days. The NSA has known about the "going dark" problem for a long time now and this is the reaction. So this situation can be considered a sort of a triumph. For most people metadata is no real threat to them. Generally it is already publicly known who your friends and family are and those are the people most interact with online. It is mostly valuable that no one else know what those interactions are even if they know when they occurred. For the important instance of businesses the situation is much the same although sometimes there might be value in traffic analysis for larger businesses that have enough traffic to analyze.

sneak 6y ago

Michael Hayden, former director of the NSA and CIA: “We kill people based on metadata.” https://youtu.be/PxwEwwlDM8Q https://youtu.be/PxwEwwlDM8Q (39s clip)

pengstrom 6y ago

Isn't the whole distinction between data and metadata rather arbitrary in this context?

azernik 6y ago

Forget ML; just a queryable database where your analysts can plug in a known surveillance target and see who they're talking to has lots of value. (IIUC, that's Palantir's original core product, not anything in the ML space.)

neolog 6y ago

> All these comments about metadata not being useful I only see one

jmnicolas 6y ago

To reinforce your point: > Ex-NSA Chief: 'We Kill People Based on Metadata' https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-we-kill-people-based-on-metadata https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-...