9 ms·

A Little-Known Company That Enables Mass Surveillance

127 points by a-no-n 10y ago

reirob 10y ago
"[..]Alongside its government clients, Endace has many major corporate customers. Endace’s sales lists include finance industry giants such as Morgan Stanley, Reuters, and Bank of America. [..]" What for do finance companies need systems that intercept data?
- q-base 10y ago
  You could probably sell just about anything to those under a FOMO umbrella.
- ajdlinux 10y ago
  I would assume internal network security and audit trail purposes.
- tptacek 10y ago
  To monitor their own networks, which are gigantic. The top tier of the finance market had networks so complicated and sensitive that they would surprise us and buy Arbor Networks products that were designed for AT&T's backbone. Some of them even needed the BGP monitoring stuff we'd been doing. These companies (Reuters included) basically run their own facsimiles of the Internet, but to carry money instead of cat pictures. Their network monitoring needs are intense.
deleted 10y ago
[deleted]
Ganoes47 10y ago
Can someone ELI5 how they actually capture all these data ? Are they basically cracking encryption ? I thought the kind of encryption provided by VPN services (256-bit AES/CBC) was strong enough? If that's what they do, aren't they violating privacy laws ? Aren't they breaching companies such as Google, Facebook etc... T&C ? Are they installing some kinds of trojans, keyloggers and stuff on 3rd parties computers ? Isn't what they are selling black hat hacking solutions ? Or are they only capturing clear traffic ? which is not necessarily very meaningful. It says : "extract information about people’s usage of services such as Gmail, Hotmail, WhatsApp, and Facebook" The latest terms and conditions you had to acknowledge recently to continue using watsapp (yes, I read them!) mentionned that they don't keep a record of the content being exchanged via watsapp. So, is watsapp lying ? Or what does this Endace system records ? Watsapp T&C also say that they use a strong encryption. so, FTW?
- rkallos 10y ago
  It seems they can only guess at the contents of encrypted packets. https://www.endace.com/deep-packet-inspection.html https://www.endace.com/deep-packet-inspection.html
- tptacek 10y ago
  They can't do anything with encrypted data, except (badly) try to detect that it's encrypted in the first place.
deleted 10y ago
[deleted]
b0ner_t0ner 10y ago
Are we still safe using Tor?
- Ganoes47 10y ago
  Are we even safe with a VPN ?
  - strathmeyer 10y ago
    A VPN is someone else's computer. Why would that be safer than using your own?
    - q-base 10y ago
      Because of eavesdropping on the output of set computer does not tell anything about who is making the actual request? Or are you implying that it will be possible to backtrack with this type of surveillance?
      - criddell 10y ago
        No, I think he was asking how you can trust that the VPN end point you are connected to hasn't been hacked.
brador 10y ago
Ponder: Is your available ISP speed restricted, through backdoor channels and red letters, to not overcome the throughput bandwidth of these devices? Or, another way, is the maximum throughput of these monitoring setups limiting ISP maximum offered speeds in the countries that use them? US, UK, Aus, Canada, the eyes, all have unusually low maximum consumer speeds vs. non implicated countries such as Japan, korea, even China, given the technology available today.
- Ganoes47 10y ago
  It's difficult to go lower than what Australia offers in terms of BW. In Australia, in 2016, you are among the privileged ones if you have like 6Mbps DL at home.
jgrahamc 10y ago
These DPI companies always make me smile because 20 years ago I was the inventor on this patent: https://patents.google.com/patent/US6182146B1/en https://patents.google.com/patent/US6182146B1/en It describes a way of doing DPI to identify protocols that are not running on standard ports. We used this for a protocol analysis product [0] that did network monitoring (for accounting purposes inside companies and led to companies discovering what people were mis-using their network connections for---hello PointCast[1]) and for prediction of network scaling needs. And all that was based on stuff I'd been doing from about 1984 [2]. Bottom line: scooping up packets is easy; encrypt your shit. [0] https://www.cnet.com/au/news/tool-gauges-web-apps/ https://www.cnet.com/au/news/tool-gauges-web-apps/ [1] https://en.wikipedia.org/wiki/PointCast_(dotcom) https://en.wikipedia.org/wiki/PointCast_(dotcom) [2] http://blog.jgc.org/2011/01/network-protocol-analysis-prior-art.html http://blog.jgc.org/2011/01/network-protocol-analysis-prior-...
tptacek 10y ago
This article perfectly illustrates a major flaw in surveillance journalism. As luck would have it, I'm pretty familiar with Endace --- or was, back in 2003-2005. I was at Arbor Networks then. Arbor does large-scale network instrumentation for anti-DDoS and performance monitoring. By the time I left, every major ISP in the world had their network instrumented with Arbor gear. We'd had lots of conversations with Endace. We were as a firm extremely interested in any technology we could buy off the rack to get performant access to raw packets and telemetry data --- Arbor had no hardware engineers, and everything they shipped at the time shipped on COTS X86 rackmounts running OpenBSD. My point here is not just that there are multiple uses for the kind of stuff Endace makes, but also that I vividly remember Endace because very few companies made products in this space at all. Obviously, any company that can facilitate efficient access to, storage of, and analysis of raw traffic data is going to have multiple markets to sell to. And we should not make apologies for companies that take the extra money --- sell their souls, so to speak --- by offering their products to facilitate dragnet surveillance. We would all do well to keep in mind that the problem with selling to this market is far worse than NSA's abuses, which are trivial compared to the abuses perpetrated by countries in the Middle East and Asia. Point being: packaging and selling for the global surveillance market is ethically hazardous in the extreme. No, the problem here is that this kind of story is unintentionally deceptive about who the real enablers of large-scale surveillance are. They're not the dinky little company in New Zealand selling packet capture technology. They're the networking and database giants, the companies our parents automatically have their retirement accounts invested in because they're huge components of the stock market, who have entire teams of people, euphemistically named (maybe something like "public sector" or "APAC public sector" or "GSA" or "defense"), packaging and selling 8-9 figure "solutions" to government around the world. Compared the giants, Endace is a gnat. They're not the enablers. We know who the real enablers are. You can tell, because of the article's lurid descriptions of Endace's major transactions with GCHQ --- the focus of the article. They've got smoking gun proof: invoices for $300,000 and $160,000. Or: less than SourceFire would have charged Chick-Fil-A† to install commercial Snort boxes. † I have no idea if Chick-Fil-A was a SourceFire customer.
- tedunangst 10y ago
  If we're going to hunt all the witches, may as well practice on the little ones.
jlgaddis 10y ago
Gigamon, who has both an office in Milpitas and a government sales division (including for overseas governments), makes photonic tapping systems that are also pretty useful for this mass surveillance kind of stuff. Five years ago, they had a box capable of handling 1 Tbps and assured us that bigger, beefier ones were coming. US DoD was a customer then and, I'm sure, still is.
omgtehlion 10y ago
[slightly offtopic] Does anyone have a driver for Endace DAG 9.2X and is willing to share? )

127 points by a-no-n 10y ago

"[..]Alongside its government clients, Endace has many major corporate customers. Endace’s sales lists include finance industry giants such as Morgan Stanley, Reuters, and Bank of America. [..]" What for do finance companies need systems that intercept data?

q-base 10y ago
You could probably sell just about anything to those under a FOMO umbrella.
ajdlinux 10y ago
I would assume internal network security and audit trail purposes.
tptacek 10y ago
To monitor their own networks, which are gigantic. The top tier of the finance market had networks so complicated and sensitive that they would surprise us and buy Arbor Networks products that were designed for AT&T's backbone. Some of them even needed the BGP monitoring stuff we'd been doing. These companies (Reuters included) basically run their own facsimiles of the Internet, but to carry money instead of cat pictures. Their network monitoring needs are intense.

deleted 10y ago

[deleted]

Ganoes47 10y ago

Can someone ELI5 how they actually capture all these data ? Are they basically cracking encryption ? I thought the kind of encryption provided by VPN services (256-bit AES/CBC) was strong enough? If that's what they do, aren't they violating privacy laws ? Aren't they breaching companies such as Google, Facebook etc... T&C ? Are they installing some kinds of trojans, keyloggers and stuff on 3rd parties computers ? Isn't what they are selling black hat hacking solutions ? Or are they only capturing clear traffic ? which is not necessarily very meaningful. It says : "extract information about people’s usage of services such as Gmail, Hotmail, WhatsApp, and Facebook" The latest terms and conditions you had to acknowledge recently to continue using watsapp (yes, I read them!) mentionned that they don't keep a record of the content being exchanged via watsapp. So, is watsapp lying ? Or what does this Endace system records ? Watsapp T&C also say that they use a strong encryption. so, FTW?

rkallos 10y ago
It seems they can only guess at the contents of encrypted packets. https://www.endace.com/deep-packet-inspection.html https://www.endace.com/deep-packet-inspection.html
tptacek 10y ago
They can't do anything with encrypted data, except (badly) try to detect that it's encrypted in the first place.

deleted 10y ago

[deleted]

b0ner_t0ner 10y ago

Are we still safe using Tor?

Ganoes47 10y ago
Are we even safe with a VPN ?
- strathmeyer 10y ago
  A VPN is someone else's computer. Why would that be safer than using your own?
  - q-base 10y ago
    Because of eavesdropping on the output of set computer does not tell anything about who is making the actual request? Or are you implying that it will be possible to backtrack with this type of surveillance?
    - criddell 10y ago
      No, I think he was asking how you can trust that the VPN end point you are connected to hasn't been hacked.

brador 10y ago

Ponder: Is your available ISP speed restricted, through backdoor channels and red letters, to not overcome the throughput bandwidth of these devices? Or, another way, is the maximum throughput of these monitoring setups limiting ISP maximum offered speeds in the countries that use them? US, UK, Aus, Canada, the eyes, all have unusually low maximum consumer speeds vs. non implicated countries such as Japan, korea, even China, given the technology available today.

Ganoes47 10y ago
It's difficult to go lower than what Australia offers in terms of BW. In Australia, in 2016, you are among the privileged ones if you have like 6Mbps DL at home.

jgrahamc 10y ago

These DPI companies always make me smile because 20 years ago I was the inventor on this patent: https://patents.google.com/patent/US6182146B1/en https://patents.google.com/patent/US6182146B1/en It describes a way of doing DPI to identify protocols that are not running on standard ports. We used this for a protocol analysis product [0] that did network monitoring (for accounting purposes inside companies and led to companies discovering what people were mis-using their network connections for---hello PointCast[1]) and for prediction of network scaling needs. And all that was based on stuff I'd been doing from about 1984 [2]. Bottom line: scooping up packets is easy; encrypt your shit. [0] https://www.cnet.com/au/news/tool-gauges-web-apps/ https://www.cnet.com/au/news/tool-gauges-web-apps/ [1] https://en.wikipedia.org/wiki/PointCast_(dotcom) https://en.wikipedia.org/wiki/PointCast_(dotcom) [2] http://blog.jgc.org/2011/01/network-protocol-analysis-prior-art.html http://blog.jgc.org/2011/01/network-protocol-analysis-prior-...

tptacek 10y ago

This article perfectly illustrates a major flaw in surveillance journalism. As luck would have it, I'm pretty familiar with Endace --- or was, back in 2003-2005. I was at Arbor Networks then. Arbor does large-scale network instrumentation for anti-DDoS and performance monitoring. By the time I left, every major ISP in the world had their network instrumented with Arbor gear. We'd had lots of conversations with Endace. We were as a firm extremely interested in any technology we could buy off the rack to get performant access to raw packets and telemetry data --- Arbor had no hardware engineers, and everything they shipped at the time shipped on COTS X86 rackmounts running OpenBSD. My point here is not just that there are multiple uses for the kind of stuff Endace makes, but also that I vividly remember Endace because very few companies made products in this space at all. Obviously, any company that can facilitate efficient access to, storage of, and analysis of raw traffic data is going to have multiple markets to sell to. And we should not make apologies for companies that take the extra money --- sell their souls, so to speak --- by offering their products to facilitate dragnet surveillance. We would all do well to keep in mind that the problem with selling to this market is far worse than NSA's abuses, which are trivial compared to the abuses perpetrated by countries in the Middle East and Asia. Point being: packaging and selling for the global surveillance market is ethically hazardous in the extreme. No, the problem here is that this kind of story is unintentionally deceptive about who the real enablers of large-scale surveillance are. They're not the dinky little company in New Zealand selling packet capture technology. They're the networking and database giants, the companies our parents automatically have their retirement accounts invested in because they're huge components of the stock market, who have entire teams of people, euphemistically named (maybe something like "public sector" or "APAC public sector" or "GSA" or "defense"), packaging and selling 8-9 figure "solutions" to government around the world. Compared the giants, Endace is a gnat. They're not the enablers. We know who the real enablers are. You can tell, because of the article's lurid descriptions of Endace's major transactions with GCHQ --- the focus of the article. They've got smoking gun proof: invoices for $300,000 and $160,000. Or: less than SourceFire would have charged Chick-Fil-A† to install commercial Snort boxes. † I have no idea if Chick-Fil-A was a SourceFire customer.

tedunangst 10y ago
If we're going to hunt all the witches, may as well practice on the little ones.

jlgaddis 10y ago

Gigamon, who has both an office in Milpitas and a government sales division (including for overseas governments), makes photonic tapping systems that are also pretty useful for this mass surveillance kind of stuff. Five years ago, they had a box capable of handling 1 Tbps and assured us that bigger, beefier ones were coming. US DoD was a customer then and, I'm sure, still is.

omgtehlion 10y ago

[slightly offtopic] Does anyone have a driver for Endace DAG 9.2X and is willing to share? )