5 ms·

The Feds Cracked El Chapo's Encrypted Comms Network by Flipping His System Admin

89 points by lnguyen 8y ago

morley 8y ago
I'm curious about this encrypted VoIP network. I've never heard of an off-the-shelf product like that, so was it custom-written? What was the interface like? The source NYT article has some more details, but not enough to answer these questions: One of Mr. Guzmán’s Colombian suppliers, Jorge Cifuentes, who introduced the kingpin to the I.T. expert, testified last month that Mr. Rodriguez had promised to arrange secure communications for what amounted to the entire cartel’s leadership. His system operated on VoiP, or voice over internet protocol, Mr. Marston said on Tuesday, and was accessible only to those within the network. According to Mr. Cifuentes, Mr. Guzmán was able to sign in through Wi-Fi even from his hide-outs in the Sierra Madre mountains. https://www.nytimes.com/2019/01/08/nyregion/el-chapo-trial.html https://www.nytimes.com/2019/01/08/nyregion/el-chapo-trial.h...
- segfaultbuserr 8y ago
  Why can't it be a common off-the-shelf VoIP server with a custom deployment? e.g. a Mumble server behind TLS with client authorization?
- dylz 8y ago
  I would imagine this is just a standard open source SIP server or something plus VPN, with encryption turned on.
- newman314 8y ago
  IMO, the article was a little breathless in its reporting. I wonder if the encryption here is just SRTP/ZRTP.
- nimbius 8y ago
  Asterisk has had SIP/TLS for a 5 years now. Ive tried it a few times, and its not too bad with ECDSA or smaller RSA keys. https://wiki.asterisk.org/wiki/display/AST/SIP+TLS+Transport https://wiki.asterisk.org/wiki/display/AST/SIP+TLS+Transport
- Kenji 8y ago
  Easy. Just make a mumble server. The server is authenticated with a public key and the communication is encrypted. The only way to listen to the communication is if you pwn one of the clients or the server. Btw I can really recommend mumble. At this stage it is plug & play and the quality is really good. I set it up for my parents (on my own mumble server running on my linux server) and we only mumble call these days.
- jasonjayr 8y ago
  I have a tinc + asterisk setup to my cell phone (for my personal/home line) and openvpn + asterisk to desk Yealink phones (They have built-in openvpn clients) (for business) Fairly easy to maintain + grow once you get some basics out of the way. The net result is that all the signaling + transport is encrypted as far as non-VPN nodes are concerned.
  - addajones 8y ago
    I’d love to know about this setup! What hardware you run it on and where you get the #’s from for the home line. Trying to set up something like this for myself at home and business. Thank you!!
    - jasonjayr 8y ago
      The business line has a T1, and a Sangoma Vega 100 T1/SIP gateway, with SIP trunking service from https://voip.ms https://voip.ms as a backup if there's trouble w/ the T1. (My personal phone service is just voip.ms at this point) Yealink T23P Phones have a OpenVPN client built in (some newer Grandstream phones do too), these are relatively inexpensive VoIP desk phones. Once you configure them, and an OpenVPN server you can plug them anywhere you have internet (NAT'ed or otherwise), and not have to worry about NAT Traversal, or other issues. Asterisk is just running on a stock PC. I'm running on a Supermicro Board with an Atom processor, and for our call volume (50 extension phones, 23 inbound lines, 30 on an inbound call queue, 3 simultaneous calls average, 15 @ our known peak, no transcoding (all uLaw)). The OpenVPN server is a a separate machine, for lighter usage the VPN + Asterisk could probably be the same machine. For the cell phone, I have CSipSimple running there, with a OpenVPN client to connect to the network. In my configuration, the phones + Asterisk are not using SSL/TLS directly, but the VPN secures the traffic over untrusted LANs. Calls within the system are on the protected LAN, but once they reach out to the PSTN, all bets are off. I've setup a smaller office with a Raspberry Pi 3, and a Grandstream SIP/Analog Gateway, ( 7 Grandstream phones, 3 lines, no transcoding, not very heavy use), and they haven't had any complaints. (see http://www.raspberry-asterisk.org/ http://www.raspberry-asterisk.org/) If they used SIP trunking over their internet connection, they woulden't need the SIP/Analog gateway, which was the single most expensive piece of equipment in this setup (@ US$399-ish)
- Liblor 8y ago
  According to this [1] dutch article, they used a private Blackberry server. [1] https://www.volkskrant.nl/nieuws-achtergrond/nederlandse-politie-tapte-anderhalf-jaar-lang-alle-communicatie-van-mexicaanse-drugsbaron-el-chapo-~bab33a30/ https://www.volkskrant.nl/nieuws-achtergrond/nederlandse-pol...
paxys 8y ago
Reminds me of https://xkcd.com/538/ https://xkcd.com/538/ No matter the technological sophistication, humans will always be the weakest link in any secure system.
- 1001101 8y ago
  This is a form of: https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis
- unparagoned 8y ago
  God, I recognise that specific comic just from the number in the url...
giancarlostoro 8y ago
I remember an article on HN about disappearing techs who were kidnapped by cartels to build them a custom telecoms network or something. Wondering how related this could be to that. Edit - found the article: https://news.ycombinator.com/item?id=9145003 https://news.ycombinator.com/item?id=9145003
jotm 8y ago
Oh they turned him to their side. I was wondering how he could give away anything while they showed him the finger(s) :D
- onetimemanytime 8y ago
  DEA /FBI offers /can offer millions to cooperating witnesses and a new identity. Or life in prison. Or death in the hands of the cartel if they find out about the chat. They kinda make it very easy to choose. Jorge Salcedo that took down Cali cartel got million$ for his cooperation.
  - Apocryphon 8y ago
    The question is, how do these people avoid vengeful followers of the former cartel bosses?
    - jplayer01 8y ago
      I assume getting paid is combined with witness protection, or you'd be right, they're screwed.
    - gh02t 8y ago
      Presumably, the same way Mafia informants do - witness protection. The government spirits them away and sets them up with an entirely new identity. There have been some leaks, but generally these people just disappear and become someone else. https://en.wikipedia.org/wiki/United_States_Federal_Witness_Protection_Program https://en.wikipedia.org/wiki/United_States_Federal_Witness_...
    - onetimemanytime 8y ago
      You're dead anyway. Or jailed and possibly be killed there. With USA on your side you have a better chance of living a life close to normal, relatively speaking. I guess that Sinaloa will put this guy #1 or close to it (maybe the Flores twins beat them to #1) on their revenge list. 20-50 kilos of cocaine to the sicarios...a drop in the bucket since they buy for $2k a kilo or so.
ChrisArchitect 8y ago
servers were in Canada?? interesting
HashThis 8y ago
The battleground in crypto is so often around crypto key management. With the Snowden revelations, it was obvious that the NSA strategy is going after keys in targets key management systems first. Then cryptoanalysis becomes O(n)
rollulus 8y ago
The article mentions that the server was moved to the Netherlands, but not the reason. According to the Dutch press, it was because the FBI asked so, because we're not so difficult installing wire taps on the internet [1]. [1]: https://www.volkskrant.nl/nieuws-achtergrond/nederlandse-politie-tapte-anderhalf-jaar-lang-alle-communicatie-van-mexicaanse-drugsbaron-el-chapo-~bab33a30/ https://www.volkskrant.nl/nieuws-achtergrond/nederlandse-pol...
carlosdp 8y ago
> Amid the accounts of corruption, murder, and drug smuggling, Vice News’ Keegan Hamilton wrote on Twitter, there was a brief moment of levity when lights in the courtroom went out. When the electricity returned, someone shouted “He’s gone!”, referring to Guzmán’s habit of escaping from prison. “Everybody laughed, except maybe the U.S. Marshals,” Hamilton wrote. It's interesting how even such a serious situation as this can have these moments of laughter.
exabrial 8y ago
I hope this guy is in witness protection!
qrbLPHiKpiux 8y ago
The feds didn't crack, I recall in the NYT article, the encryption keys were handed over from someone who flipped.
takinola 8y ago
Asking for a ... friend. What is the mitigation against attacks like this? Seriously, it would appear that every organization is at risk of having a trusted insider hand over keys to a competitor, criminals, etc. There must be a way to detect or protect against this, right?
- wmf 8y ago
  Dual control (aka the two-man rule), separation of duties (person who requests a change isn't allowed to approve/implement it), mandatory vacation, etc. These policies have existing in fields like finance for a while.
- CodeWriter23 8y ago
  Don't do the crime if you can't do the time.
eoinclancy1 8y ago
Actually there are a few providers that do encrypted VoIP, or more specifically encrypted SIP. Telnyx offer a private infrastructure deployed around the world with low latency and the really good call quality. https://telnyx.com/solutions/global-ip-network?utm_source=organic_social&utm_medium=hackernews_organic&utm_campaign=encrypted-voip-january-2019 https://telnyx.com/solutions/global-ip-network?utm_source=or...
TheMagicHorsey 8y ago
Well, we know at least one guy who is going to be dead shortly.

89 points by lnguyen 8y ago

I'm curious about this encrypted VoIP network. I've never heard of an off-the-shelf product like that, so was it custom-written? What was the interface like? The source NYT article has some more details, but not enough to answer these questions: One of Mr. Guzmán’s Colombian suppliers, Jorge Cifuentes, who introduced the kingpin to the I.T. expert, testified last month that Mr. Rodriguez had promised to arrange secure communications for what amounted to the entire cartel’s leadership. His system operated on VoiP, or voice over internet protocol, Mr. Marston said on Tuesday, and was accessible only to those within the network. According to Mr. Cifuentes, Mr. Guzmán was able to sign in through Wi-Fi even from his hide-outs in the Sierra Madre mountains. https://www.nytimes.com/2019/01/08/nyregion/el-chapo-trial.html https://www.nytimes.com/2019/01/08/nyregion/el-chapo-trial.h...

segfaultbuserr 8y ago
Why can't it be a common off-the-shelf VoIP server with a custom deployment? e.g. a Mumble server behind TLS with client authorization?
dylz 8y ago
I would imagine this is just a standard open source SIP server or something plus VPN, with encryption turned on.
newman314 8y ago
IMO, the article was a little breathless in its reporting. I wonder if the encryption here is just SRTP/ZRTP.
nimbius 8y ago
Asterisk has had SIP/TLS for a 5 years now. Ive tried it a few times, and its not too bad with ECDSA or smaller RSA keys. https://wiki.asterisk.org/wiki/display/AST/SIP+TLS+Transport https://wiki.asterisk.org/wiki/display/AST/SIP+TLS+Transport
Kenji 8y ago
Easy. Just make a mumble server. The server is authenticated with a public key and the communication is encrypted. The only way to listen to the communication is if you pwn one of the clients or the server. Btw I can really recommend mumble. At this stage it is plug & play and the quality is really good. I set it up for my parents (on my own mumble server running on my linux server) and we only mumble call these days.
jasonjayr 8y ago
I have a tinc + asterisk setup to my cell phone (for my personal/home line) and openvpn + asterisk to desk Yealink phones (They have built-in openvpn clients) (for business) Fairly easy to maintain + grow once you get some basics out of the way. The net result is that all the signaling + transport is encrypted as far as non-VPN nodes are concerned.
- addajones 8y ago
  I’d love to know about this setup! What hardware you run it on and where you get the #’s from for the home line. Trying to set up something like this for myself at home and business. Thank you!!
  - jasonjayr 8y ago
    The business line has a T1, and a Sangoma Vega 100 T1/SIP gateway, with SIP trunking service from https://voip.ms https://voip.ms as a backup if there's trouble w/ the T1. (My personal phone service is just voip.ms at this point) Yealink T23P Phones have a OpenVPN client built in (some newer Grandstream phones do too), these are relatively inexpensive VoIP desk phones. Once you configure them, and an OpenVPN server you can plug them anywhere you have internet (NAT'ed or otherwise), and not have to worry about NAT Traversal, or other issues. Asterisk is just running on a stock PC. I'm running on a Supermicro Board with an Atom processor, and for our call volume (50 extension phones, 23 inbound lines, 30 on an inbound call queue, 3 simultaneous calls average, 15 @ our known peak, no transcoding (all uLaw)). The OpenVPN server is a a separate machine, for lighter usage the VPN + Asterisk could probably be the same machine. For the cell phone, I have CSipSimple running there, with a OpenVPN client to connect to the network. In my configuration, the phones + Asterisk are not using SSL/TLS directly, but the VPN secures the traffic over untrusted LANs. Calls within the system are on the protected LAN, but once they reach out to the PSTN, all bets are off. I've setup a smaller office with a Raspberry Pi 3, and a Grandstream SIP/Analog Gateway, ( 7 Grandstream phones, 3 lines, no transcoding, not very heavy use), and they haven't had any complaints. (see http://www.raspberry-asterisk.org/ http://www.raspberry-asterisk.org/) If they used SIP trunking over their internet connection, they woulden't need the SIP/Analog gateway, which was the single most expensive piece of equipment in this setup (@ US$399-ish)
Liblor 8y ago
According to this [1] dutch article, they used a private Blackberry server. [1] https://www.volkskrant.nl/nieuws-achtergrond/nederlandse-politie-tapte-anderhalf-jaar-lang-alle-communicatie-van-mexicaanse-drugsbaron-el-chapo-~bab33a30/ https://www.volkskrant.nl/nieuws-achtergrond/nederlandse-pol...

paxys 8y ago

Reminds me of https://xkcd.com/538/ https://xkcd.com/538/ No matter the technological sophistication, humans will always be the weakest link in any secure system.

1001101 8y ago
This is a form of: https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis
unparagoned 8y ago
God, I recognise that specific comic just from the number in the url...

giancarlostoro 8y ago

I remember an article on HN about disappearing techs who were kidnapped by cartels to build them a custom telecoms network or something. Wondering how related this could be to that. Edit - found the article: https://news.ycombinator.com/item?id=9145003 https://news.ycombinator.com/item?id=9145003

jotm 8y ago

Oh they turned him to their side. I was wondering how he could give away anything while they showed him the finger(s) :D

onetimemanytime 8y ago
DEA /FBI offers /can offer millions to cooperating witnesses and a new identity. Or life in prison. Or death in the hands of the cartel if they find out about the chat. They kinda make it very easy to choose. Jorge Salcedo that took down Cali cartel got million$ for his cooperation.
- Apocryphon 8y ago
  The question is, how do these people avoid vengeful followers of the former cartel bosses?
  - jplayer01 8y ago
    I assume getting paid is combined with witness protection, or you'd be right, they're screwed.
  - gh02t 8y ago
    Presumably, the same way Mafia informants do - witness protection. The government spirits them away and sets them up with an entirely new identity. There have been some leaks, but generally these people just disappear and become someone else. https://en.wikipedia.org/wiki/United_States_Federal_Witness_Protection_Program https://en.wikipedia.org/wiki/United_States_Federal_Witness_...
  - onetimemanytime 8y ago
    You're dead anyway. Or jailed and possibly be killed there. With USA on your side you have a better chance of living a life close to normal, relatively speaking. I guess that Sinaloa will put this guy #1 or close to it (maybe the Flores twins beat them to #1) on their revenge list. 20-50 kilos of cocaine to the sicarios...a drop in the bucket since they buy for $2k a kilo or so.

ChrisArchitect 8y ago

servers were in Canada?? interesting

HashThis 8y ago

The battleground in crypto is so often around crypto key management. With the Snowden revelations, it was obvious that the NSA strategy is going after keys in targets key management systems first. Then cryptoanalysis becomes O(n)

rollulus 8y ago

The article mentions that the server was moved to the Netherlands, but not the reason. According to the Dutch press, it was because the FBI asked so, because we're not so difficult installing wire taps on the internet [1]. [1]: https://www.volkskrant.nl/nieuws-achtergrond/nederlandse-politie-tapte-anderhalf-jaar-lang-alle-communicatie-van-mexicaanse-drugsbaron-el-chapo-~bab33a30/ https://www.volkskrant.nl/nieuws-achtergrond/nederlandse-pol...

carlosdp 8y ago

> Amid the accounts of corruption, murder, and drug smuggling, Vice News’ Keegan Hamilton wrote on Twitter, there was a brief moment of levity when lights in the courtroom went out. When the electricity returned, someone shouted “He’s gone!”, referring to Guzmán’s habit of escaping from prison. “Everybody laughed, except maybe the U.S. Marshals,” Hamilton wrote. It's interesting how even such a serious situation as this can have these moments of laughter.

exabrial 8y ago

I hope this guy is in witness protection!

qrbLPHiKpiux 8y ago

The feds didn't crack, I recall in the NYT article, the encryption keys were handed over from someone who flipped.

takinola 8y ago

Asking for a ... friend. What is the mitigation against attacks like this? Seriously, it would appear that every organization is at risk of having a trusted insider hand over keys to a competitor, criminals, etc. There must be a way to detect or protect against this, right?

wmf 8y ago
Dual control (aka the two-man rule), separation of duties (person who requests a change isn't allowed to approve/implement it), mandatory vacation, etc. These policies have existing in fields like finance for a while.
CodeWriter23 8y ago
Don't do the crime if you can't do the time.

eoinclancy1 8y ago

Actually there are a few providers that do encrypted VoIP, or more specifically encrypted SIP. Telnyx offer a private infrastructure deployed around the world with low latency and the really good call quality. https://telnyx.com/solutions/global-ip-network?utm_source=organic_social&utm_medium=hackernews_organic&utm_campaign=encrypted-voip-january-2019 https://telnyx.com/solutions/global-ip-network?utm_source=or...

TheMagicHorsey 8y ago

Well, we know at least one guy who is going to be dead shortly.