4 ms·
> handle the encryption of every packet passing between servers within and across vast data centres This sounds like it’s doing packet processing in software,
by Cyph0n 7y ago
> handle the encryption of every packet passing between servers within and across vast data centres
This sounds like it’s doing packet processing in software, which doesn’t seem scalable, especially for the traffic volume I would assume Apple handles. Anyone have a clue what kind of traffic volume and bandwidth we’re looking at here?
Granted, I might be overestimating the requirements given the industry I work in (service provider routing).
- stock_toaster 7y ago> This sounds like it’s doing packet processing in software From the article: > Based on a custom implementation of IPsec Seems pretty clear.
- ThePowerOfFuet 7y ago>custom implementation of IPsec What could possibly go wrong.
- kjaftaedi 7y agoI'm sure this is just referring to various 'extension headers' in IPSec, which is how the protocol was designed. Sure, poor implementations can lead to problems, but the same holds true for just about anything.
- thedance 7y agoNone of the big clouds use hardware network encryption. All of the resources in your network are at the leaves, so you might as well use their CPUs to do it. I think Google published that crypto and compression in their internal RPC stack cost them less than 1% of their global CPU resources.
- damagednoob 7y ago> I think Google published that crypto and compression in their internal RPC stack cost them less than 1% of their global CPU resources. Not sure if this makes your point compelling. At Google's scale, if you were able to reduce global CPU usage by 0.1%, that would probably be a massive win.
- thedance 7y agoReduce it how? By deploying expensive, unreliable, and hard-to-manage vendor crypto garbage? That is undoubtedly not a trade that a Google would make.
- Cyph0n 7y agoYou reduce it by using custom routing ASICs that can handle orders of magnitude more traffic than a CPU with equal or lower power consumption. To put things into perspective: Cisco’s latest routing ASIC — the Cisco Silicon One — can handle 10.8 Tbps (in fixed mode). I’m sure Google and others have evaluated this, but it’s just kind of surprising that they opt to do per-packet processing in software. Disclaimer: I write router software at Cisco.
- thedance 7y ago10tb sounds like a big scary number but when you think about it that's only 100 100gb ports. It should be no surprise to you that a company making their own switches and NICs, where every RPC is individually authenticated and possibly also encrypted with session keys specific to the peer has no need of vendor "acceleration". https://cloud.google.com/blog/products/networking/google-cloud-networking-in-depth-how-andromeda-2-2-enables-high-throughput-vms https://cloud.google.com/blog/products/networking/google-clo... https://cloud.google.com/security/encryption-in-transit/application-layer-transport-security https://cloud.google.com/security/encryption-in-transit/appl...
- Cyph0n 7y ago10 Tbps is not a scary number? I am honestly still in awe of how such a small box can route an entire city’s worth of traffic. Different perspectives I suppose. Regarding your points: I am not sure I completely follow. Firstly, as far as I know, Google does not make its own switching or routing ASICs. Secondly, virtually all switching and routing ASICs are highly programmable. So if you need a custom protocol, you can implement it using the vendor’s microcode (e.g., P4). In other words, you are not limited to the protocols that the vendor has implemented. Given the above, I don’t see what kind of technical requirements Google has that would disqualify the use of routing ASICs.
- astrange 7y agoThat doesn't mean you're doing crypto with integer operations. Newer CPUs support AES in hardware.
- thedance 7y agoWell, of course. Practically all of the primitives in openssl boringssl etc are coded in assembly.