6 ms·
Meta built large-scale cryptographic monitoring
- cactusplant7374 2y ago> Since there is a limit to how much data a symmetric cryptographic key can protect Is this regulatory related or self imposed?
- tptacek 2y agoNeither: it's because (overgeneralizing) there are bounds on the number of times nonces can be generated under a key without risk of collision, which in AEAD constructions is a devastating event.
- jgalt212 2y agoFor a given nonce collision, how many blocks or messages can be decrypted?
- richardwhiuk 2y agoFor some modes, you can break every message encrypted with that key.
- deleted 2y ago[deleted]
- HeatrayEnjoyer 2y agoBut shouldn't that bound be astronomical?
- stavros 2y agoI'm fairly out of my depth here, but the birthday paradox is a harsh mistress.
- ikiris 2y agoEspecially when the pool of possibilities is equally massive
- tptacek 2y agoGCM is like 4 billion messages, which is not astronomical at Facebook's scale.
- pstoll 2y agoMeta (we) process many billions of messages per second. Finding enough randomness is a thing at scale.
- HeatrayEnjoyer 2y agoPer day? but not per second. would require every human being sending @ message every 8 seconds
- pstoll 2y agoPer second. Messages in the Kafka sense not the WhatsApp sense.
- bdd 2y agoIf you want to read more about cryptographic key wear out, Soatok wrote an excellent explainer on this a few years ago https://soatok.blog/2020/12/24/cryptographic-wear-out-for-symmetric-encryption/ https://soatok.blog/2020/12/24/cryptographic-wear-out-for-sy...
- smitelli 2y agoScribe[1], Scuba[2], Hive[3], and folly[4]. It's neat for sure, but speaking as an outsider without the resources to learn (much less create!) an entire universe of data infrastructure and SDKs, it feels like I'm in the corner eating dinner at the kids' table. At any rate, I'm glad somebody out there is doing this stuff. [1]: https://engineering.fb.com/2019/10/07/core-infra/scribe/ https://engineering.fb.com/2019/10/07/core-infra/scribe/ [2]: https://research.facebook.com/publications/scuba-diving-into-data-at-facebook/ https://research.facebook.com/publications/scuba-diving-into... [3]: https://research.facebook.com/publications/hive-a-warehousing-solution-over-a-map-reduce-framework/ https://research.facebook.com/publications/hive-a-warehousin... [4]: https://github.com/facebook/folly/tree/main https://github.com/facebook/folly/tree/main
- deleted 2y ago[deleted]
- paxys 2y agoYou can absolutely learn it. Elasticsearch, Logstash, Kibana, Grafana, Prometheus. Set up a local Docker Compose deployment of the full stack and pass through some dummy events. Write an app to generate synthetic load. Play around with aggregations and visualizations. Tech blogs use a lot of buzzwords but you just need to know the fundamental concepts.
- deleted 2y ago[deleted]
- bagels 2y agoWhat it's like to be a Facebook engineer in this context: It's your oncall week, and all of a sudden you get an urgent task that you must resolve in the next few days: Figure out how to replace a crypto algorithm in some component your code depends on or else we tell your manager that you're not playing ball.
- sharpy 2y agoNever worked at facebook, but typically at a large companies, things like this are sent with instructions on how to go about doing that, and most often just making sure your dependencies are updated so next run of automated build/deploy will pick up the change.
- pdpi 2y agoWhen I was there, the experience here would've been more like "you see some comms on Workplace saying this thing went live". For a small number of people, it would look like "It's your oncall week, and you unexpectedly receive a large diff stack from a complete stranger". Most engineers at Meta won't touch anything related to C++ during their tenure there, and you can safely assume all this stuff would be abstracted away from them. If you are on a C++ project, odds are that your experience here would be "I'm on-call and got pinged by a random person saying they were submitting a diff stack to implement this feature". Except if your project was sufficiently large, you'd probably have known about this ages ago so the diff wouldn't actually be unexpected, nor the submitter a complete stranger.
- yodsanklai 2y ago> you'd probably have known about this ages but nobody paid attention and it falls on oncall's plate
- bagels 2y agoSure, sometimes I got a bunch of automod diffs. Sometimes it was: fix this now. It's impossible to know everything that every infra team is cooking up.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]