4 ms·
Remediation for this vulnerability basically caused complete gridlock for the internal tools at a certain FAANG company today.
by LennyWhiteJr 6y ago
Remediation for this vulnerability basically caused complete gridlock for the internal tools at a certain FAANG company today.
- mkl100 6y agoThe "elite" engineers are using ctypes in production? ctypes has never been considered even remotely secure, it can call any library function, including, drumroll, sprintf!
- saagarjha 6y agoI’m sure the issue was more about patching their Python.
- Twirrim 6y agoThere's hints elsewhere it's Amazon. Their build system is interesting. One of the quirks is you define your application, and every library/package that you depend on. They don't depend on system libraries for their application code. The idea is to get as consistent an operating environment where possible. It's both generally amazing, and an absolute pain in the arse (usually when you least want it to be, because some upstream package changed their dependencies and you end up with version conflicts to unpick). When Heartbleed came out, the patches landed in the Amazon build system for the OpenSSL package something like midnight. By the time I got in to the office in the morning, almost every service had been fully rebuilt with patches, and services that do CI/CD had already had the patches deployed. Services that didn't have CI/CD were already kicking off deployments of their front end fleets. IIRC they were paging teams when relevant packages were complete to make sure deployments got kicked off ASAP. So in this case, someone will have patched Python packages for relevant versions, built an updated version, and everything that depends on it will have immediately recompiled, and from there all the packages that depend on those, and so on down the line. Given how python is used a lot for operations etc. I wouldn't be surprised to find a significant chunk of Amazon got rebuilt today, even in cases where Python wasn't being exposed to external users, or even used by the service directly. That's a lot of components, and probably left zero capacity left for anything unrelated, and no doubt there will be quibbles about the ordering in which things got built.
- kiwijamo 6y agoFAANG?
- TheAdamAndChe 6y agoFacebook, Apple, Amazon, Netflix, Google
- bialpio 6y agoWhat's the one that also includes Microsoft? FANMAG? I recall there was something but can't remember what...
- Tijdreiziger 6y agoIIRC that would be FAMANG, but I like FanMag.
- SturgeonsLaw 6y agoI prefer the Spoonerism but I don't think it'll take off
- grenoire 6y agoThere are other more offensive anagrams I've seen used...
- giantrobot 6y agoIn the early 2000s COBRA reorganized as FAANG in response to changing geopolitical realities.
- basementcat 6y agoWho’s GI Joe now?
- whatshisface 6y ago
- koolba 6y agoI bet it’d be an even bigger deal at a couple financial services companies if they took security updates to internal codebases seriously.
- belval 6y agoThat was handled poorly to say the least.
- logicslave 6y agohahha, I happen to be working on this at said faang...
- saagarjha 6y agoFacebook?
- granzymes 6y agoAmazon.
- biosed 6y agoYip, everyone was paged last night, been in a bad way since. Got grief for commenting as much in thread.