4 ms·
Adventures of Linux Userspace at Meta [video]
- theyinwhy 3y agoMeta at CCC, I did not think that was possible.
- jaeckel 3y agoTheir media hosting platform is used by many conferences. That's a video of a non-CCC conference, called asg, earlier in 2023, not to be confused with the recently held congress or other CCC headlined events.
- dontupvoteme 3y agoIt's changed a lot in the last few years. Rather captured one might say.
- pengaru 3y agoMeta was a primary sponsor for the most recent All-Systems-Go, and employs several folks contributing upstream to systemd. Also note Lennart is no longer even at RedHat, he's at Microsoft nowadays. I'm surprised Microsoft wasn't a major All-Systems-Go sponsor alongside Meta... There's been a lot of earth shifting under these projects, for better or worse.
- theyinwhy 3y agoWas All Systems Go organized by CCC? I don't find this information on their website. My comment was not about Meta sponsoring stuff. It was about the long history of CCC being against companies like Meta and my puzzlement that they provide such a company a platform.
- dfc 3y agohttps://all-systems-go.io/ https://all-systems-go.io/
- JustAPerson 3y agoSlides: https://cfp.all-systems-go.io/media/all-systems-go-2023/submissions/YAHVZG/resources/ASG_2023_Adventures_of_Linux_Userspace__lLHnm5Y.pdf https://cfp.all-systems-go.io/media/all-systems-go-2023/subm...
- pixelbeat__ 3y agoIt's worth noting that this discusses the centos based provisioning and hardware management platform. The actual distribution used to _run_ all Meta backend services is completely separate and built from source (it does share the (non centos) kernel). This is done for flexibility, performance, service isolation reasons
- loeg 3y ago> The actual distribution used to _run_ all Meta backend services is completely separate and built from source (it does share the kernel). That does not seem to be true. The hosts I'm looking at have systemd and glibc RPMs that were built on centos.org hosts, with coreutils from redhat.com. The kernel was built on a Facebook host, but that's it (of the handful of components I've spot-checked).
- codetrotter 3y agoMaybe they just don’t trust you enough and all of the machines you think are physical hosts that you ssh into are in fact nothing but containers, and there is a whole other team of people at said company controlling the real servers :thinking_face:
- pixelbeat__ 3y agoUse ldd on any service binary to see that it's linked against a separate distro. (It's best to use the ldd from the corresponding platform). There are actually three distros on each host. Current runtime platform, previous runtime platform, centos platform.
- trws 3y agoFor cases where you might not know which ldd will actually work, I like to use patchelf or readelf to get the interpreter then use the —list arg directly. That way it always actually gives correct results, using a different loader can change the selected library paths even if it thinks it works. One-liner would be approximately $(patchelf —print-interpreter tgt) —list tgt My apologies if the dashes became an em-dash or similar, on a phone.
- otterley 3y agoI'm curious as to why Meta settled on CentOS Stream. Isn't that unstable by design? How do they manage change in such a system? What are its benefits and drawbacks vis-a-vis the alternatives?
- leoh 3y agoIt may be due to them testing fairly rigorously before deployment and thereby benefitting from the latest kernel security patches, for example. Google does something similar with Debian internally on workstations.
- matthews2 3y agoCentOS Stream isn’t unstable in the way that something like Arch Linux or Debian Sid is. They effectively only got rid of point releases. Instead of going from CentOS 7 to 7.1, you just regularly get updates. Since they’re a part of the same major release, they don’t contain breaking changes. It’s no different from running Debian Stable with the -updates repository enabled.
- elromulous 3y agoJust to add to this. It's basically a way to get a rolling release. Google did the same thing with Debian and rodete (ROlling DEbian TEsting). https://cloud.google.com/blog/topics/developers-practitioners/how-google-got-to-rolling-linux-releases-for-desktops https://cloud.google.com/blog/topics/developers-practitioner...
- softirq 3y agoBut it does mean that the kernel ABI is never stable for kernel modules. In RHEL, the ABI is frozen for point releases.
- michel-slm 3y agoRight. But it does not matter in this case - Meta rolls out its own kernel (we employ a lot of kernel engineers, and PSA our Linux teams are hiring software engineers!) and we maintain our systemd backports and some other packages in the CentOS Hyperscale SIG
- brcmthrowaway 3y agoOut of 1 million standard LAMP stack machines, how many will randomly fail?
- dijit 3y agoOn a long enough timeline? All of them.
- pi-e-sigma 3y agoOnly if you assume they would be kept up and running until they fail instead of simply being decommissioned at some point and turned into e-junk.
- yjftsjthsd-h 3y agoFor that to be true requires a really long timeline.
- quickthrower2 3y agoIf the failures are independent random events, yes.
- jamesbfb 3y agoI’ve been really enjoying this year’s batch ccc video's. I hope some decent English translations start to crop up for the non-English presentations. The Breaking Train DRM presentation was fantastic!