3 ms·

Netflix video delivery. WhatsApp. And countless others.

by assimpleaspossi 3y ago

ladyanita22 3y ago
WhatsApp not anymore, and Netflix uses it only partially (only for CDN I believe, the rest of the infra is Linux)
- assimpleaspossi 3y ago
  "Only partially" is a huge "partial" considering that it's all video content delivery everywhere Netflix operates.
toast0 3y ago
WhatsApp is no longer running FreeBSD. Prior to acquisition, everything was bare metal managed hosting at SoftLayer and we had all FreeBSD except one Linux host for reasons I can't remember (maybe an expirement for calling?). After acquisition, there was a migration to Facebook hosting that included moving to Facebook's flavor of containerized Linux. Not because Linux is better, but to fit better within Facebook's operations[1], and Erlang runs on many platforms, so it was a much smaller effort to get our code running on Linux than to get FB's server management software to work for FreeBSD. Server hardware was quite a bit different, so we had no apples to apples comparisons of which OS was more efficient or whatever else. During initial migration, BEAM support of kqueue was much better than epoll, but that got worked out, and I feel like Linux's memory usage reporting is worse than FreeBSD's, but it's a weakness of both. I was never comfortable in the FB server environment, so I left in late 2019, when the FreeBSD server count was reduced to a small enough number that I ran out of things to do. [1] Much of the server team had experience with acquisitions at Yahoo! and the difficulties of making an operations team focused on one OS support acquired teams on another OS. With the many other technical and policy differences between WA and FB, eliminating the OS difference was an easy choice to reduce friction. Our host count, which was large at SoftLayer, was small at Facebook, even after factoring in increased numbers because the servers were smaller and the operations less stable.

Netflix video delivery. WhatsApp. And countless others.

ladyanita22 3y ago

WhatsApp not anymore, and Netflix uses it only partially (only for CDN I believe, the rest of the infra is Linux)

assimpleaspossi 3y ago
"Only partially" is a huge "partial" considering that it's all video content delivery everywhere Netflix operates.

toast0 3y ago

WhatsApp is no longer running FreeBSD. Prior to acquisition, everything was bare metal managed hosting at SoftLayer and we had all FreeBSD except one Linux host for reasons I can't remember (maybe an expirement for calling?). After acquisition, there was a migration to Facebook hosting that included moving to Facebook's flavor of containerized Linux. Not because Linux is better, but to fit better within Facebook's operations[1], and Erlang runs on many platforms, so it was a much smaller effort to get our code running on Linux than to get FB's server management software to work for FreeBSD. Server hardware was quite a bit different, so we had no apples to apples comparisons of which OS was more efficient or whatever else. During initial migration, BEAM support of kqueue was much better than epoll, but that got worked out, and I feel like Linux's memory usage reporting is worse than FreeBSD's, but it's a weakness of both. I was never comfortable in the FB server environment, so I left in late 2019, when the FreeBSD server count was reduced to a small enough number that I ran out of things to do. [1] Much of the server team had experience with acquisitions at Yahoo! and the difficulties of making an operations team focused on one OS support acquired teams on another OS. With the many other technical and policy differences between WA and FB, eliminating the OS difference was an easy choice to reduce friction. Our host count, which was large at SoftLayer, was small at Facebook, even after factoring in increased numbers because the servers were smaller and the operations less stable.