4 ms·

Two thoughts: 1. GP makes me wonder whether the FreeBSD definition of LTS is less than people expect. 2. Maybe to properly admin a modern FreeBSD box the admi

by ianai 5y ago

Two thoughts: 1. GP makes me wonder whether the FreeBSD definition of LTS is less than people expect. 2. Maybe to properly admin a modern FreeBSD box the admin is supposed to be making the underlying system version easily replaceable. i.e. Make the actual OS files changing have no effect upon hosted system/application stability.

toast0 5y ago
> Maybe to properly admin a modern FreeBSD box the admin is supposed to be making the underlying system version easily replaceable. This is how things worked at Yahoo and WhatsApp. For the most part, a server kept whatever version of FreeBSD it was installed with, and the application logic lived in /usr/local/ or /home/y/ or /home/whatsapp/ Keep your 'build' machine to compile production executables on the oldest OS release of your fleet, and everything should run everywhere. (You'll need to use the compat-X stuff so your FreeBSD X+1 machines can run FreeBSD X binaries, but that's easy) You can use stuff from the base system, but not if you expect recent changes. I don't remember ever doing an OS upgrade at Yahoo, although it must have happened. In my group, we only got updates with new hardware; which was fine, hardware was changing rapidly. At WhatsApp, I drove the OS updates towards the end of WhatsApp's use of FreeBSD. A fair amount was just not wanting to have 4 different major versions out there, but there were also forced updates to get important network bugfixes or performance improvements. Sometimes it was easier to find the handful of boxes that were too old to use something nice in a script and update them, rather than fixing the script to not use the nice feature. When we ran into bugs on older versions, upgrading to a version that already had it fixed was sometimes an option, but sometimes it'd need to be backported, and sometimes the bug was still in -current. We didn't run into too many of those though, our local patch set was never more than about 10 patches.
- deleted 5y ago
  [deleted]
- darkhelmet 5y ago
  At Yahoo, there was a tool that did in-place OS upgrades for FreeBSD, but you're right - in most cases, people installed whatever was current of the day and then relied on yinst to make it work. ybsdupgrade did work and was mostly a non-event. I don't recall whether we had it work across major version upgrades though.
  - toast0 5y ago
    Not surprised there was a tool, but I don't remember it. :) most of my time at Yahoo was on 4.x, so minor upgrades only would have been fine. (Some 6 and 7 towards the end, but also a little bit of RHEL, which I was happy to get away from)
    - ianai 5y ago
      I wonder if FreeBSD would benefit from some documentation or packaging to make this more of a covered area? Personally, the times I upgraded a FreeBSD box the process felt very smooth. To the point that I’m trying to understand @rsyncs point and seem to be missing something critical.
      - toast0 5y ago
        FreeBSD has a tool for binary upgrades now, and has had a process for source based upgrades forever. This is all documented in the FreeBSD Handbook, and I'd be surprised if the source based upgrade wasn't in there since the beginning of the Handbook, which I think came out with FreeBSD-3. But, darkhelmet and I were discussing updating FreeBSD at Yahoo; doing the update from the Handbook wouldn't be the right way to do it, because you would miss out on Yahoo specific patches, which were likely required for some key Yahoo software. I don't think it was common knowledge how to do it, so some documentation might have been nice, but mostly teams at Yahoo just didn't do it at that time for better or worse.
        ianai 5y ago
        The tooling at Y! sounded quite good from the descriptions. Kind of confirmed my thoughts that the proper way to run a FreeBSD system is to make hopping versions not affect hosted applications through specific tooling and/or infrastructure. Especially the bit about building on older versions and such. Not really sure what more would address the version complaints from @rsync.
      - throw0101a 5y ago
        > I wonder if FreeBSD would benefit from some documentation or packaging to make this more of a covered area? FreBSD also supports in-place upgrades for major version numbers: * https://docs.freebsd.org/en/books/handbook/cutting-edge/ https://docs.freebsd.org/en/books/handbook/cutting-edge/ * https://www.cyberciti.biz/open-source/freebsd-13-released-how-to-update-upgrade-freebsd-12-to-13/ https://www.cyberciti.biz/open-source/freebsd-13-released-ho... There could be policy reasons why someone may decent not to do it: if you have a large fleet of cattle machines it may be easier to just redeploy from scratch. Home(lab) users and pet machines at companies may do otherwise.
- oblio 5y ago
  > At WhatsApp, I drove the OS updates towards the end of WhatsApp's use of FreeBSD. What did they move to? And do you know why they switched?
  - iv42 5y ago
    Linux, as part of integration with Facebook's infrastructure: https://news.ycombinator.com/item?id=22028689 https://news.ycombinator.com/item?id=22028689

by ianai 5y ago

toast0 5y ago

> Maybe to properly admin a modern FreeBSD box the admin is supposed to be making the underlying system version easily replaceable. This is how things worked at Yahoo and WhatsApp. For the most part, a server kept whatever version of FreeBSD it was installed with, and the application logic lived in /usr/local/ or /home/y/ or /home/whatsapp/ Keep your 'build' machine to compile production executables on the oldest OS release of your fleet, and everything should run everywhere. (You'll need to use the compat-X stuff so your FreeBSD X+1 machines can run FreeBSD X binaries, but that's easy) You can use stuff from the base system, but not if you expect recent changes. I don't remember ever doing an OS upgrade at Yahoo, although it must have happened. In my group, we only got updates with new hardware; which was fine, hardware was changing rapidly. At WhatsApp, I drove the OS updates towards the end of WhatsApp's use of FreeBSD. A fair amount was just not wanting to have 4 different major versions out there, but there were also forced updates to get important network bugfixes or performance improvements. Sometimes it was easier to find the handful of boxes that were too old to use something nice in a script and update them, rather than fixing the script to not use the nice feature. When we ran into bugs on older versions, upgrading to a version that already had it fixed was sometimes an option, but sometimes it'd need to be backported, and sometimes the bug was still in -current. We didn't run into too many of those though, our local patch set was never more than about 10 patches.

deleted 5y ago
[deleted]
darkhelmet 5y ago
At Yahoo, there was a tool that did in-place OS upgrades for FreeBSD, but you're right - in most cases, people installed whatever was current of the day and then relied on yinst to make it work. ybsdupgrade did work and was mostly a non-event. I don't recall whether we had it work across major version upgrades though.
- toast0 5y ago
  Not surprised there was a tool, but I don't remember it. :) most of my time at Yahoo was on 4.x, so minor upgrades only would have been fine. (Some 6 and 7 towards the end, but also a little bit of RHEL, which I was happy to get away from)
  - ianai 5y ago
    I wonder if FreeBSD would benefit from some documentation or packaging to make this more of a covered area? Personally, the times I upgraded a FreeBSD box the process felt very smooth. To the point that I’m trying to understand @rsyncs point and seem to be missing something critical.
    - toast0 5y ago
      FreeBSD has a tool for binary upgrades now, and has had a process for source based upgrades forever. This is all documented in the FreeBSD Handbook, and I'd be surprised if the source based upgrade wasn't in there since the beginning of the Handbook, which I think came out with FreeBSD-3. But, darkhelmet and I were discussing updating FreeBSD at Yahoo; doing the update from the Handbook wouldn't be the right way to do it, because you would miss out on Yahoo specific patches, which were likely required for some key Yahoo software. I don't think it was common knowledge how to do it, so some documentation might have been nice, but mostly teams at Yahoo just didn't do it at that time for better or worse.
      - ianai 5y ago
        The tooling at Y! sounded quite good from the descriptions. Kind of confirmed my thoughts that the proper way to run a FreeBSD system is to make hopping versions not affect hosted applications through specific tooling and/or infrastructure. Especially the bit about building on older versions and such. Not really sure what more would address the version complaints from @rsync.
    - throw0101a 5y ago
      > I wonder if FreeBSD would benefit from some documentation or packaging to make this more of a covered area? FreBSD also supports in-place upgrades for major version numbers: * https://docs.freebsd.org/en/books/handbook/cutting-edge/ https://docs.freebsd.org/en/books/handbook/cutting-edge/ * https://www.cyberciti.biz/open-source/freebsd-13-released-how-to-update-upgrade-freebsd-12-to-13/ https://www.cyberciti.biz/open-source/freebsd-13-released-ho... There could be policy reasons why someone may decent not to do it: if you have a large fleet of cattle machines it may be easier to just redeploy from scratch. Home(lab) users and pet machines at companies may do otherwise.
oblio 5y ago
> At WhatsApp, I drove the OS updates towards the end of WhatsApp's use of FreeBSD. What did they move to? And do you know why they switched?
- iv42 5y ago
  Linux, as part of integration with Facebook's infrastructure: https://news.ycombinator.com/item?id=22028689 https://news.ycombinator.com/item?id=22028689