15 ms·

The LTS, long support version and stuff are all confessions of a technical and organisational failures If you are not able to upgrade your stuff every 2 to 3 y

The LTS, long support version and stuff are all confessions of a technical and organisational failures If you are not able to upgrade your stuff every 2 to 3 years, then you will not be able to upgrade your stuff after 5, 10 or 15 years. After so long time, that untouched pill of cruft will be considered as legacy, built by people gone long ago. It will be a massive project, an entire rebuild/refactor/migration of whatever you have. "If you do not know how to do planned maintenance, then you will learn with incidents"

cyanydeez 11mo ago
Sure. But infrastructure will always be seen as a one time cost because enshittifiction ensures every company with merit transitions from merit leaders to MBA leaders. This happens so often its basically a failure of capitalism.
wiseowise 11mo ago
Why do you need to “upgrade your stuff” every 2-3 years?
- JackSlateur 11mo ago
  Why do you need to clean your house every week/couple of weeks ? Why not clean only once a year ? Keeping your infrastructure/code somehow uptodate ensures: - each time you have to upgrade, this is not a big deal - you have less breaking changes at each iteration, thus less work to do - when you must upgrade for some reasons, the step is, again, not so big - you are sure you own the infrastructure. That current people owns it (versus people who left the company 8 years ago) - you benefits from innovation (yes, there is) and/or performance improvements (yes, there is) Keeping your stuff rotting in a dark room brings nothing good
  - exe34 11mo ago
    It didn't need to be this way. It's a choice made by companies who stand to gain from the continuous churn.
  - laurowyn 11mo ago
    Why not think of it a different way; why do we need to put up with breaking changes at all? I'd much rather stand up a replacement system adjacent to the current one, and then switch over, than run the headache of debugging breaking changes every single release. To me, this is the difference between an update and an upgrade. An update just fixes things that are broken. And upgrade adds/removes/changes features from how they were before. I'm all for keeping things up to date. And software vendors should support that as much as possible. But forcing me to deal with a new set of challenges every few weeks is ridiculous. This idea of rapid releases with continuous development is great when that's the fundamental point of the product. But stability is a feature too, and a far more important one in my opinion. I'd much rather a stable platform to build upon, than a rickety one that keeps changing shape every other week that I need to figure out what changed and how that impacts my system, because it means I can spend all of my time _using_ the platform rather than fixing it. This is why bleeding edge releases exist. For people who want the latest and greatest, and are willing to deal with the instability issues and want to help find and squash bugs. For the rest of us, we just want to use the system, not help develop it. Give me a stable system, ship me bug fixes that don't fundamentally break how anything works, and let me focus on my specific task. If that costs money, so be it, but I don't want to have to take one day per week running updates to find something else is broken and have to debug and fix it. That's not what I'm here to do. And as for cleaning the house - we always have the option of hiring a cleaner. That costs us money, but they keep the house cleanliness stable whilst we focus on something else to make enough money to cover the cleaner's cost plus some profit.
    - JackSlateur 11mo ago
      Because many components are "all or nothing" And also because, for the others, you have to migrate everybody from the "old" to the "new"; Large project, low value, nobody cares, "just to your job and don't bother us with your shit"
    - dpoloncsak 11mo ago
      I assume the cost of duplicating every piece of infra we have to create a suitable 'replacement system' would cost more than the impact the few times we have had downtime due to breaking changes in updates. Ymmv ofc
  - epistasis 11mo ago
    Considering "upgrading" to be "cleaning" is very odd. Same with "rotting". Perhaps this is a side effect of dealing with software development ecosystems with huge dependency trees? There's a lot of software not like that at all. No dependencies. No internet connection. No multi kilobyte lock files detailing long lists of continual software churn and bug fixes.
  - Kenji 11mo ago
    I bought a hammer 2 decades ago and it still works flawlessly without me performing any maintenance. One day, the idiots in software engineering will realize that this is how tools are supposed to behave.
  - layer8 11mo ago
    Why would the same exact software be considered “unclean” or “rotten” a few years down the line when it previously wasn’t? What has changed? Did it need to?
  - icedchai 11mo ago
    I've lost count of how many Ubuntu upgrades resulted in some weird problem (network interfaces renamed, lost default route, systemd timeouts taking 5 minutes, etc.) There is an argument for staying on the latest stable version.
    - tokai 11mo ago
      Thats also an argument for not using Ubuntu.
      - icedchai 11mo ago
        When I eventually rebuild things I’ll be going with Debian.
        kasabali 11mo ago
        Unfortunately Debian wouldn't be exempt from "network interfaces renamed, lost default route, systemd timeouts taking 5 minutes, etc." because systemd is systemd (and Debian systemd maintainers just go along with upstream and do nothing to mitigate it) I think interface names changed like at least 2 times in the last 4 releases.
        icedchai 11mo ago
        Maybe? I run Debian on several VMs and my experience has proven otherwise. Major upgrades go a lot smoother than on Ubuntu.
        kasabali 11mo ago
        Smoother, sure. I'm talking about interface name changes. it's nice you haven't encountered these releases but they sure happened in Debian, too.
    - loosescrews 11mo ago
      I think you are talking about an upgrade install. Those have a long history of breaking things. You would have to be crazy to attempt one of those on a critical production system. What you would do for anything important is build a new separate system and then migrate to that once it is working. You can then migrate back if you discover issues too.
  - wiseowise 11mo ago
    > Why do you need to clean your house every week/couple of weeks ? Why not clean only once a year ? OS is not a physical house with life waste. Rest of your message doesn’t make any sense for majority of industry. For anything dealing with manufacturing stability is much more important that marginal performance gains. Any downtime is losing money.
aboringusername 11mo ago
I'm not sure why there's a need to update anything every 2-3 years. In fact, the pace of change becomes exhausting in itself. In my day-to-day life, things are mostly well designed systems and processes; there's a stable code of practice when driving cars, going to the shops, picking up the shopping, paying for the items and then storing them. What part of that process needs to change every 2-3 years? Because some 'angel investor' says we need growth which means pushing updates to make it appear like you're doing something? old.reddit has worked the same for the last 10 years now, new.reddit is absolutely awful. That's what 2-3 years of 'change' gets you. In fact, this website itself remains largely the same. Why change for the sake of it?
- JackSlateur 11mo ago
  In your day-to-day life, you do chore regurarly Why not cleaning the room only once every 2-3 years ?
  - frankchn 11mo ago
    I do chores regularly, and I apply security patches regularly. Major operating system version upgrades can be more akin to upgrading all the furniture and electronics in my house at the same time.
  - exe34 11mo ago
    Why don't you move house every 6 months?
    - JackSlateur 11mo ago
      Upgrade != replace with something new (like using another OS or whatever)
      - numpad0 11mo ago
        update/upgrade == replacing with something worse. "because that's what you do" is not a valid justification.
Y_Y 11mo ago
Consider that the average CTO is about 50† and that roughly people expect to retire at 65 and die at 80. If you can get away with one or zero overhauls of your infra during your tenure then that's probably a hell of a lot easier than every two to three years. † https://www.zippia.com/chief-technology-officer-jobs/demographics/ https://www.zippia.com/chief-technology-officer-jobs/demogra...
da_chicken 11mo ago
I don't agree, and this feels like something written by someone who has never managed actual systems running actual business operations. Operating systems in particular need to manage the hardware, manage memory, manage security, and otherwise absolutely need to shut up and stay out of the fucking way. Established software changes SLOWLY. It doesn't need to reinvent itself with a brand new dichotomy every 3 years. Nobody builds a server because they want to run the latest version of Python. They built it to run the software they bought 10 years ago for $5m and for which they're paying annual support contracts of $50k. They run what the support contracts require them to run, and they don't want to waste time with an OS upgrade because the cost of the downtime is too high and none of the software they use is going to utilize any of the newly available features. All it does is introduce a new way for the system to fail in ways you're not yet familiar with. It adds ZERO value because all we actually want and need is the same shit but with security patches. Genuinely I want HN to understand that not everyone is running a 25 person startup running a microservice they hope to scale to Twitter proportions. Very few people in IT are working in the tech industry. Most IT departments are understaffed and underfunded. If we can save three weeks of time over 10 years by not having to rebuild an entire system every 3 years, it's very much worth it.
- JackSlateur 11mo ago
  Just for the context, I am employed by a multi-billion company (which has more than 100k people) Here, I'm in charge of some low level infrastructure components (the kind on which absolutely everything rely on, 5sec of downtime = 5sec of everything is down) On one of my scope, I've inherited from a 15 years-old junkyard The kind with a yearly support The kind that costs millions The kind that is so complex, that has seen so less evolutions other the years that nobody knows it anymore (even the people who were there 15y ago) The kind that slows everybody else because it cannot meet other teams' needs Long story short, I've got a flamethrower and we are purging everything Management is happy, customers are happy too, my mates also enjoy working with sane tech (and not braindamaged shit)
  - cpncrunch 11mo ago
    Sounds like that is a different issue. I prefer to avoid spending a few weeks migrating software that i understand and support to a new OS when i dont have to. Some of it is 30 years old, but it has had all the bugs worked out.
bityard 11mo ago
You would be amazed how many fortune 500 companies are still using RHEL/CentOS 7 in business critical systems. (I was, anyway.)
- xorcist 11mo ago
  That's .. not the least bit surprising. It's not ancient or anything. It's still under commercial support from the vendor, even if it is sunset.
kwar13 11mo ago
What kind of argument is to "upgrade your stuff every 2 to 3 years". What are you upgrading for? If the software runs fine and does it job without issues, what "stuff" is there to upgrade?
- Nextgrid 11mo ago
  > What are you upgrading for? So that whoever is doing the upgrade can justify their salary and continued employment.
gosub100 11mo ago
This is what someone would say who has never work on anything serious, or in a regulated industry.
- foofoo12 11mo ago
  Yep, let alone life critical systems. You don't fuck with them just because.
dawnerd 11mo ago
Not necessarily. There are cases where hardware support ends and trying to get drivers for a newer kernel is basically impossible without a lot of work. For example, one of my highpoint HBAs are completely unusable without running an older kernel. I imagine there’s more custom designed hardware with the same problem out there.
curt15 11mo ago
>If you are not able to upgrade your stuff every 2 to 3 years, then you will not be able to upgrade your stuff after 5, 10 or 15 years. What happens if your software takes 2 years to develop?
- JackSlateur 11mo ago
  If your software needs the world from stop moving for 2 years so that it can be prepared and released successfully, I am afraid your software will never be released :(
deleted 11mo ago
[deleted]

cyanydeez 11mo ago

Sure. But infrastructure will always be seen as a one time cost because enshittifiction ensures every company with merit transitions from merit leaders to MBA leaders. This happens so often its basically a failure of capitalism.

wiseowise 11mo ago

Why do you need to “upgrade your stuff” every 2-3 years?

JackSlateur 11mo ago
Why do you need to clean your house every week/couple of weeks ? Why not clean only once a year ? Keeping your infrastructure/code somehow uptodate ensures: - each time you have to upgrade, this is not a big deal - you have less breaking changes at each iteration, thus less work to do - when you must upgrade for some reasons, the step is, again, not so big - you are sure you own the infrastructure. That current people owns it (versus people who left the company 8 years ago) - you benefits from innovation (yes, there is) and/or performance improvements (yes, there is) Keeping your stuff rotting in a dark room brings nothing good
- exe34 11mo ago
  It didn't need to be this way. It's a choice made by companies who stand to gain from the continuous churn.
- laurowyn 11mo ago
  Why not think of it a different way; why do we need to put up with breaking changes at all? I'd much rather stand up a replacement system adjacent to the current one, and then switch over, than run the headache of debugging breaking changes every single release. To me, this is the difference between an update and an upgrade. An update just fixes things that are broken. And upgrade adds/removes/changes features from how they were before. I'm all for keeping things up to date. And software vendors should support that as much as possible. But forcing me to deal with a new set of challenges every few weeks is ridiculous. This idea of rapid releases with continuous development is great when that's the fundamental point of the product. But stability is a feature too, and a far more important one in my opinion. I'd much rather a stable platform to build upon, than a rickety one that keeps changing shape every other week that I need to figure out what changed and how that impacts my system, because it means I can spend all of my time _using_ the platform rather than fixing it. This is why bleeding edge releases exist. For people who want the latest and greatest, and are willing to deal with the instability issues and want to help find and squash bugs. For the rest of us, we just want to use the system, not help develop it. Give me a stable system, ship me bug fixes that don't fundamentally break how anything works, and let me focus on my specific task. If that costs money, so be it, but I don't want to have to take one day per week running updates to find something else is broken and have to debug and fix it. That's not what I'm here to do. And as for cleaning the house - we always have the option of hiring a cleaner. That costs us money, but they keep the house cleanliness stable whilst we focus on something else to make enough money to cover the cleaner's cost plus some profit.
  - JackSlateur 11mo ago
    Because many components are "all or nothing" And also because, for the others, you have to migrate everybody from the "old" to the "new"; Large project, low value, nobody cares, "just to your job and don't bother us with your shit"
  - dpoloncsak 11mo ago
    I assume the cost of duplicating every piece of infra we have to create a suitable 'replacement system' would cost more than the impact the few times we have had downtime due to breaking changes in updates. Ymmv ofc
- epistasis 11mo ago
  Considering "upgrading" to be "cleaning" is very odd. Same with "rotting". Perhaps this is a side effect of dealing with software development ecosystems with huge dependency trees? There's a lot of software not like that at all. No dependencies. No internet connection. No multi kilobyte lock files detailing long lists of continual software churn and bug fixes.
- Kenji 11mo ago
  I bought a hammer 2 decades ago and it still works flawlessly without me performing any maintenance. One day, the idiots in software engineering will realize that this is how tools are supposed to behave.
- layer8 11mo ago
  Why would the same exact software be considered “unclean” or “rotten” a few years down the line when it previously wasn’t? What has changed? Did it need to?
- icedchai 11mo ago
  I've lost count of how many Ubuntu upgrades resulted in some weird problem (network interfaces renamed, lost default route, systemd timeouts taking 5 minutes, etc.) There is an argument for staying on the latest stable version.
  - tokai 11mo ago
    Thats also an argument for not using Ubuntu.
    - icedchai 11mo ago
      When I eventually rebuild things I’ll be going with Debian.
      - kasabali 11mo ago
        Unfortunately Debian wouldn't be exempt from "network interfaces renamed, lost default route, systemd timeouts taking 5 minutes, etc." because systemd is systemd (and Debian systemd maintainers just go along with upstream and do nothing to mitigate it) I think interface names changed like at least 2 times in the last 4 releases.
        icedchai 11mo ago
        Maybe? I run Debian on several VMs and my experience has proven otherwise. Major upgrades go a lot smoother than on Ubuntu.
        kasabali 11mo ago
        Smoother, sure. I'm talking about interface name changes. it's nice you haven't encountered these releases but they sure happened in Debian, too.
  - loosescrews 11mo ago
    I think you are talking about an upgrade install. Those have a long history of breaking things. You would have to be crazy to attempt one of those on a critical production system. What you would do for anything important is build a new separate system and then migrate to that once it is working. You can then migrate back if you discover issues too.
- wiseowise 11mo ago
  > Why do you need to clean your house every week/couple of weeks ? Why not clean only once a year ? OS is not a physical house with life waste. Rest of your message doesn’t make any sense for majority of industry. For anything dealing with manufacturing stability is much more important that marginal performance gains. Any downtime is losing money.

aboringusername 11mo ago

I'm not sure why there's a need to update anything every 2-3 years. In fact, the pace of change becomes exhausting in itself. In my day-to-day life, things are mostly well designed systems and processes; there's a stable code of practice when driving cars, going to the shops, picking up the shopping, paying for the items and then storing them. What part of that process needs to change every 2-3 years? Because some 'angel investor' says we need growth which means pushing updates to make it appear like you're doing something? old.reddit has worked the same for the last 10 years now, new.reddit is absolutely awful. That's what 2-3 years of 'change' gets you. In fact, this website itself remains largely the same. Why change for the sake of it?

JackSlateur 11mo ago
In your day-to-day life, you do chore regurarly Why not cleaning the room only once every 2-3 years ?
- frankchn 11mo ago
  I do chores regularly, and I apply security patches regularly. Major operating system version upgrades can be more akin to upgrading all the furniture and electronics in my house at the same time.
- exe34 11mo ago
  Why don't you move house every 6 months?
  - JackSlateur 11mo ago
    Upgrade != replace with something new (like using another OS or whatever)
    - numpad0 11mo ago
      update/upgrade == replacing with something worse. "because that's what you do" is not a valid justification.

Y_Y 11mo ago

Consider that the average CTO is about 50† and that roughly people expect to retire at 65 and die at 80. If you can get away with one or zero overhauls of your infra during your tenure then that's probably a hell of a lot easier than every two to three years. † https://www.zippia.com/chief-technology-officer-jobs/demographics/ https://www.zippia.com/chief-technology-officer-jobs/demogra...

da_chicken 11mo ago

I don't agree, and this feels like something written by someone who has never managed actual systems running actual business operations. Operating systems in particular need to manage the hardware, manage memory, manage security, and otherwise absolutely need to shut up and stay out of the fucking way. Established software changes SLOWLY. It doesn't need to reinvent itself with a brand new dichotomy every 3 years. Nobody builds a server because they want to run the latest version of Python. They built it to run the software they bought 10 years ago for $5m and for which they're paying annual support contracts of $50k. They run what the support contracts require them to run, and they don't want to waste time with an OS upgrade because the cost of the downtime is too high and none of the software they use is going to utilize any of the newly available features. All it does is introduce a new way for the system to fail in ways you're not yet familiar with. It adds ZERO value because all we actually want and need is the same shit but with security patches. Genuinely I want HN to understand that not everyone is running a 25 person startup running a microservice they hope to scale to Twitter proportions. Very few people in IT are working in the tech industry. Most IT departments are understaffed and underfunded. If we can save three weeks of time over 10 years by not having to rebuild an entire system every 3 years, it's very much worth it.

JackSlateur 11mo ago
Just for the context, I am employed by a multi-billion company (which has more than 100k people) Here, I'm in charge of some low level infrastructure components (the kind on which absolutely everything rely on, 5sec of downtime = 5sec of everything is down) On one of my scope, I've inherited from a 15 years-old junkyard The kind with a yearly support The kind that costs millions The kind that is so complex, that has seen so less evolutions other the years that nobody knows it anymore (even the people who were there 15y ago) The kind that slows everybody else because it cannot meet other teams' needs Long story short, I've got a flamethrower and we are purging everything Management is happy, customers are happy too, my mates also enjoy working with sane tech (and not braindamaged shit)
- cpncrunch 11mo ago
  Sounds like that is a different issue. I prefer to avoid spending a few weeks migrating software that i understand and support to a new OS when i dont have to. Some of it is 30 years old, but it has had all the bugs worked out.

bityard 11mo ago

You would be amazed how many fortune 500 companies are still using RHEL/CentOS 7 in business critical systems. (I was, anyway.)

xorcist 11mo ago
That's .. not the least bit surprising. It's not ancient or anything. It's still under commercial support from the vendor, even if it is sunset.

kwar13 11mo ago

What kind of argument is to "upgrade your stuff every 2 to 3 years". What are you upgrading for? If the software runs fine and does it job without issues, what "stuff" is there to upgrade?

Nextgrid 11mo ago
> What are you upgrading for? So that whoever is doing the upgrade can justify their salary and continued employment.

gosub100 11mo ago

This is what someone would say who has never work on anything serious, or in a regulated industry.

foofoo12 11mo ago
Yep, let alone life critical systems. You don't fuck with them just because.

dawnerd 11mo ago

Not necessarily. There are cases where hardware support ends and trying to get drivers for a newer kernel is basically impossible without a lot of work. For example, one of my highpoint HBAs are completely unusable without running an older kernel. I imagine there’s more custom designed hardware with the same problem out there.

curt15 11mo ago

>If you are not able to upgrade your stuff every 2 to 3 years, then you will not be able to upgrade your stuff after 5, 10 or 15 years. What happens if your software takes 2 years to develop?

JackSlateur 11mo ago
If your software needs the world from stop moving for 2 years so that it can be prepared and released successfully, I am afraid your software will never be released :(

deleted 11mo ago

[deleted]