5 ms·

My horribly out of date system works, therefore I should never strive to improve it or god forbid update it (since that involves “fucking with it” in ways that

by Operyl 7y ago

My horribly out of date system works, therefore I should never strive to improve it or god forbid update it (since that involves “fucking with it” in ways that can break it from version to version)? That gets you technical debt and that’s not fun.

m0zg 7y ago
I'll tell you more. Much of the world is run by "horribly out of date" systems that nobody has touched in years _because they work_. And it all works fine. No "cloud", no Rust or Go rewrites, no Haskell, no fancy javascript frameworks or anything like that. Just boring ol' files, boring relational DBs with boring schemas, constraints and stored procedures, boring old languages, boring old hardware, boring old operating systems underneath it all. Don't screw with it and it will work for a decade. Start screwing with it and it will be busted every month, like Google Cloud. You can't create "technical debt" if you don't change anything in the first place.
- owenmarshall 7y ago
  I got an email yesterday that told me the boring old HPUX server (which was racked before my intern was even born) barfed all over its boring old 50-pin SCSI drive and ops went scrambling to find one in storage so the boring old Oracle DB that was responsible for production lines running could be recovered. Took us around an hour, cost us a boring 5 figures. Luckily our sysadmin knows how to hide “unused parts” for days like that or we’d have been really in trouble. > You can’t create “technical debt” if you don’t change anything in the first place. Rubbish. The bits really do rot, and if you don’t do _something_ on occasion you end up with an entire data center no one wants to touch because the dust in the servers might be structural at this point. I’m not saying go rewrite your apps against the Kafka instance your junior devs are fucking with, but you have to do something to fight the entropy.
  - m0zg 7y ago
    The first part of your post sounds like a success story to me. You got many years of use out of that server. 5 figures is cheap comparing to the cost (including the collateral damage cost) of a hotshot SWE or devops guy who insists on using the most resume-worthy, most bleeding-edge technology available.
    - owenmarshall 7y ago
      I wish I could upvote you twice - I just finished a multi-week effort to unwind (defuse?) some of the resume-driven architectures that were left behind when resume-driven development was successful.
  - titzer 7y ago
    > you have to do something to fight the entropy. Stuff breaks. So you fix it. Boring old stuff needs fixing too sometimes. Problem is, old stuff gets obsolete, can't get replacement parts, because of progress. (or something). It's the same story since the first looms were made centuries ago. What you can't fix you can't really depend on. Our time scales are just compressed to ridiculousness because the pace of change is off the charts these days. So basically, you can't really depend on anything working more than a few months before falling over. Sucks.
  - kalleboo 7y ago
    There's boring and there's legacy. Legacy is when the hardware is unsupported and doesn't get software/security updates. You don't want to let it become legacy. The counter-story to yours is running that database on MongoDB in the cloud on a cluster. Instead you'd be having crazy MongoDB issues, data inconsistencies, connectivity issues when the cloud is down, etc etc. The solution is somewhere in the middle. You can have modern, supported hardware running a LTS Linux and that counts as boring.
    - owenmarshall 7y ago
      I think you are right. Where I’ve seen success in “the boring middle” is when an appropriate amount of tension exists in the engineering organization: you want some teams and groups pushing to try new things, but they need to push against a boundary - preferably something, not someone - and the boundary should define your organization’s best practices and standards. This way a team doesn’t get to sneak clustered Mongo into the cloud and make your ordering systems talk to it. But over time boring IT turns into legacy, and without some tension to the system pushing it forward your standards end up locking you into legacy forever.
  - pferde 7y ago
    Sounds like you were bitten by the technical debt that was created by placing an important database behind a single point of failure - a single physical server. Of course something would break sooner or later, PA-RISC and Itanium servers were great, but still not magic. Important things go onto clusters, or at least have a (hot or cold) standby server.
- dgacmu 7y ago
  That's not entirely true. you don't have to try to add features in order for the operating environment of a legacy system to change. More users, transaction count fields overflowing, timestamp fuels hard coated without the century or with 32 bit time_t values... Or it may simply not meet the needs of users anymore. I would hardly hold the air traffic control system up as a model to aspire to, for example. The only reason we run the old one is that the upgrade attempts all failed.
  - m0zg 7y ago
    Nothing ever is "entirely" true.
    - why_only_15 7y ago
      Of course - the point this person was making is that this is substantially false in the way they described.
- 8bitchemistry 7y ago
  > You can't create "technical debt" if you don't change anything in the first place. Tell that to your security team.
- rrdharan 7y ago
  Curious how you would imagine handling something like GDPR or SOX compliance in this alternative world you’re proposing. You can’t magically foresee new requirements and new implied complexity for all future time.
  - hdfbdtbcdg 7y ago
    To many European companies GDPR did not really change the operational requirements - only the penalties for not meeting them.
    - owenmarshall 7y ago
      That’s too clever by half. Avoiding substantial financial penalties for not meeting an operational requirement is an operational requirement.
- t0astbread 7y ago
  > Don't screw with it and it will work for a decade. Don't screw with it and it will have security issues after a few months?
- rjf72 7y ago
  COBOL is a big example of this. It's still ubiquitous in many industries where reliability is near the top of priorities. And I imagine in 50 years we'll still very likely still have many critical systems operating on COBOL. I wonder how many will be running 'The Next Big Thing' language/api then...?
cortesoft 7y ago
Technical debt is about the increasing difficulty to add features to a system.... if you aren't adding features, technical debt is not really an issue.

by Operyl 7y ago

m0zg 7y ago

I'll tell you more. Much of the world is run by "horribly out of date" systems that nobody has touched in years _because they work_. And it all works fine. No "cloud", no Rust or Go rewrites, no Haskell, no fancy javascript frameworks or anything like that. Just boring ol' files, boring relational DBs with boring schemas, constraints and stored procedures, boring old languages, boring old hardware, boring old operating systems underneath it all. Don't screw with it and it will work for a decade. Start screwing with it and it will be busted every month, like Google Cloud. You can't create "technical debt" if you don't change anything in the first place.

owenmarshall 7y ago
I got an email yesterday that told me the boring old HPUX server (which was racked before my intern was even born) barfed all over its boring old 50-pin SCSI drive and ops went scrambling to find one in storage so the boring old Oracle DB that was responsible for production lines running could be recovered. Took us around an hour, cost us a boring 5 figures. Luckily our sysadmin knows how to hide “unused parts” for days like that or we’d have been really in trouble. > You can’t create “technical debt” if you don’t change anything in the first place. Rubbish. The bits really do rot, and if you don’t do _something_ on occasion you end up with an entire data center no one wants to touch because the dust in the servers might be structural at this point. I’m not saying go rewrite your apps against the Kafka instance your junior devs are fucking with, but you have to do something to fight the entropy.
- m0zg 7y ago
  The first part of your post sounds like a success story to me. You got many years of use out of that server. 5 figures is cheap comparing to the cost (including the collateral damage cost) of a hotshot SWE or devops guy who insists on using the most resume-worthy, most bleeding-edge technology available.
  - owenmarshall 7y ago
    I wish I could upvote you twice - I just finished a multi-week effort to unwind (defuse?) some of the resume-driven architectures that were left behind when resume-driven development was successful.
- titzer 7y ago
  > you have to do something to fight the entropy. Stuff breaks. So you fix it. Boring old stuff needs fixing too sometimes. Problem is, old stuff gets obsolete, can't get replacement parts, because of progress. (or something). It's the same story since the first looms were made centuries ago. What you can't fix you can't really depend on. Our time scales are just compressed to ridiculousness because the pace of change is off the charts these days. So basically, you can't really depend on anything working more than a few months before falling over. Sucks.
- kalleboo 7y ago
  There's boring and there's legacy. Legacy is when the hardware is unsupported and doesn't get software/security updates. You don't want to let it become legacy. The counter-story to yours is running that database on MongoDB in the cloud on a cluster. Instead you'd be having crazy MongoDB issues, data inconsistencies, connectivity issues when the cloud is down, etc etc. The solution is somewhere in the middle. You can have modern, supported hardware running a LTS Linux and that counts as boring.
  - owenmarshall 7y ago
    I think you are right. Where I’ve seen success in “the boring middle” is when an appropriate amount of tension exists in the engineering organization: you want some teams and groups pushing to try new things, but they need to push against a boundary - preferably something, not someone - and the boundary should define your organization’s best practices and standards. This way a team doesn’t get to sneak clustered Mongo into the cloud and make your ordering systems talk to it. But over time boring IT turns into legacy, and without some tension to the system pushing it forward your standards end up locking you into legacy forever.
- pferde 7y ago
  Sounds like you were bitten by the technical debt that was created by placing an important database behind a single point of failure - a single physical server. Of course something would break sooner or later, PA-RISC and Itanium servers were great, but still not magic. Important things go onto clusters, or at least have a (hot or cold) standby server.
dgacmu 7y ago
That's not entirely true. you don't have to try to add features in order for the operating environment of a legacy system to change. More users, transaction count fields overflowing, timestamp fuels hard coated without the century or with 32 bit time_t values... Or it may simply not meet the needs of users anymore. I would hardly hold the air traffic control system up as a model to aspire to, for example. The only reason we run the old one is that the upgrade attempts all failed.
- m0zg 7y ago
  Nothing ever is "entirely" true.
  - why_only_15 7y ago
    Of course - the point this person was making is that this is substantially false in the way they described.
8bitchemistry 7y ago
> You can't create "technical debt" if you don't change anything in the first place. Tell that to your security team.
rrdharan 7y ago
Curious how you would imagine handling something like GDPR or SOX compliance in this alternative world you’re proposing. You can’t magically foresee new requirements and new implied complexity for all future time.
- hdfbdtbcdg 7y ago
  To many European companies GDPR did not really change the operational requirements - only the penalties for not meeting them.
  - owenmarshall 7y ago
    That’s too clever by half. Avoiding substantial financial penalties for not meeting an operational requirement is an operational requirement.
t0astbread 7y ago
> Don't screw with it and it will work for a decade. Don't screw with it and it will have security issues after a few months?
rjf72 7y ago
COBOL is a big example of this. It's still ubiquitous in many industries where reliability is near the top of priorities. And I imagine in 50 years we'll still very likely still have many critical systems operating on COBOL. I wonder how many will be running 'The Next Big Thing' language/api then...?

cortesoft 7y ago

Technical debt is about the increasing difficulty to add features to a system.... if you aren't adding features, technical debt is not really an issue.