8 ms·

From someone who went through it and dealt with code, it was a real problem but I also think it was handled poorly publicly. The issues were known for a long t

From someone who went through it and dealt with code, it was a real problem but I also think it was handled poorly publicly. The issues were known for a long time, but the media hyped it into a frenzy because a few higher profile companies and a lot of government systems had not been updated. In fact, there were still a number of government systems that were monkey patched with date workarounds and not properly fixed well into the 2000's (I don't know about now but it wouldn't shock me). There was a decent influx of older devs using the media hype as a way to get nice consulting dollars, nothing wrong with that, but in the end the problem and associated fix was not really a major technical hurdle, except for a few cases. It is also important to understand a lot of systems were not in a SQL databases at the time, many were in ISAM, Pic, dBase (ouch), dbm's (essentially NoSql before NoSql hype) or custom db formats (like flat files etc) that required entire databases to be rewritten, or migrated to new solutions. My 2 cents, it was a real situation that if ignored could have been a major economic crisis, most companies were addressing it in various ways in plenty of time but the media latched on to a set of high profile companies/government systems that were untouched and hyped it. If you knew any Cobol or could work a Vax or IBM mainframe you could bank some decent money. I was mainly doing new dev work but I did get involved in fixing a number of older code bases, mainly on systems in non-popular languages or on different hardware/OS because I have a knack for that and had experience on most server/mainframe architectures you could name at that time.

wefarrell 7y ago
It a shame that we as engineers can't just say "this maintenance is required to fix this known issue. It's not a huge deal but will cause trouble if it's not dealt with". Instead we have to be all doom and gloom and tell management that the company/world will end.
- mgkimsal 7y ago
  Colleague was just dealing with a clients of their who was still using TLS1.0. They're running classic ASP on Window Server 2008, and can't (effectively) migrate. Colleague had been raising the alarm for months (since they started on the project) that "this is going to mean all your systems will stop working in early 2020" but no one seemed to care or understand. They did, last week, put in ... haproxy as an SSL terminator in front of the main server, and will test a switch over this week. This was 8 months of foot-dragging for about 3 hours of setup/config, and a couple more hours of testing. When all your clients are hitting a web server, and their browsers will all stop rejecting your certs, things will get ugly fast - as in "your business will effectively stop functioning". It just sounded like "doom and gloom" but... how do you message this effectively? It requires the receiving parties to actually understand the impact of what you're saying, regardless of terms you use.
  - dillonmckay 7y ago
    Similar story, had an older version of MySQL on a 2008r2 server until a few weeks ago. Advocating for 2 years to migrate off that box.
  - wefarrell 7y ago
    I hope your colleague had a paper trail of the alarms he raised when it came time to point fingers. For executives with limited IT experience I honestly don't know if there is a good solution, other than having them deal with the disaster and having a clear paper trail that points the finger in their direction. They won't make the same mistake twice.
    - mgkimsal 7y ago
      He did/does.
  - cogman10 7y ago
    Money, Money is the way you talk that makes business listen. You say "If we don't solve this by X date, we are looking at losing the ability to take in revenue and possible law suits for failure to perform. It will take Y amount of time to accomplish this" If you don't couch things in terms of time, money, resources, client impact. They will not care. You can say "Hey, it is super bad that we are running Tomcat 7.0.0 there are a lot of security vulnerabilities" and what they will hear is "blah, blah, blah, we can delay this".
  - davengh 7y ago
    We've been forced to deal with the TLS issue by our software product's customers. Some of our technically-minded folks have been raising the issue for a while but new features are sexy and sell, and fixing not-yet-broken code doesn't. Amazing/discouraging that only the threat of immediate loss of six figures of income gets any attention.
    - mgkimsal 7y ago
      Yeah... "let's do Fitbit integration!" seems to win out over "let's make sure we can upgrade core systems to make sure they don't break in March".
unilynx 7y ago
> I don't know about now but it wouldn't shock me We'll be up for a few Y2K bugs at the start of every decade because 'year += year < n ? 2000 : 1900' was such an easy workaround, for n=20,30,40,... https://www.pymnts.com/news/payment-methods/2020/new-years-bug-takes-nyc-parking-meters-offline/ https://www.pymnts.com/news/payment-methods/2020/new-years-b...
- LeifCarrotson 7y ago
  There's less cause to use small data sizes for timestamps and date codes now. Storage has grown by orders of magnitude, the idea that a numeric data type would only be large enough to store a 2-digit year or that you would want to save disk space by abbreviating an extra 2 letters is foreign to a lot of new developers. And the 20-year-old systems are slowly dissapearing...
  - bluGill 7y ago
    More importantly we have figured out algorithms for time that account for timezone, daylight savings time, different calendar systems, and probably something else I'm not aware of. These all work by counting seconds since a fixed date. Note that the algorithms were known since at least the late 1960s. Storage space even then shouldn't have been enough of a concern. However people still screw it up all the time.
    - LeifCarrotson 7y ago
      And in that vein, no one (sane) is writing their own date libraries anymore, there's no reason to try to customize that for your application.
    - davismwfl 7y ago
      This is the accurate point. We had epoch date tracking since the 1960s or so and the coding of a two digit year was out of laziness and convenience in most cases. And not bashing anyone, but most of the systems I saw using the lazy method were more business systems where there wasn't thoughtful engineering a lot of the time, but more just get it done. Those two don't have to be mutually exclusive IMO, but in the 70s/80's it seems a lot of the time it was, simply because IMO we had predominately really two classes of developers, scientific and business where business devs took less of an engineering mindset and more get it done in a specific timeframe so they "hacked" it out. Plus it was new, so stuff happens. Sadly this stuff still goes on today, just in slightly different ways. Data type and data structure choices are critical, and it isn't as important to me that someone know how to write an algorithm for a RB-tree (or pick your DS), but it is critical an engineer knows which data structure/type to choose for specific use cases. This is one of my biggest complaints with the way a lot of engineering interviews are structured today, they want you to regurgitate known algorithms which is a waste of time and personal memory, find out if the person knows when they'd use a specific data structure and why the rest is easily researched. I mention this because to me the fundamental flaw of the y2k problem was one of data structure and data type choice.
_red 7y ago
>dbase At the time I was managing a dBase / FoxPro medical software package...we were a small staff who had to come up with Y2K mitigation on our own. Our problem is we only had source code for "our" part of the chain...other data was being fed into the system from external systems where we had no vendor support. Thus our only conceivable plan was to do the old: If $year<10; date="20$year" else date="19$year" It worked in 99.9% of the cases which was enough for us to limp thru and just fix the bad cases by hand as they happened. Eventually we migrated off the whole stack over the next few years so stopped being a problem. I'm sure many mitigation strategies did the same....
- davismwfl 7y ago
  This is what I remember a lot of too. While they are little hacks which are imperfect, they bought companies enough time to resolve the issue more thoroughly.
- andyjpb 7y ago
  A much more insidious problem with the Y2K bug was the leap year calculation. As you point out, the 20-digit-year thing was relatively easy to fix. https://en.wikipedia.org/wiki/Year_2000_problem#Leap_years https://en.wikipedia.org/wiki/Year_2000_problem#Leap_years
  - protomyth 7y ago
    I still love the fact that if you only implemented the first rule or had the knowledge to implement all 3 rules, it would totally work, but if you implemented 2 of the 3 rules you were wrong. It taught me a great lesson about results not proving correctness as how you got there could bite you later.
    - kccqzy 7y ago
      I don't get why people don't know all three rules. In elementary school when the calendar is taught, the complete rules of the leap year were simply taught by the teacher. We even joked about people born on the 29th of February. Why wasn't this taught everywhere?
dsfdsfsdfasd 7y ago
I'd argue if it wasn't for the media "doom and gloom" a lot of companies (maybe even the vast majority), would not have had their C level folks divert money towards fixing the issue. The simple reason is that any one company may have been liable for a very small portion, which if it failed, would not have caused much trouble. But that failure combined with many other failures down the entire chain of connected and unconnected software would have added up to something much greater than the sum of parts. We saw similar stuff happen with Flash, Silverlight (which wasn't as reported a concern since silverlight usage was so low, but I saw it within my company). The media pressure was a significant reason why every company needed to have a plan to deal with it.
marcus_holmes 7y ago
> but the media hyped it into a frenzy because Because they make more money if people are scared and clicking refresh every few minutes.

wefarrell 7y ago

It a shame that we as engineers can't just say "this maintenance is required to fix this known issue. It's not a huge deal but will cause trouble if it's not dealt with". Instead we have to be all doom and gloom and tell management that the company/world will end.

mgkimsal 7y ago
Colleague was just dealing with a clients of their who was still using TLS1.0. They're running classic ASP on Window Server 2008, and can't (effectively) migrate. Colleague had been raising the alarm for months (since they started on the project) that "this is going to mean all your systems will stop working in early 2020" but no one seemed to care or understand. They did, last week, put in ... haproxy as an SSL terminator in front of the main server, and will test a switch over this week. This was 8 months of foot-dragging for about 3 hours of setup/config, and a couple more hours of testing. When all your clients are hitting a web server, and their browsers will all stop rejecting your certs, things will get ugly fast - as in "your business will effectively stop functioning". It just sounded like "doom and gloom" but... how do you message this effectively? It requires the receiving parties to actually understand the impact of what you're saying, regardless of terms you use.
- dillonmckay 7y ago
  Similar story, had an older version of MySQL on a 2008r2 server until a few weeks ago. Advocating for 2 years to migrate off that box.
- wefarrell 7y ago
  I hope your colleague had a paper trail of the alarms he raised when it came time to point fingers. For executives with limited IT experience I honestly don't know if there is a good solution, other than having them deal with the disaster and having a clear paper trail that points the finger in their direction. They won't make the same mistake twice.
  - mgkimsal 7y ago
    He did/does.
- cogman10 7y ago
  Money, Money is the way you talk that makes business listen. You say "If we don't solve this by X date, we are looking at losing the ability to take in revenue and possible law suits for failure to perform. It will take Y amount of time to accomplish this" If you don't couch things in terms of time, money, resources, client impact. They will not care. You can say "Hey, it is super bad that we are running Tomcat 7.0.0 there are a lot of security vulnerabilities" and what they will hear is "blah, blah, blah, we can delay this".
- davengh 7y ago
  We've been forced to deal with the TLS issue by our software product's customers. Some of our technically-minded folks have been raising the issue for a while but new features are sexy and sell, and fixing not-yet-broken code doesn't. Amazing/discouraging that only the threat of immediate loss of six figures of income gets any attention.
  - mgkimsal 7y ago
    Yeah... "let's do Fitbit integration!" seems to win out over "let's make sure we can upgrade core systems to make sure they don't break in March".

unilynx 7y ago

> I don't know about now but it wouldn't shock me We'll be up for a few Y2K bugs at the start of every decade because 'year += year < n ? 2000 : 1900' was such an easy workaround, for n=20,30,40,... https://www.pymnts.com/news/payment-methods/2020/new-years-bug-takes-nyc-parking-meters-offline/ https://www.pymnts.com/news/payment-methods/2020/new-years-b...

LeifCarrotson 7y ago
There's less cause to use small data sizes for timestamps and date codes now. Storage has grown by orders of magnitude, the idea that a numeric data type would only be large enough to store a 2-digit year or that you would want to save disk space by abbreviating an extra 2 letters is foreign to a lot of new developers. And the 20-year-old systems are slowly dissapearing...
- bluGill 7y ago
  More importantly we have figured out algorithms for time that account for timezone, daylight savings time, different calendar systems, and probably something else I'm not aware of. These all work by counting seconds since a fixed date. Note that the algorithms were known since at least the late 1960s. Storage space even then shouldn't have been enough of a concern. However people still screw it up all the time.
  - LeifCarrotson 7y ago
    And in that vein, no one (sane) is writing their own date libraries anymore, there's no reason to try to customize that for your application.
  - davismwfl 7y ago
    This is the accurate point. We had epoch date tracking since the 1960s or so and the coding of a two digit year was out of laziness and convenience in most cases. And not bashing anyone, but most of the systems I saw using the lazy method were more business systems where there wasn't thoughtful engineering a lot of the time, but more just get it done. Those two don't have to be mutually exclusive IMO, but in the 70s/80's it seems a lot of the time it was, simply because IMO we had predominately really two classes of developers, scientific and business where business devs took less of an engineering mindset and more get it done in a specific timeframe so they "hacked" it out. Plus it was new, so stuff happens. Sadly this stuff still goes on today, just in slightly different ways. Data type and data structure choices are critical, and it isn't as important to me that someone know how to write an algorithm for a RB-tree (or pick your DS), but it is critical an engineer knows which data structure/type to choose for specific use cases. This is one of my biggest complaints with the way a lot of engineering interviews are structured today, they want you to regurgitate known algorithms which is a waste of time and personal memory, find out if the person knows when they'd use a specific data structure and why the rest is easily researched. I mention this because to me the fundamental flaw of the y2k problem was one of data structure and data type choice.

_red 7y ago

>dbase At the time I was managing a dBase / FoxPro medical software package...we were a small staff who had to come up with Y2K mitigation on our own. Our problem is we only had source code for "our" part of the chain...other data was being fed into the system from external systems where we had no vendor support. Thus our only conceivable plan was to do the old: If $year<10; date="20$year" else date="19$year" It worked in 99.9% of the cases which was enough for us to limp thru and just fix the bad cases by hand as they happened. Eventually we migrated off the whole stack over the next few years so stopped being a problem. I'm sure many mitigation strategies did the same....

davismwfl 7y ago
This is what I remember a lot of too. While they are little hacks which are imperfect, they bought companies enough time to resolve the issue more thoroughly.
andyjpb 7y ago
A much more insidious problem with the Y2K bug was the leap year calculation. As you point out, the 20-digit-year thing was relatively easy to fix. https://en.wikipedia.org/wiki/Year_2000_problem#Leap_years https://en.wikipedia.org/wiki/Year_2000_problem#Leap_years
- protomyth 7y ago
  I still love the fact that if you only implemented the first rule or had the knowledge to implement all 3 rules, it would totally work, but if you implemented 2 of the 3 rules you were wrong. It taught me a great lesson about results not proving correctness as how you got there could bite you later.
  - kccqzy 7y ago
    I don't get why people don't know all three rules. In elementary school when the calendar is taught, the complete rules of the leap year were simply taught by the teacher. We even joked about people born on the 29th of February. Why wasn't this taught everywhere?

dsfdsfsdfasd 7y ago

I'd argue if it wasn't for the media "doom and gloom" a lot of companies (maybe even the vast majority), would not have had their C level folks divert money towards fixing the issue. The simple reason is that any one company may have been liable for a very small portion, which if it failed, would not have caused much trouble. But that failure combined with many other failures down the entire chain of connected and unconnected software would have added up to something much greater than the sum of parts. We saw similar stuff happen with Flash, Silverlight (which wasn't as reported a concern since silverlight usage was so low, but I saw it within my company). The media pressure was a significant reason why every company needed to have a plan to deal with it.

marcus_holmes 7y ago

> but the media hyped it into a frenzy because Because they make more money if people are scared and clicking refresh every few minutes.