4 ms·
This probably isn't the type of war story you were expecting, but... Many years ago I was the leader of an application maintenance team. The application had an
by Alex63 6y ago
This probably isn't the type of war story you were expecting, but...
Many years ago I was the leader of an application maintenance team. The application had an n-tier client-server architecture. The database was DB2, running on an MVS system (with CICS).
We had some records that had incorrect information in them about the business process (basically what state they were in in the workflow). The team had to reset the business process info to the correct value. We wrote an UPDATE statement, and tested it in our non-production environment. Everything looked good.
In order to perform the clean up, we needed to create some new JCL to run the UPDATE statement in production. Because we had not yet learned better, the process involved re-keying the SQL into a new JCL file. Unfortunately, when the developer re-keyed the statement into the new JCL, he accidentally inserted a ';' before the WHERE clause.
The morning after the job was supposed to run, I asked the developer whether it had worked. "Yes," he told me. I asked him how many records had been updated in the end. He checked. Then he told me a number that was so big I knew something must have gone wrong. As you have probably already realized, we had updated every record in the table. We had changed the state of every workflow in our system.
We were very fortunate that we had a nightly backup AND I had a more senior manager who realized that we could repair the records using the data in the backup without doing a full restore. The org lost about a day of productivity, and we learned to never re-enter code that had been successfully tested. We parameterized our JCL after that.