5 ms·

The worst programming experience of my life related to lost source code. I was working for a place that had a service running from a Java app that was customiz

The worst programming experience of my life related to lost source code. I was working for a place that had a service running from a Java app that was customized for each customer, about 200 copies of roughly the same app. There was source control at some point, and when a new customer was being on boarded they’d just make the customizations they required, compile the app, and deploy it. By the time I worked there (years later), all source code had been lost, and I was tasked with creating a CI/CD system that included managing these apps (they’d now decided they needed to be maintained). For each one I had to decompile the whole thing, rewrite the code into a human readable format, redeploy and test it. They were filled with all sorts of horrible anti-patterns too, like hard coded file paths, and some of them were only used once a year (but for an absolutely business critical process). About half way through we had a major data center failure that meant we had to do a failover, which of course broke most of these apps with all their hard coded configuration. So instead of having months to work through all of this garbage, I was asked to get them all working immediately. I got them all working in 3 days, which I thought was quite a monumental accomplishment. But my CEO was very dissatisfied that it took so long. I quit a couple weeks later, and still almost regret putting so much effort in to saving them. A friend of mine still works there and apparently all of these applications are in exactly the same state as I left them.

oceanghost 5y ago
That's brutal. You were right to quit. There's a certain personality type that will never appreciate or reward hard work.
- AmericanChopper 5y ago
  Yeah I was definitely right to quit, my career has been pretty great since then. But I did get a lot out of the few years I worked there, even though a lot of it sucked. It was my first proper job and they’d basically let me work on anything I wanted to, even though I was incredibly green and was just figuring it all out as I went. I definitely fit a lot more experience and learning into those years than you’d reasonably expect a person to. I think a lot of my success today came from turning their incompetence into opportunities.
  - ed_elliott_asc 5y ago
    I be also had a job early in my career where I was basically allowed to work on anything that interests me and 20 years later I’m still relying on that experience. Maybe this is something every junior should do - try a bit of everything and see what you like.
    - distances 5y ago
      That's why I think a good consulting agency can be a great first place to work. Lots of projects, hopefully different technologies, perhaps international clients with travel. Then once you know what you like and specialize a bit in it, it's easier to step to some product company as a senior. Plus, in my experience consulting companies are more focused in employee well-being as it's their only asset.
- KronisLV 5y ago
  > There's a certain personality type that will never appreciate or reward hard work. Now, how would someone figure out these personality types ahead of time? Any red flags to detect people or workplaces to avoid in a professional capacity?
  - etripe 5y ago
    Ask them flatout how they reward initiative and accomplishment. If they start fumbling, there's your answer. If they don't, ask them for a concrete example. If that's when they struggle, there's your answer.
  - rawoke083600 5y ago
    1. "...Able to work under pressure and meet deadlines.." PASS ! 2. "I(boss) coded the first version..." 50% of the time it means: Your solutions or design decision will never be "as good as his" and you will forever hear "I did it like this and that. It should be fast to implement xyz"
  - oceanghost 5y ago
    I think the problem ties largely in mental illness... Sociopathic behavior, psychopathic behavior are pretty much the norm in C level management. There's a book called "Dangerous Personalities" which takes a dive into practical psychology for these sorts of people.
pram 5y ago
Hilarious. Why didn't you refactor the customizations into a config file, and just deploy a single jar though? If you went through the effort of rewriting the entire thing!
- AmericanChopper 5y ago
  The issue I found when I was first looking at it was they were all a little bit different in random, and confounding ways. Also, whatever the “base version” of the application was had been undergoing development over the years when all these forks were being made. Looking back on it, that likely would have been the best approach. But at the time the complexity of designing something like that was a bit beyond me.
- oceanghost 5y ago
  I think the OP is saying each program was slightly different. Like it had been forked 200 times.
- gridspy 5y ago
  The OP wasn't re-writing them. Instead the OP fed the old version into a tool (Disassembler) which spat out terrible source code, worse than the original with no comments and bad variable names. This was the only way to get source at all. Then they made the disassembled source compile in a really nice way. Finally the boss asked them to make huge changes to this horrible source with time pressure.
  - AmericanChopper 5y ago
    Pretty much this. A lot of the work would have been roughly equivalent to trying to de-minify some JS. With a modern IDE the task would have been a lot easier. I don’t know if Eclipse had refactoring assistance built into back then, but if it did, I’d never even heard of the idea. I was sitting on not much more than a year of programming experience at the time.
    - grujicd 5y ago
diskzero 5y ago
On the topic of lost source code... we had a few blessed binaries that were vital to building System 8 and System 9 at Apple. There was a single build machine that could generate these binaries from other object code but the source was long gone. Oh please oh please old Quadra 700, don't give out on us! Those days are over right? Surely nothing like that is happening with OSX...
- oceanghost 5y ago
  That's awesome. :-) ]
- hedberg10 5y ago
  Every company has a shrine to vital things that can not break under any circumstances. We have a holy Win7 Thinkpad. Often there are gurus too and monks live in the caves next to the relics, trying to guard the sacred grounds and spreading the gospel.

oceanghost 5y ago

That's brutal. You were right to quit. There's a certain personality type that will never appreciate or reward hard work.

AmericanChopper 5y ago
Yeah I was definitely right to quit, my career has been pretty great since then. But I did get a lot out of the few years I worked there, even though a lot of it sucked. It was my first proper job and they’d basically let me work on anything I wanted to, even though I was incredibly green and was just figuring it all out as I went. I definitely fit a lot more experience and learning into those years than you’d reasonably expect a person to. I think a lot of my success today came from turning their incompetence into opportunities.
- ed_elliott_asc 5y ago
  I be also had a job early in my career where I was basically allowed to work on anything that interests me and 20 years later I’m still relying on that experience. Maybe this is something every junior should do - try a bit of everything and see what you like.
  - distances 5y ago
    That's why I think a good consulting agency can be a great first place to work. Lots of projects, hopefully different technologies, perhaps international clients with travel. Then once you know what you like and specialize a bit in it, it's easier to step to some product company as a senior. Plus, in my experience consulting companies are more focused in employee well-being as it's their only asset.
KronisLV 5y ago
> There's a certain personality type that will never appreciate or reward hard work. Now, how would someone figure out these personality types ahead of time? Any red flags to detect people or workplaces to avoid in a professional capacity?
- etripe 5y ago
  Ask them flatout how they reward initiative and accomplishment. If they start fumbling, there's your answer. If they don't, ask them for a concrete example. If that's when they struggle, there's your answer.
- rawoke083600 5y ago
  1. "...Able to work under pressure and meet deadlines.." PASS ! 2. "I(boss) coded the first version..." 50% of the time it means: Your solutions or design decision will never be "as good as his" and you will forever hear "I did it like this and that. It should be fast to implement xyz"
- oceanghost 5y ago
  I think the problem ties largely in mental illness... Sociopathic behavior, psychopathic behavior are pretty much the norm in C level management. There's a book called "Dangerous Personalities" which takes a dive into practical psychology for these sorts of people.

pram 5y ago

Hilarious. Why didn't you refactor the customizations into a config file, and just deploy a single jar though? If you went through the effort of rewriting the entire thing!

AmericanChopper 5y ago
The issue I found when I was first looking at it was they were all a little bit different in random, and confounding ways. Also, whatever the “base version” of the application was had been undergoing development over the years when all these forks were being made. Looking back on it, that likely would have been the best approach. But at the time the complexity of designing something like that was a bit beyond me.
oceanghost 5y ago
I think the OP is saying each program was slightly different. Like it had been forked 200 times.
gridspy 5y ago
The OP wasn't re-writing them. Instead the OP fed the old version into a tool (Disassembler) which spat out terrible source code, worse than the original with no comments and bad variable names. This was the only way to get source at all. Then they made the disassembled source compile in a really nice way. Finally the boss asked them to make huge changes to this horrible source with time pressure.
- AmericanChopper 5y ago
  Pretty much this. A lot of the work would have been roughly equivalent to trying to de-minify some JS. With a modern IDE the task would have been a lot easier. I don’t know if Eclipse had refactoring assistance built into back then, but if it did, I’d never even heard of the idea. I was sitting on not much more than a year of programming experience at the time.
  - grujicd 5y ago

diskzero 5y ago

On the topic of lost source code... we had a few blessed binaries that were vital to building System 8 and System 9 at Apple. There was a single build machine that could generate these binaries from other object code but the source was long gone. Oh please oh please old Quadra 700, don't give out on us! Those days are over right? Surely nothing like that is happening with OSX...

oceanghost 5y ago
That's awesome. :-) ]
hedberg10 5y ago
Every company has a shrine to vital things that can not break under any circumstances. We have a holy Win7 Thinkpad. Often there are gurus too and monks live in the caves next to the relics, trying to guard the sacred grounds and spreading the gospel.