3 ms·
My biggest f'up, so far.. It happened on my third week as a junior developer on a very nice startup company - its kinda my big dream to work in a startup. It
by codenut 14y ago
My biggest f'up, so far..
It happened on my third week as a junior developer on a very nice startup company - its kinda my big dream to work in a startup.
It was a friday morning and I was just starting my day at work(I was working remotely) when suddenly one of the cofounders sent out an email that our website is timing out. So I checked out out nagios to see if the website is receiving a large amount of traffic and surprisingly I can even count the number of connections using my fingers. I was a newly hire back then and our lead developer is currently flying on his way back home. My other teammate is not yet online because he is in a different timezone and it is not yet his time to work. So basically I was the only developer available at the time. When my figure out that I have no idea of what is happening, he asked me to just shut down the server so that our customers will not be able to process erroneous transactions. The website is hosted on AWS EC2 and I cannot find our Amazon login credentials(its either I was too dumb at the time or too nervous because later I found out that our lead dev gave it to us a week before) then I decided to shutdown the sever through cli, you know shutdown -h now.
Now the other developer got online and asked me what happened. I told him everything then he decided to power up the server so that he can investigate the issue. He logged in to AWS console but he cannot find the server. It turns out that the server's shutdown behavior was set to terminate. And yes, I just destroyed/deleted the server that the website is using. To cut the story short, our lead developer came online and he rebuilt a new server. But still the timing out issue is still there. He found out that it was coming from a MySQL connection and the root cause was that select statement that is very slow. And guess who wrote that query. Yeah its me. A new release was just deployed the previous day and that query was used in one of the new features. The website became operational the following day and everything came back to normal.
The next day I became emotional and was depressed the following week that I handed down my resignation because I felt like I dont deserve to work for their company. They tried to talked me out on not leaving. The lead dev even said nice things to me(that Im a good coder and even him will write the same kind of select query if it was assigned to him). But my mind was too clouded and made a very poor judgment to pursue my resignation. And here I am now stuck on a corporate job trying to figure things out and getting my shit back together hoping someday I can work in a startup again and not f'up.
- josephkern 14y agoSorry to hear that, don't be so hard on yourself in the future. This is hard work (even if it is fun). As a post-mortem you did one thing right and one thing wrong. What you did right: You accepted responsibility. Good, this is a hard trait to find. The "perfect" engineer mixes equal parts intellegence and humility. What you did wrong: You didn't trust your team. They were all very supportive of you, but you didn't trust their assesment of you. Trust your team; trust yourself. Have confidence in your ability to learn from your mistakes. You'll do fine. Start looking for another start-up job while working at a corp. The best time to find a new job, is when you already have one.