3 ms·
Year 2006, we had a very high-traffic website running with 1 MySQL server and 1 web server (PHP). Maybe high availability or resillience terms were not coined y
by cagataygurturk 9y ago
Year 2006, we had a very high-traffic website running with 1 MySQL server and 1 web server (PHP). Maybe high availability or resillience terms were not coined yet, that's why we were comfortable with having one server per each function. Web server had two ethernet cards, one is with private IP and one is publicly accessible IP. After a while, the platform started to crash and I would be called by my loyal users before Pingdom alerts reach to me, then I would call the datacenter technicians to press restart button of the web server. Obviously it was a lengthy process for recovery, with a lot of human involved.
After a while, I discovered that the issue was about web server's ethernet card attached to internal network and used to connect MySQL server. When that ethernet card stops working, the platform would crash. On the other hand, it was also possible to connect to MySQL using the other ethernet card via public IP. It would reduce the performance of the platform, since all the bandwidth of that card (100 mbps!) is already eaten by HTTP traffic, but at least it would keep it running.
I ended up writing a script at my home computer, checking if the platform is up or not. Once the faulty ethernet card fails, it would connect to FTP, change PHP configuration to use the other ethernet interface to connect to MySQL server, and send an e-mail to datacenter technicians to press restart button.
This script successfully did its job during 3 months, until I eventually replaced the faulty ethernet card and fixed the issue.
Isn't it "Invent and Simplify" like Jeff Bezos says?