3 ms·

You can watch a very entertaining talk from one of the Google engineers who went to fix the website at the time: https://www.youtube.com/watch?v=GLQyj-kBRdo htt

• nodesocket 11y ago
  The most ridiculous part, they didn't have a way to run distributed commands over ssh. When a filesystem mounted in read-only mode, somebody had to ssh into each of the 175 servers manually and run commands to fix it.
  • psykovsky 11y ago
    What do you mean by "distributed commands over ssh"?
    • garrettgrimsley 11y ago
      Say you want to run a command on a group of machines using SSH. Rather than manually running the command on each remote machine you can automate their execution over the group using what is known as distributed commands. http://www.tecmint.com/using-dsh-distributed-shell-to-run-linux-commands-across-multiple-machines/ http://www.tecmint.com/using-dsh-distributed-shell-to-run-li... http://www.ibm.com/developerworks/aix/library/au-satdistadmin/#N100E1 http://www.ibm.com/developerworks/aix/library/au-satdistadmi...