4 ms·

I accidentally recursively chown'ed / to myself on a server in Antarctica that was a critical gateway for our geophysical network, the night before we were plan

I accidentally recursively chown'ed / to myself on a server in Antarctica that was a critical gateway for our geophysical network, the night before we were planned to leave the ice. Luckily, my flight was delayed... I spent the next day wiping the server and setting it up from scratch, after giving up on trying to recover from all the bizarre problems that stem from owning all the system files (including broken ssh). You should try it sometime!

JKirchartz 7y ago
I did this on a VM once - it's fun for about 15 minutes... then burn it all and start from scratch.
- ianai 7y ago
  Is there a way to save and revert such metadata? I can’t think of anything that stores just FS metadata.
  - BenjiWiebe 7y ago
    I think rsync has an option to copy everything but file contents. Couple that with 'find' running 'touch' and it should be fine. (create blank files, copy attributes to them)
  - Lammy 7y ago
    FreeBSD does! You can rescue yourself from this with mtree. Something like: > cd / > mtree -U -f /etc/mtree/BSD.root.dist > mtree -U -f /etc/mtree/BSD.var.dist > mtree -U -f /etc/mtree/BSD.include.dist > mtree -U -f /etc/mtree/BSD.sendmail.dist > mtree -U -f /etc/mtree/BSD.usr.dist
mondoshawan 7y ago
I did something similar on a central box used for mail aliases, cron jobs, and kerberos called "cartman" by copying and pasting a cronjob line at a root shell. I only noticed when things started failing, and spent the next day rebuilding the permissions all over the disk. chmod -R is powerful. :-)
Balgair 7y ago
> You should try it sometime! An overnight mission at McMurdo or a server reset without internet access on another device? I'll take the overnight.
- bostik 7y ago
  > An overnight mission at McMurdo Depending on the time the year, perhaps. What's the longest possible time from sunset to sunrise at those latitudes?
  - Balgair 7y ago
    179 'days' (4296 hours total). Missions vary in length though.
shifto 7y ago
Sysadminning in Antarctica. This really speaks to the imagination! Can't imagine how different it would be from my normal sysadmin work.
- foobarbecue 7y ago
  You should meet Holly, who was the main McMurdo sysadmin most of the seasons I was there. I can't remember his last name but that guy was really wonderful. Super competent, friendly, and seemed to love his job.
  - shifto 7y ago
    Wish I could! But my Google-fu turns up empty. You'd think working on Antarctica would make it easier to find someone on LinkedIn e.g.

JKirchartz 7y ago

I did this on a VM once - it's fun for about 15 minutes... then burn it all and start from scratch.

ianai 7y ago
Is there a way to save and revert such metadata? I can’t think of anything that stores just FS metadata.
- BenjiWiebe 7y ago
  I think rsync has an option to copy everything but file contents. Couple that with 'find' running 'touch' and it should be fine. (create blank files, copy attributes to them)
- Lammy 7y ago
  FreeBSD does! You can rescue yourself from this with mtree. Something like: > cd / > mtree -U -f /etc/mtree/BSD.root.dist > mtree -U -f /etc/mtree/BSD.var.dist > mtree -U -f /etc/mtree/BSD.include.dist > mtree -U -f /etc/mtree/BSD.sendmail.dist > mtree -U -f /etc/mtree/BSD.usr.dist

mondoshawan 7y ago

I did something similar on a central box used for mail aliases, cron jobs, and kerberos called "cartman" by copying and pasting a cronjob line at a root shell. I only noticed when things started failing, and spent the next day rebuilding the permissions all over the disk. chmod -R is powerful. :-)

Balgair 7y ago

> You should try it sometime! An overnight mission at McMurdo or a server reset without internet access on another device? I'll take the overnight.

bostik 7y ago
> An overnight mission at McMurdo Depending on the time the year, perhaps. What's the longest possible time from sunset to sunrise at those latitudes?
- Balgair 7y ago
  179 'days' (4296 hours total). Missions vary in length though.

shifto 7y ago

Sysadminning in Antarctica. This really speaks to the imagination! Can't imagine how different it would be from my normal sysadmin work.

foobarbecue 7y ago
You should meet Holly, who was the main McMurdo sysadmin most of the seasons I was there. I can't remember his last name but that guy was really wonderful. Super competent, friendly, and seemed to love his job.
- shifto 7y ago
  Wish I could! But my Google-fu turns up empty. You'd think working on Antarctica would make it easier to find someone on LinkedIn e.g.