2 ms·
AWS random system failures
Anyone else noticing random system failures on AWS? Starting two days ago we've been noticing servers randomly die. At first it was simple matter of forcing the host to stop and then start. Now we're getting disk corruption as well... e.g.
```
tting hostname localhost.localdomain: [ OK ]
Setting up Logical Volume Management: /var/run/lvm/lvmetad.socket: connect failed: Connection refused
WARNING: Failed to connect to lvmetad. Falling back to device scanning.
[ OK ]
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext4 (1) -- /] fsck.ext4 -a /dev/xvda1
/ contains a file system with errors, check forced.
/: Inodes that were part of a corrupted orphan linked list found.
/: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
(i.e., without -a or -p options)
[FAILED]
* An error occurred during the file system check.
* Dropping you to a shell; the system will reboot
* wh/dev/fd/9: line 2: plymouth: command not found
Give root password for maintenance
(or type Control-D to continue): [ 18.428094] random: crng init done
```
That's from the system log... lots of fun to recover these...
- taf2 9y agoIt looks or sounds like one of the us-east1 regions is having issues with the new m5 series hosts... they die and can't be rebooted.