47 ms·

I wish the process of restoring a non-booting Debian-derived system were easier. After needing to do it on several dual-boot systems, I keep the procedure hand

I wish the process of restoring a non-booting Debian-derived system were easier. After needing to do it on several dual-boot systems, I keep the procedure handy now. Here it is for posterity: 1) Boot Linux from your distro CD or DVD 2) Get a shell 3) Mount your normal Linux partitions. (Make sure you know where they are. The following example assumes / on /dev/sda1 and /boot on /dev/sda2.) E.g. mount /dev/sda1 /mnt Note: If you have a separate partition for /boot, then mount it too: mount /dev/sda2 /mnt/boot 4) Mount the special nodes: mount --bind /dev /mnt/dev && mount --bind /dev/pts /mnt/dev/pts && mount --bind /proc /mnt/proc && mount --bind /sys /mnt/sys 5) Change your shell's root: chroot /mnt 6) Re-install grub: grub-install /dev/sda 7) Update the grub boot menu: update-grub 8) Undo chroot: exit 9) Unmount the special nodes: umount /mnt/dev && umount /mnt/dev/pts && umount /mnt/proc && umount /mnt/sys 10) Remove media and reboot

freeone3000 6y ago
This is essentially identical to the procedure I did 15 years ago. What's the issue?
- tux3 6y ago
  I don't think GP is complaining because the procedure changed in the last 15 years, but precisely because it didn't. Surely, there's room for improvement.
  - vetinari 6y ago
    Well, he still doesn't use UEFI. He could, it has been available for more than decade (i.e. better part of those 15 years), so the procedure didn't change because he has chosen so. Not that it is a bad thing, some people do prefer that kind of stability. But then, why complain about that? Btw, Intel did plan to remove CSM (legacy BIOS compatibility) with Tiger Lake and require UEFI class 3. We will see if they will follow through. Then the procedure WILL change, and we will hear from people who didn't expect it.
    - yjftsjthsd-h 6y ago
      Wouldn't UEFI make it more complex, since you now potentially need to mount / and /boot and /boot/efi all separately before you can fully recover grub?
      - vetinari 6y ago
        Not more complex, but different. It does not have magic sectors on disk, in MBR for boot loader, on somewhere in dummy area of the filesystem for grubenv, everything happens on the EFI partition and with EFI variables (stored in nvram). Creating bootable EFI media means having vfat-formatted filesystem and copying files there. No need for tools like Rufus. Your firmware would find it at boot time, and allow you to boot from it; most UEFI implementation have boot manager built-in.
  - yjftsjthsd-h 6y ago
    I think the issue boils down to the fact that it depends almost completely on exactly how the system is put together. I mean, the process is: 1. Boot into a working-enough live system - only way to improve this is to put a recovery system on the same system, but then you risk making it unbootable as well 2. Mount needed filesystems - depends completely on what filesystems there are, which is extremely variable; my only thought would be to use labels and hope that the installed system labels root/boot/efi the way you expect 3. Mount special filesystems - this is possible to automate; ex. arch-chroot (https://wiki.archlinux.org/index.php/Chroot#Using_arch-chroot https://wiki.archlinux.org/index.php/Chroot#Using_arch-chroo...) does it, but that requires that you know what you need, so it's going to be brittle unless you stick with distro-provided tools (hence, arch chroot) 4. Fix the system - again, totally depends on what happened and how it should be set up; yes, in the trivial case you could make a "just-reinstall-grub.sh", but it'll fall apart for non-trivial setups If every system looked the same, then yes you could make a livecd that automatically booted, set up mounts, chrooted in, fixed grub, and rebooted out, but this is the world of Linux-based systems so even within a single distro there's worlds of difference.
    - the8472 6y ago
      Semi-automating the common cases with a GUI would probably help many of the less sophisticated users. E.g. scan the existing system for partitions and ask them which one they want to repair. Windows' repair mechanisms also only work in the common case and you have resort to quite similar CLI stepss when they don't.
mjevans 6y ago
I usually do the bind mounts a little differently: for ii in dev dev/pts proc sys ; do mount --bind /$ii /target/$ii ; done Unmount: umount /target/dev/* /target/* /target This can also be useful on newer systems where systemd systems need to be chrooted. systemd-nspawn --directory /target --boot -- --unit rescue.target
jeroenhd 6y ago
This strongly depends on the issue though. In most dual booting cases I've had to deal with recently, a single efibootmgr command was enough. Windows and some Linux distros like to replace the fallback EFI command and the two can conflict every now and then. By properly configuring the UEFI to pick the right OS at boot (sudo efibootmgr -o 0001,0000 or something like that) should be enough. If you're still using MBR or you have a shitty motherboard with bad UEFI support you're right that you're still forced to do a full Grub reinstall. I've had to do recovery on a non-booting Windows machine that broke after a recent update and it was just impossible to recover. Some file on the FS was corrupted, but SFC couldn't recover it and wouldn't report what file was corrupted. Rebooting into recovery takes forever, boot recovery constantly fails and the only saving grace is the "reinstall Windows" that uses a complete reinstall as a fix for a broken OS. Yes, fixing Linux is difficult, but at least it's doable. On Windows you're basically stuck with the two or three auto-recovery options or an OS + software reinstall. I can't speak for macOS but I can't imagine it being much easier (especially because the limited variety of macOS hardware makes it more unlikely for the OS to fail catastrophically because that's easier to test for). It shouldn't be too hard to create a Linux boot ISO that lists installed operating systems and then automounts them for recovery. It's only a matter of parsing GRUB config files + fstab + crypttab and some predefined mount patterns for distros after all.
- arprocter 6y ago
  Mac lets you reinstall the OS from the recovery partition without losing data https://support.apple.com/en-us/HT204904 https://support.apple.com/en-us/HT204904 If the recovery partition is toast you can do a clean install over wifi - Option-Command-R or Shift-Option-Command-R (would only work over an open wifi the last time I did it)
  - CaliforniaKarl 6y ago
    It also works over Ethernet!
    - arprocter 6y ago
yaantc 6y ago
After several Win10 upgrades nuked grub on a dual boot system, I have found a much simpler process. Create a rEFInd [1] USB boot key once and for all. Then whenever grub is hosed, boot on this key: rEFInd will find the Debian partition and allows to run it directly. Then once in Debian just do an "update-grub" to fix things. [1] https://www.rodsbooks.com/refind/getting.html https://www.rodsbooks.com/refind/getting.html
- manjalyc 6y ago
  You could just install rEFInd and make it the default bootloader with its auto find mode enabled.
  - jrwr 6y ago
    win10 install nukes it sometimes as well. its a PITA
    - sk0g 6y ago
      Have you found a way to deal with this? Swear I have refind on the non-Windows disk, but Windows updates sometimes replaces either the partition, or the default boot device. Mega frustrating!
      - manjalyc 6y ago
        https://wiki.archlinux.org/index.php/Unified_Extensible_Firmware_Interface#Windows_changes_boot_order https://wiki.archlinux.org/index.php/Unified_Extensible_Firm...
    - boomboomsubban 6y ago
      Over two years of dual booting with Windows 8 I never had it screw up my UEFI boot. The MBR I had tried in the past never lasted but with UEFI it just left everything on the ESP alone.
    - deleted 6y ago
      [deleted]
ohazi 6y ago
This is essentially the process used for installing the base system in Arch. As usual, the Arch Linux wiki page on this topic [1] is a good resource, even if you're using a different system. If you want to learn more about how a typical Linux system is organized, but don't have the time or patience to go through the Linux From Scratch book [2], installing and configuring Arch once is pretty insightful and also kind of fun. [1] https://wiki.archlinux.org/index.php/Installation_guide https://wiki.archlinux.org/index.php/Installation_guide [2] http://www.linuxfromscratch.org/lfs/ http://www.linuxfromscratch.org/lfs/
- yjftsjthsd-h 6y ago
  I mean, it's just a chroot; it's the basic working "run a system without booting from it" that underlies a lot of base system tinkering and fixing.
  - ohazi 6y ago
    There are lots of people who have used Linux for years but have never actually gone through this exercise. It's not very fun to have to do it for the first time when an important system fails to boot. It's great that the modern Linux install process is so easy, but one drawback is that it also makes it easy to gloss over how the system is put together. I just meant that it can be insightful to put together a toy system from scratch, so that you can learn (at your own pace) how chroots work, learn the conventions around manually recreating your root hierarchy in /mnt, learn what systemd services you actually need because you need to manually turn them on, rather than having a bunch of stuff you don't understand enabled by default, etc.
- lmm 6y ago
  I was going to say Gentoo, for much the same reasons.
boardwaalk 6y ago
Man, I’m glad ArchLinux has arch-chroot to do this for you!
pwdisswordfish2 6y ago
Why is the Debian-derived system non-booting? Many years ago I had disk error problem and could not boot. I tried a popular, Linux-based "system rescue" CD and it could not boot without accessing the disk. I tried my own "live" NetBSD USB stick and booted up no problem. No disk access needed. I stayed away from Linux for many years as a result of experiences like that. I have been using Linux lately and I continue to find more stuff like this that would just never happen on BSD. I do not understand why people use grub, let alone grub2. There are plenty of other bootloaders. What is wrong with syslinux?
- floatboth 6y ago
  GRUB2 is the most flexible bootloader. I use it for a multi-boot flash drive, used for booting 64-bit FreeBSD on a 32-bit-EFI machine (earliest Mac mini + 64-bit CPU)… But yeah for normal desktop usage… there's rEFInd.
  - pwdisswordfish2 6y ago
    Flexibilty would have been my first guess. Are there any other reasons besides flexibility?
    - RealStickman_ 6y ago
      I'd say support in distros. With grub you can throw pretty much any distro onto your machine and it will show up (except Solus) . Also I don't think you can just choose which bootloader you want in most distros except Arch and Debian if I remember correctly. You can do that manually of course, but grub is easy and it works.
      - zaarn 6y ago
        In arch you can choose the bootloader as well. The recommendation I hear a lot and that I use is systemd-boot (formerly gummiboot), which strangely doesn't have much to do with systemd. It can pick up Windows for multiboot and is straightforward to configure (can even be configured from a windows live disk if you mount that FAT partition).
cmurf 6y ago
This isn't applicable to a UEFI Secure Boot system, where the shim and grubx64 binaries are distribution signed and installed from the package - not via grub2-install, which on UEFI produces an unsigned binary and will not boot under UEFI Secure Boot.
monadic2 6y ago
UEFI makes this problem much easier—even if you still use a bootloader (no need!) there’s not much need to update it after it starts working.
account42 6y ago
Congratulations, you just learned how to install Gentoo.

freeone3000 6y ago

This is essentially identical to the procedure I did 15 years ago. What's the issue?

tux3 6y ago
I don't think GP is complaining because the procedure changed in the last 15 years, but precisely because it didn't. Surely, there's room for improvement.
- vetinari 6y ago
  Well, he still doesn't use UEFI. He could, it has been available for more than decade (i.e. better part of those 15 years), so the procedure didn't change because he has chosen so. Not that it is a bad thing, some people do prefer that kind of stability. But then, why complain about that? Btw, Intel did plan to remove CSM (legacy BIOS compatibility) with Tiger Lake and require UEFI class 3. We will see if they will follow through. Then the procedure WILL change, and we will hear from people who didn't expect it.
  - yjftsjthsd-h 6y ago
    Wouldn't UEFI make it more complex, since you now potentially need to mount / and /boot and /boot/efi all separately before you can fully recover grub?
    - vetinari 6y ago
      Not more complex, but different. It does not have magic sectors on disk, in MBR for boot loader, on somewhere in dummy area of the filesystem for grubenv, everything happens on the EFI partition and with EFI variables (stored in nvram). Creating bootable EFI media means having vfat-formatted filesystem and copying files there. No need for tools like Rufus. Your firmware would find it at boot time, and allow you to boot from it; most UEFI implementation have boot manager built-in.
- yjftsjthsd-h 6y ago
  I think the issue boils down to the fact that it depends almost completely on exactly how the system is put together. I mean, the process is: 1. Boot into a working-enough live system - only way to improve this is to put a recovery system on the same system, but then you risk making it unbootable as well 2. Mount needed filesystems - depends completely on what filesystems there are, which is extremely variable; my only thought would be to use labels and hope that the installed system labels root/boot/efi the way you expect 3. Mount special filesystems - this is possible to automate; ex. arch-chroot (https://wiki.archlinux.org/index.php/Chroot#Using_arch-chroot https://wiki.archlinux.org/index.php/Chroot#Using_arch-chroo...) does it, but that requires that you know what you need, so it's going to be brittle unless you stick with distro-provided tools (hence, arch chroot) 4. Fix the system - again, totally depends on what happened and how it should be set up; yes, in the trivial case you could make a "just-reinstall-grub.sh", but it'll fall apart for non-trivial setups If every system looked the same, then yes you could make a livecd that automatically booted, set up mounts, chrooted in, fixed grub, and rebooted out, but this is the world of Linux-based systems so even within a single distro there's worlds of difference.
  - the8472 6y ago
    Semi-automating the common cases with a GUI would probably help many of the less sophisticated users. E.g. scan the existing system for partitions and ask them which one they want to repair. Windows' repair mechanisms also only work in the common case and you have resort to quite similar CLI stepss when they don't.

mjevans 6y ago

I usually do the bind mounts a little differently: for ii in dev dev/pts proc sys ; do mount --bind /$ii /target/$ii ; done Unmount: umount /target/dev/* /target/* /target This can also be useful on newer systems where systemd systems need to be chrooted. systemd-nspawn --directory /target --boot -- --unit rescue.target

jeroenhd 6y ago

This strongly depends on the issue though. In most dual booting cases I've had to deal with recently, a single efibootmgr command was enough. Windows and some Linux distros like to replace the fallback EFI command and the two can conflict every now and then. By properly configuring the UEFI to pick the right OS at boot (sudo efibootmgr -o 0001,0000 or something like that) should be enough. If you're still using MBR or you have a shitty motherboard with bad UEFI support you're right that you're still forced to do a full Grub reinstall. I've had to do recovery on a non-booting Windows machine that broke after a recent update and it was just impossible to recover. Some file on the FS was corrupted, but SFC couldn't recover it and wouldn't report what file was corrupted. Rebooting into recovery takes forever, boot recovery constantly fails and the only saving grace is the "reinstall Windows" that uses a complete reinstall as a fix for a broken OS. Yes, fixing Linux is difficult, but at least it's doable. On Windows you're basically stuck with the two or three auto-recovery options or an OS + software reinstall. I can't speak for macOS but I can't imagine it being much easier (especially because the limited variety of macOS hardware makes it more unlikely for the OS to fail catastrophically because that's easier to test for). It shouldn't be too hard to create a Linux boot ISO that lists installed operating systems and then automounts them for recovery. It's only a matter of parsing GRUB config files + fstab + crypttab and some predefined mount patterns for distros after all.

arprocter 6y ago
Mac lets you reinstall the OS from the recovery partition without losing data https://support.apple.com/en-us/HT204904 https://support.apple.com/en-us/HT204904 If the recovery partition is toast you can do a clean install over wifi - Option-Command-R or Shift-Option-Command-R (would only work over an open wifi the last time I did it)
- CaliforniaKarl 6y ago
  It also works over Ethernet!
  - arprocter 6y ago

yaantc 6y ago

After several Win10 upgrades nuked grub on a dual boot system, I have found a much simpler process. Create a rEFInd [1] USB boot key once and for all. Then whenever grub is hosed, boot on this key: rEFInd will find the Debian partition and allows to run it directly. Then once in Debian just do an "update-grub" to fix things. [1] https://www.rodsbooks.com/refind/getting.html https://www.rodsbooks.com/refind/getting.html

manjalyc 6y ago
You could just install rEFInd and make it the default bootloader with its auto find mode enabled.
- jrwr 6y ago
  win10 install nukes it sometimes as well. its a PITA
  - sk0g 6y ago
    Have you found a way to deal with this? Swear I have refind on the non-Windows disk, but Windows updates sometimes replaces either the partition, or the default boot device. Mega frustrating!
    - manjalyc 6y ago
      https://wiki.archlinux.org/index.php/Unified_Extensible_Firmware_Interface#Windows_changes_boot_order https://wiki.archlinux.org/index.php/Unified_Extensible_Firm...
  - boomboomsubban 6y ago
    Over two years of dual booting with Windows 8 I never had it screw up my UEFI boot. The MBR I had tried in the past never lasted but with UEFI it just left everything on the ESP alone.
  - deleted 6y ago
    [deleted]

ohazi 6y ago

This is essentially the process used for installing the base system in Arch. As usual, the Arch Linux wiki page on this topic [1] is a good resource, even if you're using a different system. If you want to learn more about how a typical Linux system is organized, but don't have the time or patience to go through the Linux From Scratch book [2], installing and configuring Arch once is pretty insightful and also kind of fun. [1] https://wiki.archlinux.org/index.php/Installation_guide https://wiki.archlinux.org/index.php/Installation_guide [2] http://www.linuxfromscratch.org/lfs/ http://www.linuxfromscratch.org/lfs/

yjftsjthsd-h 6y ago
I mean, it's just a chroot; it's the basic working "run a system without booting from it" that underlies a lot of base system tinkering and fixing.
- ohazi 6y ago
  There are lots of people who have used Linux for years but have never actually gone through this exercise. It's not very fun to have to do it for the first time when an important system fails to boot. It's great that the modern Linux install process is so easy, but one drawback is that it also makes it easy to gloss over how the system is put together. I just meant that it can be insightful to put together a toy system from scratch, so that you can learn (at your own pace) how chroots work, learn the conventions around manually recreating your root hierarchy in /mnt, learn what systemd services you actually need because you need to manually turn them on, rather than having a bunch of stuff you don't understand enabled by default, etc.
lmm 6y ago
I was going to say Gentoo, for much the same reasons.

boardwaalk 6y ago

Man, I’m glad ArchLinux has arch-chroot to do this for you!

pwdisswordfish2 6y ago

Why is the Debian-derived system non-booting? Many years ago I had disk error problem and could not boot. I tried a popular, Linux-based "system rescue" CD and it could not boot without accessing the disk. I tried my own "live" NetBSD USB stick and booted up no problem. No disk access needed. I stayed away from Linux for many years as a result of experiences like that. I have been using Linux lately and I continue to find more stuff like this that would just never happen on BSD. I do not understand why people use grub, let alone grub2. There are plenty of other bootloaders. What is wrong with syslinux?

floatboth 6y ago
GRUB2 is the most flexible bootloader. I use it for a multi-boot flash drive, used for booting 64-bit FreeBSD on a 32-bit-EFI machine (earliest Mac mini + 64-bit CPU)… But yeah for normal desktop usage… there's rEFInd.
- pwdisswordfish2 6y ago
  Flexibilty would have been my first guess. Are there any other reasons besides flexibility?
  - RealStickman_ 6y ago
    I'd say support in distros. With grub you can throw pretty much any distro onto your machine and it will show up (except Solus) . Also I don't think you can just choose which bootloader you want in most distros except Arch and Debian if I remember correctly. You can do that manually of course, but grub is easy and it works.
    - zaarn 6y ago
      In arch you can choose the bootloader as well. The recommendation I hear a lot and that I use is systemd-boot (formerly gummiboot), which strangely doesn't have much to do with systemd. It can pick up Windows for multiboot and is straightforward to configure (can even be configured from a windows live disk if you mount that FAT partition).

cmurf 6y ago

This isn't applicable to a UEFI Secure Boot system, where the shim and grubx64 binaries are distribution signed and installed from the package - not via grub2-install, which on UEFI produces an unsigned binary and will not boot under UEFI Secure Boot.

monadic2 6y ago

UEFI makes this problem much easier—even if you still use a bootloader (no need!) there’s not much need to update it after it starts working.

account42 6y ago

Congratulations, you just learned how to install Gentoo.