7 ms·

Linux from Scratch with Training Wheels

242 points by pthyme 5y ago

peakaboo 5y ago
This is great if you are a teenager. I used to do stuff like this all the time. But as an adult.. It's just not worth the time.
- petschge 5y ago
  This isn't (or shouldn't) be an age thing. It is a 'stage of learning" thing. If you are new to Linux (or even all Unices) and you want to get a firm understanding of the fundamentals, it may very well be worth while to go through it at least once and see all the moving parts. And see them in an order that makes their dependencies clear(er). Once you know all that and just want to keep up with the constant churn of versions to keep that one webserver running and recent? Yeah. Not worth it. Get some distro that matches your preferred characteristics and stop worrying if you need to update libBLA to recompile Apache to fix the path traversal vulnerability. That also removes the question if that library still compiles correctly with gcc 7 or if you need to upgrade to at least gcc 9... Edit: Doing it in a VM first, might help in getting the 99 percent working first linux installation into a usable shape to make it a good daily driver. In that sense it helps lowering the barrier to actually switching later on.
- elteto 5y ago
  Yes, adults should not experiment with building custom distros... as we all know all existing Linux distributions were built by teenagers.
- dang 5y ago
  "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- GuB-42 5y ago
  My company is looking for someone to build a linux system for a specialized tablet PC, with, among other things, a custom kernel and OTA updates. If you built a linux system from scratch, this will put you in a very good position to get the job.
- alekq 5y ago
  Adult people do stuff for fun, I guess.
sanirank 5y ago
I don’t know why but I find this so interesting. I totally don’t have time right now (with a young kid and trying to make some/any progress in my career as a dev now that I’ve successfully made the career change to working as a developer), but I really just like understanding how things work from the ground up - I can’t wait to have the time to build my own Linux distribution. It feels like it’ll be such a waste of time - like installing 5 or 6 different distros on an old PC I had just out of curiosity to see what they looked snd felt like. And I broke and fixed the boot loader so many times that it became kind of an enjoyable challenge getting the machine to work again - “ok, now how did I get grub2 to recognise where my partitions are…?” It seemed like spinning my wheels but I just really enjoyed it. I don’t know why. Kind of reminds me of a comment someone made here on HN about Linux a while ago - something to the effect of “I want an OS I do t want a hobby.” Except I enjoy messing around with it and spending time making it work. Not very productive though I admit.
- userbinator 5y ago
  I tried LFS a long time ago, and frankly I don't consider it much of an educational experience --- downloading, compiling, and installing a bunch of stuff repeatedly with small variations of the same commands just doesn't make you learn much. On the other hand, I recommend trying to make a minimal system with only a kernel and shell and then add to it as desired. That way you'll really get an idea of what's needed to do what (and realise how bloated common distros are). A long time ago, I went through "DOS from scratch" and "Windows from scratch" in a similar fashion.
  - guerrilla 5y ago
    You get out of it what you put into it. Don't just follow the instructions, find out what everything is and what it does, why the instructions are what they are and why they are in the order they are. Maybe learn make and autotools. And for the person with way too much time on their hands, read the manual for each. For the retired or student on break, read some of the source code.
  - deleted 5y ago
    [deleted]
BeetleB 5y ago
LFS is good for learning, but if you want a long term maintainable solution, use Gentoo instead. (Disclaimer: 18 year Gentoo user speaking)
- grae_QED 5y ago
  Gentoo is great, but similar to LFS it is kind of a time suck. I migrated from Arch to Gentoo and it took me a while to figure out the right kernel parameters and USE flags for my system.
  - codr7 5y ago
    Oh that poor laptop, spent days and days compiling Gentoo. I was just mesmerized by the scrolling compiler messages, amazed to see something that complex transform into a working system. And once it was done I usually found a reason to start over again, just one more flag to make it perfect. Oddly, I can't remember ever actually using Gentoo, just compiling.
    - robohoe 5y ago
      I vividly remember compiling KDE (12-24 hours) and Firefox (3+ hours) back in like 2006. The hype and complains about Gentoo’s compilation times were legendary back then.
      - hnbear 5y ago
        I used Gentoo as a daily driver back in early college (2002-4)-ish. Was a lot of fun, but a huge amount of time compiling Every time I decided to try a new window manager and update USE flags to do something like add GTK support for Gnome, or QT for KDE, and obviously remove whatever I had to start with for optimization it would recompile everything. I remember in particular doing something like a fresh Gnome install, starting to use it and realizing I'd missed a USE flag for audio and having to re-compile all the Gnome apps to have them be able to use sound. Was fun, but brutal in terms of efficiency.
  - 5y ago
sigjuice 5y ago
Why include a keyboard and gpu only to use -nographic later?
diego_sandoval 5y ago
I would love if there was something like "Alpine Linux from scratch", i.e. using musl libc instead of glibc, busybox instead of GNU coreutils, etc.
- wrycoder 5y ago
  Just read the basic manual install script and execute it step by step.
ohazi 5y ago
For those with the interest, but with less time or patience, I found that installing Arch on a "For messing around, I don't care if I accidentally brick it for a week" laptop was a pretty good way to quickly learn a lot more about how a typical Linux system is assembled. There's no installer... the boot media just drops you into a live console, and then you're expected to follow the installation guide [1] to set up your disk, bootstrap the core system, select packages, and then do just enough manual configuration that your new machine will survive a reboot. You'll probably spend the next week or so digging around the Arch Wiki [2] to figure out how to set up the remaining odds and ends. You're definitely not building the system from scratch, but you generally do need to understand more about where things live, how they're configured, and how to turn them on and off, which the wiki is very good at explaining. The stuff you learn here will absolutely come in handy when the same component is misbehaving on a Debian or Red Hat system. And when you're done, not only are you left with a system that's completely usable (i.e. packages that can actually be upgraded), it'll be configured exactly the way you wanted. In my case, that "messing around" laptop is now my daily driver. [1] https://wiki.archlinux.org/title/Installation_guide https://wiki.archlinux.org/title/Installation_guide [2] https://wiki.archlinux.org/title/Main_page https://wiki.archlinux.org/title/Main_page (a great resource for Linux in general, not only Arch specific things)
- kenniskrag 5y ago
  They now have an optional experimental installer https://wiki.archlinux.org/title/Archinstall https://wiki.archlinux.org/title/Archinstall
- numlock86 5y ago
  To be fair Arch comes with a lot of prebuilt and preconfigured stuff plus utilities, abstracting A LOT away from you. Prime example is getting a desktop to run: For example install the gnome package, enable gdm and install the driver package for your GPU (xf86-video-whatever) and ... you are done. Having dealt with LFS and "back in the day"-BSDs that's usually a real pain to do. Sure, you need to install the system by "hand" (pacstrap, cough) and have to configure this and that (like redirecting genfstab output, cough), but it's far far far away to be remotely from scratch. Sure, you learn more than through the average Ubuntu-like installer, but how modules work or that they even exist is something even the most Arch users don't know.
mattowen_uk 5y ago
My issue with LFS (and I've attempted it several times, and always ran out of time to complete it) is that most of the time is spent compiling all the source code. What would be easier is a step-by-step guide to 'building' the system, via fdisk onto a spare HDD, and then manually copying over all the kernel bits , stuff for a working shell etc. and editing the init scripts with explanations of what each bit does, and why you are adding them to the system. I've read lots of docs on the structure and boot process of Linux, but stepping through adding each element to blank disk would cement in the knowledge, without all the lost time that LFS adds.
- agumonkey 5y ago
  Did LFS twice, I left with a broken system I couldn't understand but many hours of sed patch [vaguely] useful syntax mastery (I tried not copy paste the commands and typed them on my own) No disrespect, but this is backward.. it was fun to see the network stack acting chaotic. IRC worked fine, HTTP did not. Terminal had color issues sometimes. Made me appreciate the intricacies but left me totally clueless about why; which I thought was the point of assembling an OS bit by bit.
- cxr 5y ago
  Alternatively, and a lot more "real", would be if the system compiler available during the step-by-step phase cared foremost about getting functioning binaries out quickly. Bootstrap your way throughout the day with "slow" binaries built with the fast compiler and keep a log of this activity, then you feed this spec into a batch tool that runs overnight and uses the industrial strength compiler to recompile whatever it can before you wake up, interrupt it, and proceed with your work the next day. In fact, this has applications beyond what's involved in bootstrapping system software for self-education. It's very interesting that all the vendors of expensive compilers reacted to the production-readiness of GCC in the 90s by choosing to wither and die instead of focusing on improving the notoriously bad developer experience of the edit-compile-test cycle for large C and C++ to justify the the purchase of their products.

242 points by pthyme 5y ago

This is great if you are a teenager. I used to do stuff like this all the time. But as an adult.. It's just not worth the time.

petschge 5y ago
This isn't (or shouldn't) be an age thing. It is a 'stage of learning" thing. If you are new to Linux (or even all Unices) and you want to get a firm understanding of the fundamentals, it may very well be worth while to go through it at least once and see all the moving parts. And see them in an order that makes their dependencies clear(er). Once you know all that and just want to keep up with the constant churn of versions to keep that one webserver running and recent? Yeah. Not worth it. Get some distro that matches your preferred characteristics and stop worrying if you need to update libBLA to recompile Apache to fix the path traversal vulnerability. That also removes the question if that library still compiles correctly with gcc 7 or if you need to upgrade to at least gcc 9... Edit: Doing it in a VM first, might help in getting the 99 percent working first linux installation into a usable shape to make it a good daily driver. In that sense it helps lowering the barrier to actually switching later on.
elteto 5y ago
Yes, adults should not experiment with building custom distros... as we all know all existing Linux distributions were built by teenagers.
dang 5y ago
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
GuB-42 5y ago
My company is looking for someone to build a linux system for a specialized tablet PC, with, among other things, a custom kernel and OTA updates. If you built a linux system from scratch, this will put you in a very good position to get the job.
alekq 5y ago
Adult people do stuff for fun, I guess.

sanirank 5y ago

I don’t know why but I find this so interesting. I totally don’t have time right now (with a young kid and trying to make some/any progress in my career as a dev now that I’ve successfully made the career change to working as a developer), but I really just like understanding how things work from the ground up - I can’t wait to have the time to build my own Linux distribution. It feels like it’ll be such a waste of time - like installing 5 or 6 different distros on an old PC I had just out of curiosity to see what they looked snd felt like. And I broke and fixed the boot loader so many times that it became kind of an enjoyable challenge getting the machine to work again - “ok, now how did I get grub2 to recognise where my partitions are…?” It seemed like spinning my wheels but I just really enjoyed it. I don’t know why. Kind of reminds me of a comment someone made here on HN about Linux a while ago - something to the effect of “I want an OS I do t want a hobby.” Except I enjoy messing around with it and spending time making it work. Not very productive though I admit.

userbinator 5y ago
I tried LFS a long time ago, and frankly I don't consider it much of an educational experience --- downloading, compiling, and installing a bunch of stuff repeatedly with small variations of the same commands just doesn't make you learn much. On the other hand, I recommend trying to make a minimal system with only a kernel and shell and then add to it as desired. That way you'll really get an idea of what's needed to do what (and realise how bloated common distros are). A long time ago, I went through "DOS from scratch" and "Windows from scratch" in a similar fashion.
- guerrilla 5y ago
  You get out of it what you put into it. Don't just follow the instructions, find out what everything is and what it does, why the instructions are what they are and why they are in the order they are. Maybe learn make and autotools. And for the person with way too much time on their hands, read the manual for each. For the retired or student on break, read some of the source code.
- deleted 5y ago
  [deleted]

BeetleB 5y ago

LFS is good for learning, but if you want a long term maintainable solution, use Gentoo instead. (Disclaimer: 18 year Gentoo user speaking)

grae_QED 5y ago
Gentoo is great, but similar to LFS it is kind of a time suck. I migrated from Arch to Gentoo and it took me a while to figure out the right kernel parameters and USE flags for my system.
- codr7 5y ago
  Oh that poor laptop, spent days and days compiling Gentoo. I was just mesmerized by the scrolling compiler messages, amazed to see something that complex transform into a working system. And once it was done I usually found a reason to start over again, just one more flag to make it perfect. Oddly, I can't remember ever actually using Gentoo, just compiling.
  - robohoe 5y ago
    I vividly remember compiling KDE (12-24 hours) and Firefox (3+ hours) back in like 2006. The hype and complains about Gentoo’s compilation times were legendary back then.
    - hnbear 5y ago
      I used Gentoo as a daily driver back in early college (2002-4)-ish. Was a lot of fun, but a huge amount of time compiling Every time I decided to try a new window manager and update USE flags to do something like add GTK support for Gnome, or QT for KDE, and obviously remove whatever I had to start with for optimization it would recompile everything. I remember in particular doing something like a fresh Gnome install, starting to use it and realizing I'd missed a USE flag for audio and having to re-compile all the Gnome apps to have them be able to use sound. Was fun, but brutal in terms of efficiency.
- 5y ago

sigjuice 5y ago

Why include a keyboard and gpu only to use -nographic later?

diego_sandoval 5y ago

I would love if there was something like "Alpine Linux from scratch", i.e. using musl libc instead of glibc, busybox instead of GNU coreutils, etc.

wrycoder 5y ago
Just read the basic manual install script and execute it step by step.

ohazi 5y ago

For those with the interest, but with less time or patience, I found that installing Arch on a "For messing around, I don't care if I accidentally brick it for a week" laptop was a pretty good way to quickly learn a lot more about how a typical Linux system is assembled. There's no installer... the boot media just drops you into a live console, and then you're expected to follow the installation guide [1] to set up your disk, bootstrap the core system, select packages, and then do just enough manual configuration that your new machine will survive a reboot. You'll probably spend the next week or so digging around the Arch Wiki [2] to figure out how to set up the remaining odds and ends. You're definitely not building the system from scratch, but you generally do need to understand more about where things live, how they're configured, and how to turn them on and off, which the wiki is very good at explaining. The stuff you learn here will absolutely come in handy when the same component is misbehaving on a Debian or Red Hat system. And when you're done, not only are you left with a system that's completely usable (i.e. packages that can actually be upgraded), it'll be configured exactly the way you wanted. In my case, that "messing around" laptop is now my daily driver. [1] https://wiki.archlinux.org/title/Installation_guide https://wiki.archlinux.org/title/Installation_guide [2] https://wiki.archlinux.org/title/Main_page https://wiki.archlinux.org/title/Main_page (a great resource for Linux in general, not only Arch specific things)

kenniskrag 5y ago
They now have an optional experimental installer https://wiki.archlinux.org/title/Archinstall https://wiki.archlinux.org/title/Archinstall
numlock86 5y ago
To be fair Arch comes with a lot of prebuilt and preconfigured stuff plus utilities, abstracting A LOT away from you. Prime example is getting a desktop to run: For example install the gnome package, enable gdm and install the driver package for your GPU (xf86-video-whatever) and ... you are done. Having dealt with LFS and "back in the day"-BSDs that's usually a real pain to do. Sure, you need to install the system by "hand" (pacstrap, cough) and have to configure this and that (like redirecting genfstab output, cough), but it's far far far away to be remotely from scratch. Sure, you learn more than through the average Ubuntu-like installer, but how modules work or that they even exist is something even the most Arch users don't know.

mattowen_uk 5y ago

My issue with LFS (and I've attempted it several times, and always ran out of time to complete it) is that most of the time is spent compiling all the source code. What would be easier is a step-by-step guide to 'building' the system, via fdisk onto a spare HDD, and then manually copying over all the kernel bits , stuff for a working shell etc. and editing the init scripts with explanations of what each bit does, and why you are adding them to the system. I've read lots of docs on the structure and boot process of Linux, but stepping through adding each element to blank disk would cement in the knowledge, without all the lost time that LFS adds.

agumonkey 5y ago
Did LFS twice, I left with a broken system I couldn't understand but many hours of sed patch [vaguely] useful syntax mastery (I tried not copy paste the commands and typed them on my own) No disrespect, but this is backward.. it was fun to see the network stack acting chaotic. IRC worked fine, HTTP did not. Terminal had color issues sometimes. Made me appreciate the intricacies but left me totally clueless about why; which I thought was the point of assembling an OS bit by bit.
cxr 5y ago
Alternatively, and a lot more "real", would be if the system compiler available during the step-by-step phase cared foremost about getting functioning binaries out quickly. Bootstrap your way throughout the day with "slow" binaries built with the fast compiler and keep a log of this activity, then you feed this spec into a batch tool that runs overnight and uses the industrial strength compiler to recompile whatever it can before you wake up, interrupt it, and proceed with your work the next day. In fact, this has applications beyond what's involved in bootstrapping system software for self-education. It's very interesting that all the vendors of expensive compilers reacted to the production-readiness of GCC in the 90s by choosing to wither and die instead of focusing on improving the notoriously bad developer experience of the edit-compile-test cycle for large C and C++ to justify the the purchase of their products.