8 ms·

Linux from Scratch

466 points by udev4096 2y ago

lrvick 2y ago
For those that want to see full source bootstrapped, deterministic, and container native LFS check out https://codeberg.org/stagex https://codeberg.org/stagex
sirsinsalot 2y ago
Ah yes, I remember the year I lost to this and the pain of cross compilation. Reader beware.
- astrobe_ 2y ago
  I've crossed-compiled things from time-to-time along the years, and it seems to me that support has improved a bit. Some time ago it was "cross..wut?" but now it is fairly commonly supported and tested. I think it's thanks to Raspi and other popular ARM-based things.
sieste 2y ago
I really like the idea, really tried following the process several times. But each time it just turned into a copy-incomprehensible-commands-into-the-terminal exercise at some point, and I lost motivation. Did anyone experience the same?
- exe34 2y ago
  it's better if you try to work out what the commands and options are doing. I did this a long time ago, and 15 years later, I realise that it gave me a big advantage over my colleagues who haven't done something like that. "missing xxxxx.so" - they don't even know what that means, whereas I'm already trying to find the so and putting it in ld_library_path to see if it helps.
  - spockz 2y ago
    In my experience, whenever a .so was missing it was either due to a missing package on the os package manager level, or a completely broken gcc or llvm setup. I never needed to explicitly add single .so files to a path. (Besides the use case where I specifically wanted to override a malloc implementation.) In which cases did/do you need to add individual files?
    - exe34 2y ago
      in this case it's an in-house monstrosity that's got a lot of various languages and build systems involved, and the moment you update a compiler or build system, everything jumps into a new location for no good reasons. it was just an example though - same issue with .h not found during compilation, or using a different compiler from the system wide one.
    - screcth 2y ago
      In my case, I work with proprietary EDA tools. Vendors love messing with LD_LIBRARY_PATH. Chaos ensues when they override a system library or two versions require mutually incompatible versions. I agree with the comment you are replying to. Having broken my home Linux installs too many times has taught me how to diagnose and fix this sort of issues.
    - nmoura 2y ago
      In my case, when I was a Slackware user before Slackbuilds was created, sometimes I wanted to try out programs for which there was no package. Usually they required a .so file that was not installed and a workaround was to put it manually from a package of another distribution compiled for the same architecture. When the .so file was there, but on another path, a symbolic link was sufficient. The ldd command was a good friend. Of course that was not the best and cleanest solution, having things outside the package management system bothered, but it was enough to experiment programs and I kept track of the changes so I could have the prior state of things. Later on, the Slackbuilds project eased the work and I contributed by writing code to automate the creation of a few packages. I learned a lot from these issues.
- keyle 2y ago
  Yep, basically. At some point you realise you could do it, but do you really want to go through it all... It's a slog. It is a fun experience though!
  - stavros 2y ago
    > It's a slog. It is a fun experience though! Aren't those two opposites?
    - wezdog1 2y ago
      Type 2 fun
    - Brian_K_White 2y ago
      You've never experienced effort as fun? That is... not flattering.
      - stavros 2y ago
        Effort, yes. Slog, no.
    - whartung 2y ago
      I have a saying for this: An "adventure" is a bad trip, remembered fondly.
- pushupentry1219 2y ago
  When I was in highschool I had a bunch of free time outside of school days. I used a YouTube guide to install Arch. I failed many many times. But in doing so I almost kind of learnt what each command was doing. I had the same experience when I installed Gentoo. In both cases at first I was copy/pasting commands, but through failure I ended up understanding a lot of what the commands did. And what someone else said below is true as well, actually try and force yourself to understand what you're typing. Even briefly just scanning a man page helps a lot.
  - bee_rider 2y ago
    I wonder if the wiki would have been an easier start, than YouTube. YouTube is nice for learning things with a visual component, but installing Linux is all text, all the time. The ability to easily hop around and re-read sections on the wiki where necessary (I mean, jumping around is possible on YouTube of course, but it is really easy on the wiki) seem like it would be a big help.
    - pino82 2y ago
      Reading and actually understanding non-trivial text is hard if you are part of a generation that was never challenged to actually learn it. For those people, YouTube (and a few similar shops) are the default way to consume any content. That's what they do all the time. Sure, they somehow know those legacy emojis that you call the latin alphabet. It will just not lead to a deep understanding of text. Another aspect is probably that watching YT clips always has a feeling of being part of something. Some movement, some bubble, some society, whatever. They don't install Arch bcs they want to learn sth, or make some use of the OS. They do it _because_ they found it on YT and they want to be part of it. Maybe they even write comments or make a 'reaction' video. Today it's Arch, tomorrow it's a special pizza recipe from that other guy on Insta. It doesn't really matter.
      - bee_rider 2y ago
        I dunno. I’m a millennial so all sorts of stuff was just ascribed to my generation. As a result, I tend to just assume these differences are overstated. I worked with college students fairly recently. They did often reach reflexively for video. But when the written material was good enough, they used it.
- pipes 2y ago
  Yes this and endlessly unzipping things. I gave up after many many hours. I doubt I learned much from it.
- kwanbix 2y ago
  Same for me. I tried it about 20 years ago. I didn't fell like I was learning anything, just copy paste like you say. I haven't retried since, but I think it would be much better if they explain what you are doing and why.
- rwalle 2y ago
  Haven't tried it, but I guess if you were doing this again today, ChatGPT would help a lot.
- vbezhenar 2y ago
  I built my LFS around 20 years ago. While I followed instructions most of the time, I built some components myself. I remember that I wrote by own init scripts and I think I wrote my own initramfs (not completely sure about it, but I definitely remember tinkering with it and it probably was LFS). I also wanted to implement an isolated build system and my own package manager (basically tar wrapper with file database to be able to uninstall, no fancy dependency stuff), but containers were not invented back then and my skills were not good enough to properly utilize fakeroot, so it never was implemented, although I spent some time experimenting. Today I would do it with docker, exciting time.
  - johnisgood 2y ago
    Wanting to (and doing it) make my own package manager because of LFS and minimalism in general, good times.
- noufalibrahim 2y ago
  I haven't tried this but I have used https://github.com/MichielDerhaeg/build-linux https://github.com/MichielDerhaeg/build-linux during my trainings and all my students quite enjoyed the experience. It basically builds a Kernel, libc, busybox, init etc. and gets the whole thing running inside qemu. I found it quite educational and worth the little time I spent on it.
  - sheepybloke 2y ago
    I agree! I find systems like OpenEmbedded's Yocto to be super helpful with learning how to build Linux.
- zvmaz 2y ago
  > Did anyone experience the same? I tried several times, and tried again just recently. I share your sentiment. It perhaps gave me a renewed appreciation of the huge benefits Linux ditributions and package managers give us.
- notorandit 2y ago
  Nope. I never execute commands without understanding its meaning. Maybe you expect to complete a LFS installation in very little time. Which cannot be the case in general and in this very case in particular
- kobalsky 2y ago
  I grabbed a random page from the manual: https://www.linuxfromscratch.org/lfs/view/stable-systemd/chapter05/gcc-pass1.html https://www.linuxfromscratch.org/lfs/view/stable-systemd/cha... Every step is explained and every used parameter is documented.
  - andrewmcwatters 2y ago
    You grabbed a random page.
  - ruszki 2y ago
    I would say "explained". As a layman, who worked with linux previously (and even tried LFS a long time ago), but definitely don't have a deep understanding of linux, and eve my knowledge is not up-to-date: The sed command is not explained at all. Even the description is non-sense if you don't know already what they are talking about. What is this "default directory", why do you need to set it, why there and only there, why that command works? Even the command itself is something, which I need to check the manual what the heck it does, because it's not a simple one. > -enable-default-pie and --enable-default-ssp The description is almost unusable. So we don't need it, but it's "cleaner", which in this context means exactly nothing. So what happens if I left out? Nothing? Then why should I care? > --disable-multilib Okay, it doesn't support "something". I have no idea what is multilib, or why I should care. Basically the next arguments' description tells me that because it wouldn't work the compilation otherwise. And then.. > --disable-threads, --disable-libatomic, --disable-libgomp, --disable-libquadmath, --disable-libssp, --disable-libvtv, --disable-libstdcxx But why would they fail? I want to understand what's happening here, and I need to blindly trust the manual because they just tell me, that "they won't work, believe us". > --enable-languages=c,c++ Why are these the only languages which we need? What are the other languages? So at the end, descriptions are not really helping to understand what's happening, if you don't know already. The last time when I started LFS (about 10 years ago), that was my main problem. That you already need to know almost everything to understand what's really happening, and why, or reading manuals, or trying to find basically unsearchable information (like why libatomic compiling would fail at this step). So after a while, I started the blind copy-pasting, because I didn't have the patience of literary months, and when I realized that this was pointless, I gave up.
- aflukasz 2y ago
  It feels to me like you have approached this with a wrong mindset, like you focused too much on finishing the whole process. With LFS you must put substantial amount of value into the journey itself. It's not about copy-pasting commands, it's about trying to actually understand what and, more importantly, why your are doing each step. My recollection is that LFS docs were good in helping with that. Or maybe it was coming from reading docs of the actual components as I went along? I don't remember, probably some mix of both. I did set LFS system up once, I think it was 2001 or 2002. I specifically remember that it took many days (due to compilation times), was very interesting, and at the end I was putting final touches to my mutt and slrn configs. With a vague memory that I had some TODO for mutt that I didn't finish, still lingering in my mind, apparently! :) All in all, great and satisfying learning experience. I would not use such system as a daily driver, though. I think it's good only if you have time and want to learn. I'm curious if someone here is brave enough to have different approach.
  - oorza 2y ago
    LFS to a computer scientist should be like soap to a chemist. Something you can do as a fun, educational experiment, but not where you source a necessary tool for your life.
  - NegativeK 2y ago
    I daily drove it for a few months, back in the early 2000s (like many in this thread, it seems.) I kept it updated by subscribing to Freshmeat emails for releases, and I'd download and install the tarball. My review is that I'm glad I did it, it's a terrible idea, and it's debatable (honestly, not facetiously) as to whether the classes I skipped were more valuable.
- mnahkies 2y ago
  Last time I tried (probably well over 10 years ago) I struggled more with how long stuff took to compile. The main learning I remember from it was the chroot concept, which then later helped me grasp containerization easier
- hi-v-rocknroll 2y ago
  Nope. It's a launchpad not meant solely to follow blindly. The point is trial and error experimentation along the way. Can I switch the order of these packages or enable dependency X? It also helps to have a fast x86_64 box with zillions of cores and plenty of RAM and SSD, and also a compiler cache like sccache to speed up rebuilds.
- sweeter 2y ago
  Yes. I got a little bit out of it, certainly, but it really is mostly an exercise of compiling packages. I was a little disappointed but in the end LFS inspired me to take on a rewrite of the Unix Coreutils in Golang. I learned far more about Unix systems from doing this than anything else. It also improved my skills a lot, I look back on some of this code and cringe a little bit. [1] https://github.com/sweetbbak/go-moreutils https://github.com/sweetbbak/go-moreutils
- donio 2y ago
  For most people the whole reason to do the exercise is to understand what is involved in a bootstrap process. When you don't understand something you stop and figure out what is going on. There is not much point in blindly running commands that you don't understand.
  - lanstin 2y ago
    Except of course getting the OS installed and running.
- deleted 2y ago
  [deleted]
- kaycey2022 2y ago
  At the risk of sounding very distasteful, you could always just ask one of the gippities to understand what a command does. Though this would be less painful, I suppose you would lose out on having to go through the man pages and learning how to navigate those effectively. I myself don't really like looking into the man pages for answers. I just go in there for confirming the answers.
devilkin 2y ago
I did this, once, on a 386. Did I learn things? Definitely. Will I ever do it again? Definitely not ;)
- yonatan8070 2y ago
  I wonder, if you were to script all the commands you ran back in the day, and ran that same script on your old 386 and on a modern system with a top-of-the-line AMD Epyc or Intel Xeon, how much faster would it run?
  - fourfour3 2y ago
    Especially with the increase in storage performance - going from a hard disk that might have even still been using PIO modes to modern NVMe would be gigantic
    - llm_trw 2y ago
      The kernel is rather larger today: https://stopbyte.com/t/how-many-lines-of-code-linux-has/455/2 https://stopbyte.com/t/how-many-lines-of-code-linux-has/455/... The same is true for all other pieces of software. Build time will always increase until no one can be bothered to build it any more.
      - fourfour3 2y ago
        I think you've missed the point: > I wonder, if you were to script all the commands you ran back in the day, and ran that same script on your old 386 and on a modern system with a top-of-the-line AMD Epyc or Intel Xeon, how much faster would it run? Implies you're compiling the 386 era versions of Linux - so the fact modern Linux is larger is immaterial.
  - llm_trw 2y ago
    There is a unit of compilation as part of the LSF book which lets you estimate the who build process. You only need to compile libc or some such.
harha_ 2y ago
What's the fun in this? I once looked into it briefly and it's what I guessed: mostly about building and installing the required software individually. That's boring, I guess the fun would be in building an actual usable distro "from scratch".
- exe34 2y ago
  > building an actual usable distro "from scratch". could you say a few words on how this would avoid building and installing the required software individually? are you thinking of osdev instead?
  - creatonez 2y ago
    I assume they were alluding to package managers, automation, and release engineering, which are somewhat covered in the LFS Hints Project (https://www.linuxfromscratch.org/hints/ https://www.linuxfromscratch.org/hints/)
- thom 2y ago
  The first run isn't much fun. There are useful skills in just knowing how to build stuff, and knowing all the constituent parts of a running system. But overall I suspect you'd learn more in a year of using Arch. But on subsequent play-throughs, you get to be creative! Want to ignore the FHS and try putting your apps somewhere weird? Want to use a different libc or init system? Build everything around an esoteric shell? Go for maximum optimisation? It's the replayability that makes Linux from Scratch fun, and yes, totally worth working out what your own distro might look like.
- mbivert 2y ago
  You can think of a LFS as training before actually tackling an original distribution: distributions are complex, even following the steps of a LFS won't necessarily yield a working system (lots of room for mistakes). Or, as a base to build a distribution: you can automatize some of the process, slap a pkgsrc on top, etc. For budding programmers, it's a great exercise: discover software, appreciate dependencies, understand components of a Linux distribution, learn how to build things from source, configure a kernel, etc.
Manfred 2y ago
You could argue that running through the woods doesn't teach you about trees, but it's really effective when you bring a book along. Linux From Scratch is a fun way to explore what parts make up a Linux distribution. I did this a few time long ago before switching to Gentoo and it really helped with appreciating the freedom to pick and choose details of your operating system without actually writing it from scratch.
squarefoot 2y ago
Related: https://buildroot.org/ https://buildroot.org/
- stingraycharles 2y ago
  This basically allows you to make an OS that’s effectively a single application, right?
  - bubblesnort 2y ago
    No. It creates a root filesystem that follows your choices of what should be included and tries to be as minimal and small as possible by default. Very suitable for embedded devices and lightweight virtual machines. You can have it build a kernel as well. I use the ability to create a cpio archive for an initramfs on the Linux boxes.
    - ahepp 2y ago
      I reckon you are already aware of this if you’re using it to generate an initramfs, but for those reading along, you can also use it as a docker image
    - joezydeco 2y ago
      I use buildroot to make a commercial product. It builds the bootloader, the kernel, initramfs and rootfs, and then my application(s) all installed in the right place and connected to systemd. The output is a set of images that burn into their respective partitions on eMMC. If you're a masochist, there's also Yocto. It'll build your compiler and crosstools as well as the entire system image.
      - ahepp 2y ago
        Buildroot will build the cross toolchain too unless you specify BR2_TOOLCHAIN_EXTERNAL, right? And likewise you can configure yocto to use an external toolchain. A lot of the byproducts are configurable. You could configure the system to not use an initramfs, use only an initramfs/squashfs, etc. It is a really great tool with great documentation. I agree that Yocto is a bit masochistic by comparison, but vendors like it because its primary purpose is to enable shitty vendor forks.
sirodoht 2y ago
I went through this over a couple of weeks last year. Very fun, I would recommend it to anyone interested to see the complexity of making a linux distro.
jaystraw 2y ago
to anyone confused why others find this fun or worthwhile, i will quote brak's dad: "you just don't get it, das all"
rkagerer 2y ago
How up to date is this? When trying to learn by copy-pasting from the internet, I sometimes get tripped up over older/newer Linuxes using different versions of commands or best practices. (e.g. ipconfig vs. ifconfig, iptables vs. successors)
- creatonez 2y ago
  It was last updated in September 2024. Distrowatch's package list can give you an idea of what version numbers to expect: https://distrowatch.com/table.php?distribution=lfs https://distrowatch.com/table.php?distribution=lfs Once you get to the BLFS part, you have more choice on whether you're going to innovate or use something that old and in maintenance mode. For example, it has instructions for both Pulseaudio and Pipewire, Xorg and Wayland, SysVinit and systemd. The instructions will rarely be outright incorrect on modern distros unless something is very strange about your environment, since a bulk of the work is done after establishing a more predictable build environment.
- avisk 2y ago
  It is up to date. I successfully built an LFS system this week by following the guide.
mydriasis 2y ago
If you're curious, you should try it. It's a great way to learn about all of the little bits and pieces that go into a working OS that you can use, and by the time you're done you'll be more comfy thinking about Linux in general. It may take some time depending on how often you come back to it, but it's definitely worth it.
dark-star 2y ago
I have used CLFS (Cross-Linux From Scratch) in the past a couple of times to build a Linux system for Sun UltraSPARC and SGI Octane systems. The things I learned, especially about cross-compiling, were invaluable, but you do have to invest some time understanding everything (even reading patches that you have to apply on top of build tools, for example) to get the most out of it. If you just copy/paste commands, well then there are faster/easier ways to get Linux up and running Edit: I just noted that CLFS has been dead for a couple of years now, which is sad. I would have loved to try an updated CLFS
cloudripper 2y ago
I gave LFS a go earlier this year. I learned a lot through the process - but I definitely went outside the guardrails. I use NixOS as my daily driver and found myself curious of whether I could complete LFS using a "Nix" approach. I was only a basic Nix user at that time and that choice made a difficult process much more difficult. However, the declarative nature of Nix meant that I had clear notes of every step of my process - and if something didn't work, I could backtrack and troubleshoot to find the root of the cause. The end result is here [0]. My understanding of Linux, of bootstrapping, cross-compilation, and Nix has grown tremendously as a result of the time I took on this project - and I still go back and reference the work from time to time. When I get some time to revisit the Nix-based LFS project, there are quite a few things I would like to clean-up, including setting kernel configs and handling post-build permissions. Nix-complexities aside, I highly recommend LFS if you like to understand how things work and don't mind a little suffering along the way. [0]: https://github.com/cloudripper/NixLFS https://github.com/cloudripper/NixLFS
- tomberek 2y ago
  This is excellent. Have you considered making a presentation or a write-up of the experience?
  - cloudripper 2y ago
    Thanks for the input. A couple of folks suggested that recently as well. Once I can clear up some bandwidth, I do intend to follow through on that. It gave me a huge appreciation for Nix as a build system, and I would love to share that if it were helpful to others.
- ayakang31415 2y ago
  The first thing that pops out on Google search of NixOS is "Declarative builds and deployments." What exactly is NixOS different from other distros, such as Ubuntu?
  - hollerith 2y ago
    NixOS separates packages. If the package foo contains a file /usr/bin/foo, NixOS installs it in /nix/store/67c25d7ad7b2b64c67c25d7ad7b2b64c-foo/usr/bin/foo. In order to make this separation work, Nix must sometimes rewrite binaries so that all references in the binary to /usr/bin/foo becomes references to /nix/store/67c25d7ad7b2b64c67c25d7ad7b2b64c-foo/usr/bin/foo. The advantage of this approach is that it gives more control to the distro maintainers and the admins of the computer, taking that control away from the "upstream" maintainers of the software being packaged. For example the software being packaged cannot just call the library bar because bar is not at /usr/lib/bar.so like it is in most Linux distros -- it is at /nix/store/17813e8b97b84e0317813e8b97b84e03-bar/usr/lib/bar.so, but of course the software does not know that unless the person creating the Nix package (the packager) arranges for the software to know it (again sometimes by doing a search-and-replace on binaries). If the upstream maintainer of foo thinks foo should link to version 6 of library bar, but you think it should link to version 5, NixOS makes it easier for you to arrange for foo to link to version 5 than most distros do (even if version 6 of bar is needed by other packages you have installed which you need to use at the same times as your using foo). Note that if this separation of packages imposed by NixOS has any beneficial security properties, it is merely security through obscurity because there is nothing preventing a binary from covertly searching through the directory listing of /nix/store/ for the name of the library it wants to call. Nevertheless it turns out the be useful to seize some control away from upstream in this way even if technically upstream could seize the control back if it were willing to complicate the software to do so. People, including the creator of Nix and NixOS (Dolstra), will tell you that NixOS's main advantage is "declarativeness" (which in the past Dolstra called "purity") or the fact that the compilation / building process is deterministic. I believe both positions (NixOS's advantage is declarativeness and the advantage is deterministic builds) are wrong. Specifically, I believe that although deterministic builds are useful, the separation of packages I described is much more useful to most users and prospective users of NixOS. Another way to summarize it is that NixOS package maintainers routinely modify the software they are packaging to use less "ambient authority".
smitty1e 2y ago
I have gone all the way through this, #21584. What a tremendous resource.
kopirgan 2y ago
This has been around for ages. I only read the contents briefly to know what the minimum components of Linux is, that too years ago. When you install any distro (even Debian) so much gets installed which newbies wouldn't even know exists or use.
throwaway2016a 2y ago
This brings back memories. I tried to do his (by using this exact site!) in college. Which, for reference, was 20ish years ago. I wasn't able to get very far, mostly because I kept running into obscure compiler and linker errors and 18 year old me wasn't clear how to fix them. Would be fun to see how much is changed since then because it does appear to be at least partially maintained.
fefe23 2y ago
LFS has been a tremendous resource over the years for me. I'm happy it exists and that people put in the time to keep it going. I often wonder why the existence of long "you also need to do this completely unintuitive thing first" documentation on the open internet isn't shaming more projects into reducing barriers to build their software.
dvno42 2y ago
Between LFS and Stage 1 and 2 Gentoo installs back in the early 2000s during High School, this gave me a big leg up in my journey of learning about computers and Linux. I can't thank those project maintainers enough for helping me get my footing at such a young age and peaking my interest in computing. I ended up printing out the LFS book in segments on one of the printers in High School and brought it home in pieces to use on the home computer.
- tehf0x 2y ago
  Amen to that! 20 years later this was my gateway drug into being addicted to computers and gave me my full stack understanding I still use every day at work. <3 Gentoo and the friendly geek at the coffee place I worked at when I was 14 who gave me my first hit and held my hand through what is effectively mental torture for most people
- oorza 2y ago
  > I ended up printing out the LFS book in segments on one of the printers in High School and brought it home in pieces to use on the home computer. I am actually flabbergasted there's another human being on Earth that has this exact same story as me. I used one of those giant 3 ring legal binders for my printed copy, lmao.
- hi-v-rocknroll 2y ago
  Yep. Recompile all the things with different USE flags.
  - _joel 2y ago
    Recompiling the entire openoffice so it loads 1 second quicker.. just using a hacked up distcc cluster and a day's worth of CPU time. Bargain!
    - account42 2y ago
      It can actually be a bargain though. CPU time I can buy with money, life time is a lot more difficult to extend (not to mention the frustration a second of startup time can cause).
f0e4c2f7 2y ago
Building using LFS has been on my list of "I really should probably do that just to learn" for about 20 years now. I'll get around to it! This year I'm finally learning Lisp (and really enjoying it).
- theanonymousone 2y ago
  Isn't an LLM making such things more approachable?
  - f0e4c2f7 2y ago
    For sure. It's been amazing honestly and I feel like it has really accelerated my learning. Enough that I'm trying to get really serious about making the most out of this new leverage seemingly out of nowhere. I use it a lot for mapping out of the initial concepts but I find one of the best use cases is after understanding the basics, explaining where I need to learn more and asking for a book recommendation. The quality of my reading list has gone up 10x this way and I find myself working through multiple books a week. Great for code too obviously, though still feels like early days there to me.
- mekster 2y ago
  If you'd done that 20 years ago, we'd been on the same page :) Though back then, I learnt well from Gentoo by setting the system up from stage 1, which was a good knowledge to keep when trying to fix things such as handling partitions in single user mode or modprobe'ing to detect hardware and such. If you can't do anything beyond relying on GUI, you could get stuck real fast.
rbattula 2y ago
tried lfs back in high school really fun experience and teaches you a lot
andrewmcwatters 2y ago
I remember trying to automate non-distribution builds of a functional GNU/Linux operating system and trying to not read Linux from Scratch but just official kernel.org documentation.[1] Unfortunately, the state of Linux documentation is so poor, you can't do it. You need to reference a number of third-party articles that kernel.org itself sometimes links you to. I believe kernel.org might also mention Linux from Scratch, but LFS does a very poor job of explaining why you need particular dependencies straight out of initramfs. You need a functional shell, and you need to get your installation on to an actual drive. Neither of those things are explained in sufficient detail with the current documentation available today. LFS at best says "throw in these ingredients," and leaves you with no other information. You can probably read man pages and piece this stuff together, but it requires that you, at the very least, hardcode where to install files with fsck/mount/cp, I think. Edit: LFS also does a really poor job of explaining why it chooses its "Basic System Software," much of which isn't actually required for a common GNU/Linux system. [1]: https://github.com/andrewmcwatters/linux-workflow https://github.com/andrewmcwatters/linux-workflow
- machinestops 2y ago
  > why you need particular dependencies straight out of initramfs. It came as a shock to me to learn that initramfs is mostly optional. It can be skipped, and you can boot straight into userland.
  - andrewmcwatters 2y ago
    Yes, but if I understand correctly, you’d need to configure for this specifically. I think standard configurations and installations use them.
    - machinestops 2y ago
      Yes, it requires specific configuration. The reason most distributions of Linux use them is it allows you to dynamically discover and load kernel modules at boot time, and provide a fallback for if mounting the disk fails.
ngneer 2y ago
This brings back memories. I built LFS on a Pentium based machine a few times and it was quite fun. I took a similar approach to building a custom bare metal system for a client that needed extra security. I sometimes wish that there was a well maintained bare bones distribution inspired by the LFS philosophy, that might help with managing complexity on the cloud. Scanning the thread, NixOS came up. I used to love Slack, too. Any other recommendations? Something with minimal dependencies yet capable of being a LAMP-like server?
- throwaway2037 2y ago
  Alpine isn't good enough as a base system?
  - _joel 2y ago
    muslc isn't as fast as glibc (last time I checked), but I guess it's fine if you're ok with that.
- greggyb 2y ago
  Alpine Linux is a very minimal base built around busybox and musl libc. Void is another minimalist distro offering a choice of glibc or musl, and using runit as an init. If you want more choices of init and a bleeding edge rolling release, Artix is a derivative of Arch with multiple init options in the core distribution. Arch and Gentoo are the classics of do-it-yourself minimalism.
krylon 2y ago
Back in the day, I made an attempt at LFS but gave up halfway through. In my defense, I had done a gentoo stage 1 emerge earlier that year. ;-) I never intended to end up with a usable system, I was just in it for the learning experience. And I did end up learning a lot.
deleted 2y ago
[deleted]
ocean_moist 2y ago
I did LFS after I started dailying gentoo with a custom kernel (~2017), it was actually fairly easy, if not time consuming. People on IRC are extremely helpful. These days I write code more than infra, but I still think it was valuable. That combined with my general interest in Linux probably saved me thousands in cloud costs. Linux is a dying art among young software engineers, but alive among young nerds. Even among infra/devops people, maybe if they learned more about linux they wouldn't reach for k8s to solve the simplest problems.
Razengan 2y ago
Anyone else annoyed by “online books” sites like these where you have to play a convoluted game find-the-breadcrumbs from the website landing page to the beginning of the actual book content..
- greggyb 2y ago
  Two clicks from the landing page. 1. Choose the book you want to read (they call them sub-projects) 2. "Download" or "Read online" Of course, they could reverse the nav order and provide first a "read online"/"download" and then let you pick which of the books you want to read, yielding ... two clicks to get to the one you want to read.
hi-v-rocknroll 2y ago
I wrote an (unreleased) Bash-based framework for Docker container- and exo-Docker VM-based LFS builds. Dir listing: https://pastebin.com/JrPkftgr https://pastebin.com/JrPkftgr Dockerfile (it's still missing throwing away intermediate layers b/c it's optimized for caching and dev speed): FROM quay.io/centos/centos:stream9 COPY files/bashrc /root/.bashrc COPY files/bash_profile /root/.bash_profile COPY --chmod=755 files/sysdiff /usr/bin/sysdiff WORKDIR /mnt/lfs COPY files/functions.inc files/ COPY --chmod=755 files/run files/ COPY scripts/0000-install-deps scripts/ RUN files/run install 0000-install-deps # Docker host runs a memcache instance to cache build artifacts in RAM COPY scripts/0010-install-sccache scripts/ COPY files/sccache-wrapper.c files/ RUN files/run install 0010-install-sccache COPY scripts/1000-setup scripts/ COPY files/hosts files/passwd files/group files/shells files/inputrc files/fstab.in files/locale.conf.in files/ RUN files/run install 1000-setup ...
jakogut 2y ago
Another project to check out if you enjoy LFS is buildroot. Buildroot uses Kconfig (and associated tools like menuconfig, config, etc.) same as the kernel, to generate a configuration for building an embedded Linux system. This configuration can be committed and patched in the repo for your project. Adding an application or library automatically pulls in dependencies. Configs can be merged with fragments that add specific features. Buildroot is capable of building its own cross compilation tool chain for many architectures, and enables developers to choose their own libc, and other toolchain configurations. It can also be configured to pull a prebuilt toolchain tarball to save time. Packages are written in GNU Make, and macros are available for pulling code from GitHub and building packages using autoconf, cmake, meson, golang, Rust, python, and many more. It works fantastically for anything from building an embedded Linux distro for something like digital signage, to a self-contained network bootable application for large scale automated provisioning of devices, to building minimal tarballs of filesystems to run as containers.
- itomato 2y ago
  It’s cool but adding more than a package or two in the buildroot way is where I would get into the weeds. I’m sometimes surprised a fork with better approach to package management doesn’t exist yet.
Scuds 2y ago
I wonder what an equivalent "BSD from scratch" is like? Linux was assembled from a collection of parts while BSD is (reputably) a lot more 'designed from the ground up' Even a modern system like Fuchsia - what's that like to build from the ground up? Or is it "You fool! Building your own kitbashed Gundam of an OS is the point."
- joveian 2y ago
  I'm not sure about the others but the full NetBSD base system can be cross compiled from Linux (and maybe some other unixy systems) using the included build script (I'm not sure what the current state of cross compiling pkgsrc is, likely not as good). You could add or remove things from the base system to learn more about how the build system works or create different install sets or something along those lines, although just looking at what the build system does might be as helpful.
cjk 2y ago
This, and its no-longer-maintained cousin, Cross Linux from Scratch[1], were instrumental in learning how to bootstrap a custom Linux distro for embedded devices (especially the CLFS Embedded variant[2]) at a previous job. [1]: https://clfs.org/ https://clfs.org/ [2]: https://clfs.org/view/clfs-embedded/arm/ https://clfs.org/view/clfs-embedded/arm/
keitmo 2y ago
I went through the LFS process back in 2003 or 2004. I didn't have a Linux system handy, so I built it with Fedora 3 running under VMWare on a Windows XP laptop. It was not speedy. I tried to automate the process as much as possible. There's a separate Automated Linux From Scratch project, but I choose to roll my own. This was my first experience with Bash scripts -- I developed skills (and bad habits) that I continue to use this day.
tapanjk 2y ago
I always thought Slackware [1] was the basic Linux distro which people could use to 'understand' Linux. How does Slackware compare to LFS in terms of understanding the Linux fundamentals? [1]: http://www.slackware.com/ http://www.slackware.com/
- SuperNinKenDo 2y ago
  Slackware is more like a BSD of Linux, to stretch an analogy. A very solid base system, all packaged up as a complete operating system. LFS is utterly different. There is no installer, there is no base system, there is nothing even approaching the power of Slackware's package manager, as barebones as it seems today. This is compile everything from source in a separate environment and put everything in the right place essentially manually*. Of course, `make`-and-friends will usually do the actual lifting, but you're in charge of giving direction, there isn't a distro community patching packages and build-scripts to adhere to a consistent FSH or the like.
janniks 2y ago
Is this still the best resource? Looking for something first principles, simple, dumbed down, stripping away the unnecessary.
trevjud000 2y ago
what's lfs and nix
garfieldnate 2y ago
I would love to see this updated with information on building a realtime Linux (which is now officially supported). I have in mind some hand-held devices that would be a lot more interesting to build with realtime support.
nineteen999 2y ago
My last LFS build was a couple of years ago and took 3-4 weeks, I decided to do a SVR4-themed distro for fun. This meant using the heirloom packaging tools (pkginfo/pkgadd/pkgrm etc) to package everything and I made the default desktop CDE with antialiased fonts. Loads of fun and nostalgia.

466 points by udev4096 2y ago

For those that want to see full source bootstrapped, deterministic, and container native LFS check out https://codeberg.org/stagex https://codeberg.org/stagex

sirsinsalot 2y ago

Ah yes, I remember the year I lost to this and the pain of cross compilation. Reader beware.

astrobe_ 2y ago
I've crossed-compiled things from time-to-time along the years, and it seems to me that support has improved a bit. Some time ago it was "cross..wut?" but now it is fairly commonly supported and tested. I think it's thanks to Raspi and other popular ARM-based things.

sieste 2y ago

I really like the idea, really tried following the process several times. But each time it just turned into a copy-incomprehensible-commands-into-the-terminal exercise at some point, and I lost motivation. Did anyone experience the same?

exe34 2y ago
it's better if you try to work out what the commands and options are doing. I did this a long time ago, and 15 years later, I realise that it gave me a big advantage over my colleagues who haven't done something like that. "missing xxxxx.so" - they don't even know what that means, whereas I'm already trying to find the so and putting it in ld_library_path to see if it helps.
- spockz 2y ago
  In my experience, whenever a .so was missing it was either due to a missing package on the os package manager level, or a completely broken gcc or llvm setup. I never needed to explicitly add single .so files to a path. (Besides the use case where I specifically wanted to override a malloc implementation.) In which cases did/do you need to add individual files?
  - exe34 2y ago
    in this case it's an in-house monstrosity that's got a lot of various languages and build systems involved, and the moment you update a compiler or build system, everything jumps into a new location for no good reasons. it was just an example though - same issue with .h not found during compilation, or using a different compiler from the system wide one.
  - screcth 2y ago
    In my case, I work with proprietary EDA tools. Vendors love messing with LD_LIBRARY_PATH. Chaos ensues when they override a system library or two versions require mutually incompatible versions. I agree with the comment you are replying to. Having broken my home Linux installs too many times has taught me how to diagnose and fix this sort of issues.
  - nmoura 2y ago
    In my case, when I was a Slackware user before Slackbuilds was created, sometimes I wanted to try out programs for which there was no package. Usually they required a .so file that was not installed and a workaround was to put it manually from a package of another distribution compiled for the same architecture. When the .so file was there, but on another path, a symbolic link was sufficient. The ldd command was a good friend. Of course that was not the best and cleanest solution, having things outside the package management system bothered, but it was enough to experiment programs and I kept track of the changes so I could have the prior state of things. Later on, the Slackbuilds project eased the work and I contributed by writing code to automate the creation of a few packages. I learned a lot from these issues.
keyle 2y ago
Yep, basically. At some point you realise you could do it, but do you really want to go through it all... It's a slog. It is a fun experience though!
- stavros 2y ago
  > It's a slog. It is a fun experience though! Aren't those two opposites?
  - wezdog1 2y ago
    Type 2 fun
  - Brian_K_White 2y ago
    You've never experienced effort as fun? That is... not flattering.
    - stavros 2y ago
      Effort, yes. Slog, no.
  - whartung 2y ago
    I have a saying for this: An "adventure" is a bad trip, remembered fondly.
pushupentry1219 2y ago
When I was in highschool I had a bunch of free time outside of school days. I used a YouTube guide to install Arch. I failed many many times. But in doing so I almost kind of learnt what each command was doing. I had the same experience when I installed Gentoo. In both cases at first I was copy/pasting commands, but through failure I ended up understanding a lot of what the commands did. And what someone else said below is true as well, actually try and force yourself to understand what you're typing. Even briefly just scanning a man page helps a lot.
- bee_rider 2y ago
  I wonder if the wiki would have been an easier start, than YouTube. YouTube is nice for learning things with a visual component, but installing Linux is all text, all the time. The ability to easily hop around and re-read sections on the wiki where necessary (I mean, jumping around is possible on YouTube of course, but it is really easy on the wiki) seem like it would be a big help.
  - pino82 2y ago
    Reading and actually understanding non-trivial text is hard if you are part of a generation that was never challenged to actually learn it. For those people, YouTube (and a few similar shops) are the default way to consume any content. That's what they do all the time. Sure, they somehow know those legacy emojis that you call the latin alphabet. It will just not lead to a deep understanding of text. Another aspect is probably that watching YT clips always has a feeling of being part of something. Some movement, some bubble, some society, whatever. They don't install Arch bcs they want to learn sth, or make some use of the OS. They do it _because_ they found it on YT and they want to be part of it. Maybe they even write comments or make a 'reaction' video. Today it's Arch, tomorrow it's a special pizza recipe from that other guy on Insta. It doesn't really matter.
    - bee_rider 2y ago
      I dunno. I’m a millennial so all sorts of stuff was just ascribed to my generation. As a result, I tend to just assume these differences are overstated. I worked with college students fairly recently. They did often reach reflexively for video. But when the written material was good enough, they used it.
pipes 2y ago
Yes this and endlessly unzipping things. I gave up after many many hours. I doubt I learned much from it.
kwanbix 2y ago
Same for me. I tried it about 20 years ago. I didn't fell like I was learning anything, just copy paste like you say. I haven't retried since, but I think it would be much better if they explain what you are doing and why.
rwalle 2y ago
Haven't tried it, but I guess if you were doing this again today, ChatGPT would help a lot.
vbezhenar 2y ago
I built my LFS around 20 years ago. While I followed instructions most of the time, I built some components myself. I remember that I wrote by own init scripts and I think I wrote my own initramfs (not completely sure about it, but I definitely remember tinkering with it and it probably was LFS). I also wanted to implement an isolated build system and my own package manager (basically tar wrapper with file database to be able to uninstall, no fancy dependency stuff), but containers were not invented back then and my skills were not good enough to properly utilize fakeroot, so it never was implemented, although I spent some time experimenting. Today I would do it with docker, exciting time.
- johnisgood 2y ago
  Wanting to (and doing it) make my own package manager because of LFS and minimalism in general, good times.
noufalibrahim 2y ago
I haven't tried this but I have used https://github.com/MichielDerhaeg/build-linux https://github.com/MichielDerhaeg/build-linux during my trainings and all my students quite enjoyed the experience. It basically builds a Kernel, libc, busybox, init etc. and gets the whole thing running inside qemu. I found it quite educational and worth the little time I spent on it.
- sheepybloke 2y ago
  I agree! I find systems like OpenEmbedded's Yocto to be super helpful with learning how to build Linux.
zvmaz 2y ago
> Did anyone experience the same? I tried several times, and tried again just recently. I share your sentiment. It perhaps gave me a renewed appreciation of the huge benefits Linux ditributions and package managers give us.
notorandit 2y ago
Nope. I never execute commands without understanding its meaning. Maybe you expect to complete a LFS installation in very little time. Which cannot be the case in general and in this very case in particular
kobalsky 2y ago
I grabbed a random page from the manual: https://www.linuxfromscratch.org/lfs/view/stable-systemd/chapter05/gcc-pass1.html https://www.linuxfromscratch.org/lfs/view/stable-systemd/cha... Every step is explained and every used parameter is documented.
- andrewmcwatters 2y ago
  You grabbed a random page.
- ruszki 2y ago
  I would say "explained". As a layman, who worked with linux previously (and even tried LFS a long time ago), but definitely don't have a deep understanding of linux, and eve my knowledge is not up-to-date: The sed command is not explained at all. Even the description is non-sense if you don't know already what they are talking about. What is this "default directory", why do you need to set it, why there and only there, why that command works? Even the command itself is something, which I need to check the manual what the heck it does, because it's not a simple one. > -enable-default-pie and --enable-default-ssp The description is almost unusable. So we don't need it, but it's "cleaner", which in this context means exactly nothing. So what happens if I left out? Nothing? Then why should I care? > --disable-multilib Okay, it doesn't support "something". I have no idea what is multilib, or why I should care. Basically the next arguments' description tells me that because it wouldn't work the compilation otherwise. And then.. > --disable-threads, --disable-libatomic, --disable-libgomp, --disable-libquadmath, --disable-libssp, --disable-libvtv, --disable-libstdcxx But why would they fail? I want to understand what's happening here, and I need to blindly trust the manual because they just tell me, that "they won't work, believe us". > --enable-languages=c,c++ Why are these the only languages which we need? What are the other languages? So at the end, descriptions are not really helping to understand what's happening, if you don't know already. The last time when I started LFS (about 10 years ago), that was my main problem. That you already need to know almost everything to understand what's really happening, and why, or reading manuals, or trying to find basically unsearchable information (like why libatomic compiling would fail at this step). So after a while, I started the blind copy-pasting, because I didn't have the patience of literary months, and when I realized that this was pointless, I gave up.
aflukasz 2y ago
It feels to me like you have approached this with a wrong mindset, like you focused too much on finishing the whole process. With LFS you must put substantial amount of value into the journey itself. It's not about copy-pasting commands, it's about trying to actually understand what and, more importantly, why your are doing each step. My recollection is that LFS docs were good in helping with that. Or maybe it was coming from reading docs of the actual components as I went along? I don't remember, probably some mix of both. I did set LFS system up once, I think it was 2001 or 2002. I specifically remember that it took many days (due to compilation times), was very interesting, and at the end I was putting final touches to my mutt and slrn configs. With a vague memory that I had some TODO for mutt that I didn't finish, still lingering in my mind, apparently! :) All in all, great and satisfying learning experience. I would not use such system as a daily driver, though. I think it's good only if you have time and want to learn. I'm curious if someone here is brave enough to have different approach.
- oorza 2y ago
  LFS to a computer scientist should be like soap to a chemist. Something you can do as a fun, educational experiment, but not where you source a necessary tool for your life.
- NegativeK 2y ago
  I daily drove it for a few months, back in the early 2000s (like many in this thread, it seems.) I kept it updated by subscribing to Freshmeat emails for releases, and I'd download and install the tarball. My review is that I'm glad I did it, it's a terrible idea, and it's debatable (honestly, not facetiously) as to whether the classes I skipped were more valuable.
mnahkies 2y ago
Last time I tried (probably well over 10 years ago) I struggled more with how long stuff took to compile. The main learning I remember from it was the chroot concept, which then later helped me grasp containerization easier
hi-v-rocknroll 2y ago
Nope. It's a launchpad not meant solely to follow blindly. The point is trial and error experimentation along the way. Can I switch the order of these packages or enable dependency X? It also helps to have a fast x86_64 box with zillions of cores and plenty of RAM and SSD, and also a compiler cache like sccache to speed up rebuilds.
sweeter 2y ago
Yes. I got a little bit out of it, certainly, but it really is mostly an exercise of compiling packages. I was a little disappointed but in the end LFS inspired me to take on a rewrite of the Unix Coreutils in Golang. I learned far more about Unix systems from doing this than anything else. It also improved my skills a lot, I look back on some of this code and cringe a little bit. [1] https://github.com/sweetbbak/go-moreutils https://github.com/sweetbbak/go-moreutils
donio 2y ago
For most people the whole reason to do the exercise is to understand what is involved in a bootstrap process. When you don't understand something you stop and figure out what is going on. There is not much point in blindly running commands that you don't understand.
- lanstin 2y ago
  Except of course getting the OS installed and running.
deleted 2y ago
[deleted]
kaycey2022 2y ago
At the risk of sounding very distasteful, you could always just ask one of the gippities to understand what a command does. Though this would be less painful, I suppose you would lose out on having to go through the man pages and learning how to navigate those effectively. I myself don't really like looking into the man pages for answers. I just go in there for confirming the answers.

devilkin 2y ago

I did this, once, on a 386. Did I learn things? Definitely. Will I ever do it again? Definitely not ;)

yonatan8070 2y ago
I wonder, if you were to script all the commands you ran back in the day, and ran that same script on your old 386 and on a modern system with a top-of-the-line AMD Epyc or Intel Xeon, how much faster would it run?
- fourfour3 2y ago
  Especially with the increase in storage performance - going from a hard disk that might have even still been using PIO modes to modern NVMe would be gigantic
  - llm_trw 2y ago
    The kernel is rather larger today: https://stopbyte.com/t/how-many-lines-of-code-linux-has/455/2 https://stopbyte.com/t/how-many-lines-of-code-linux-has/455/... The same is true for all other pieces of software. Build time will always increase until no one can be bothered to build it any more.
    - fourfour3 2y ago
      I think you've missed the point: > I wonder, if you were to script all the commands you ran back in the day, and ran that same script on your old 386 and on a modern system with a top-of-the-line AMD Epyc or Intel Xeon, how much faster would it run? Implies you're compiling the 386 era versions of Linux - so the fact modern Linux is larger is immaterial.
- llm_trw 2y ago
  There is a unit of compilation as part of the LSF book which lets you estimate the who build process. You only need to compile libc or some such.

harha_ 2y ago

What's the fun in this? I once looked into it briefly and it's what I guessed: mostly about building and installing the required software individually. That's boring, I guess the fun would be in building an actual usable distro "from scratch".

exe34 2y ago
> building an actual usable distro "from scratch". could you say a few words on how this would avoid building and installing the required software individually? are you thinking of osdev instead?
- creatonez 2y ago
  I assume they were alluding to package managers, automation, and release engineering, which are somewhat covered in the LFS Hints Project (https://www.linuxfromscratch.org/hints/ https://www.linuxfromscratch.org/hints/)
thom 2y ago
The first run isn't much fun. There are useful skills in just knowing how to build stuff, and knowing all the constituent parts of a running system. But overall I suspect you'd learn more in a year of using Arch. But on subsequent play-throughs, you get to be creative! Want to ignore the FHS and try putting your apps somewhere weird? Want to use a different libc or init system? Build everything around an esoteric shell? Go for maximum optimisation? It's the replayability that makes Linux from Scratch fun, and yes, totally worth working out what your own distro might look like.
mbivert 2y ago
You can think of a LFS as training before actually tackling an original distribution: distributions are complex, even following the steps of a LFS won't necessarily yield a working system (lots of room for mistakes). Or, as a base to build a distribution: you can automatize some of the process, slap a pkgsrc on top, etc. For budding programmers, it's a great exercise: discover software, appreciate dependencies, understand components of a Linux distribution, learn how to build things from source, configure a kernel, etc.

Manfred 2y ago

You could argue that running through the woods doesn't teach you about trees, but it's really effective when you bring a book along. Linux From Scratch is a fun way to explore what parts make up a Linux distribution. I did this a few time long ago before switching to Gentoo and it really helped with appreciating the freedom to pick and choose details of your operating system without actually writing it from scratch.

squarefoot 2y ago

Related: https://buildroot.org/ https://buildroot.org/

stingraycharles 2y ago
This basically allows you to make an OS that’s effectively a single application, right?
- bubblesnort 2y ago
  No. It creates a root filesystem that follows your choices of what should be included and tries to be as minimal and small as possible by default. Very suitable for embedded devices and lightweight virtual machines. You can have it build a kernel as well. I use the ability to create a cpio archive for an initramfs on the Linux boxes.
  - ahepp 2y ago
    I reckon you are already aware of this if you’re using it to generate an initramfs, but for those reading along, you can also use it as a docker image
  - joezydeco 2y ago
    I use buildroot to make a commercial product. It builds the bootloader, the kernel, initramfs and rootfs, and then my application(s) all installed in the right place and connected to systemd. The output is a set of images that burn into their respective partitions on eMMC. If you're a masochist, there's also Yocto. It'll build your compiler and crosstools as well as the entire system image.
    - ahepp 2y ago
      Buildroot will build the cross toolchain too unless you specify BR2_TOOLCHAIN_EXTERNAL, right? And likewise you can configure yocto to use an external toolchain. A lot of the byproducts are configurable. You could configure the system to not use an initramfs, use only an initramfs/squashfs, etc. It is a really great tool with great documentation. I agree that Yocto is a bit masochistic by comparison, but vendors like it because its primary purpose is to enable shitty vendor forks.

sirodoht 2y ago

I went through this over a couple of weeks last year. Very fun, I would recommend it to anyone interested to see the complexity of making a linux distro.

jaystraw 2y ago

to anyone confused why others find this fun or worthwhile, i will quote brak's dad: "you just don't get it, das all"

rkagerer 2y ago

How up to date is this? When trying to learn by copy-pasting from the internet, I sometimes get tripped up over older/newer Linuxes using different versions of commands or best practices. (e.g. ipconfig vs. ifconfig, iptables vs. successors)

creatonez 2y ago
It was last updated in September 2024. Distrowatch's package list can give you an idea of what version numbers to expect: https://distrowatch.com/table.php?distribution=lfs https://distrowatch.com/table.php?distribution=lfs Once you get to the BLFS part, you have more choice on whether you're going to innovate or use something that old and in maintenance mode. For example, it has instructions for both Pulseaudio and Pipewire, Xorg and Wayland, SysVinit and systemd. The instructions will rarely be outright incorrect on modern distros unless something is very strange about your environment, since a bulk of the work is done after establishing a more predictable build environment.
avisk 2y ago
It is up to date. I successfully built an LFS system this week by following the guide.

mydriasis 2y ago

If you're curious, you should try it. It's a great way to learn about all of the little bits and pieces that go into a working OS that you can use, and by the time you're done you'll be more comfy thinking about Linux in general. It may take some time depending on how often you come back to it, but it's definitely worth it.

dark-star 2y ago

I have used CLFS (Cross-Linux From Scratch) in the past a couple of times to build a Linux system for Sun UltraSPARC and SGI Octane systems. The things I learned, especially about cross-compiling, were invaluable, but you do have to invest some time understanding everything (even reading patches that you have to apply on top of build tools, for example) to get the most out of it. If you just copy/paste commands, well then there are faster/easier ways to get Linux up and running Edit: I just noted that CLFS has been dead for a couple of years now, which is sad. I would have loved to try an updated CLFS

cloudripper 2y ago

I gave LFS a go earlier this year. I learned a lot through the process - but I definitely went outside the guardrails. I use NixOS as my daily driver and found myself curious of whether I could complete LFS using a "Nix" approach. I was only a basic Nix user at that time and that choice made a difficult process much more difficult. However, the declarative nature of Nix meant that I had clear notes of every step of my process - and if something didn't work, I could backtrack and troubleshoot to find the root of the cause. The end result is here [0]. My understanding of Linux, of bootstrapping, cross-compilation, and Nix has grown tremendously as a result of the time I took on this project - and I still go back and reference the work from time to time. When I get some time to revisit the Nix-based LFS project, there are quite a few things I would like to clean-up, including setting kernel configs and handling post-build permissions. Nix-complexities aside, I highly recommend LFS if you like to understand how things work and don't mind a little suffering along the way. [0]: https://github.com/cloudripper/NixLFS https://github.com/cloudripper/NixLFS

tomberek 2y ago
This is excellent. Have you considered making a presentation or a write-up of the experience?
- cloudripper 2y ago
  Thanks for the input. A couple of folks suggested that recently as well. Once I can clear up some bandwidth, I do intend to follow through on that. It gave me a huge appreciation for Nix as a build system, and I would love to share that if it were helpful to others.
ayakang31415 2y ago
The first thing that pops out on Google search of NixOS is "Declarative builds and deployments." What exactly is NixOS different from other distros, such as Ubuntu?
- hollerith 2y ago
  NixOS separates packages. If the package foo contains a file /usr/bin/foo, NixOS installs it in /nix/store/67c25d7ad7b2b64c67c25d7ad7b2b64c-foo/usr/bin/foo. In order to make this separation work, Nix must sometimes rewrite binaries so that all references in the binary to /usr/bin/foo becomes references to /nix/store/67c25d7ad7b2b64c67c25d7ad7b2b64c-foo/usr/bin/foo. The advantage of this approach is that it gives more control to the distro maintainers and the admins of the computer, taking that control away from the "upstream" maintainers of the software being packaged. For example the software being packaged cannot just call the library bar because bar is not at /usr/lib/bar.so like it is in most Linux distros -- it is at /nix/store/17813e8b97b84e0317813e8b97b84e03-bar/usr/lib/bar.so, but of course the software does not know that unless the person creating the Nix package (the packager) arranges for the software to know it (again sometimes by doing a search-and-replace on binaries). If the upstream maintainer of foo thinks foo should link to version 6 of library bar, but you think it should link to version 5, NixOS makes it easier for you to arrange for foo to link to version 5 than most distros do (even if version 6 of bar is needed by other packages you have installed which you need to use at the same times as your using foo). Note that if this separation of packages imposed by NixOS has any beneficial security properties, it is merely security through obscurity because there is nothing preventing a binary from covertly searching through the directory listing of /nix/store/ for the name of the library it wants to call. Nevertheless it turns out the be useful to seize some control away from upstream in this way even if technically upstream could seize the control back if it were willing to complicate the software to do so. People, including the creator of Nix and NixOS (Dolstra), will tell you that NixOS's main advantage is "declarativeness" (which in the past Dolstra called "purity") or the fact that the compilation / building process is deterministic. I believe both positions (NixOS's advantage is declarativeness and the advantage is deterministic builds) are wrong. Specifically, I believe that although deterministic builds are useful, the separation of packages I described is much more useful to most users and prospective users of NixOS. Another way to summarize it is that NixOS package maintainers routinely modify the software they are packaging to use less "ambient authority".

smitty1e 2y ago

I have gone all the way through this, #21584. What a tremendous resource.

kopirgan 2y ago

This has been around for ages. I only read the contents briefly to know what the minimum components of Linux is, that too years ago. When you install any distro (even Debian) so much gets installed which newbies wouldn't even know exists or use.

throwaway2016a 2y ago

This brings back memories. I tried to do his (by using this exact site!) in college. Which, for reference, was 20ish years ago. I wasn't able to get very far, mostly because I kept running into obscure compiler and linker errors and 18 year old me wasn't clear how to fix them. Would be fun to see how much is changed since then because it does appear to be at least partially maintained.

fefe23 2y ago

LFS has been a tremendous resource over the years for me. I'm happy it exists and that people put in the time to keep it going. I often wonder why the existence of long "you also need to do this completely unintuitive thing first" documentation on the open internet isn't shaming more projects into reducing barriers to build their software.

dvno42 2y ago

Between LFS and Stage 1 and 2 Gentoo installs back in the early 2000s during High School, this gave me a big leg up in my journey of learning about computers and Linux. I can't thank those project maintainers enough for helping me get my footing at such a young age and peaking my interest in computing. I ended up printing out the LFS book in segments on one of the printers in High School and brought it home in pieces to use on the home computer.

tehf0x 2y ago
Amen to that! 20 years later this was my gateway drug into being addicted to computers and gave me my full stack understanding I still use every day at work. <3 Gentoo and the friendly geek at the coffee place I worked at when I was 14 who gave me my first hit and held my hand through what is effectively mental torture for most people
oorza 2y ago
> I ended up printing out the LFS book in segments on one of the printers in High School and brought it home in pieces to use on the home computer. I am actually flabbergasted there's another human being on Earth that has this exact same story as me. I used one of those giant 3 ring legal binders for my printed copy, lmao.
hi-v-rocknroll 2y ago
Yep. Recompile all the things with different USE flags.
- _joel 2y ago
  Recompiling the entire openoffice so it loads 1 second quicker.. just using a hacked up distcc cluster and a day's worth of CPU time. Bargain!
  - account42 2y ago
    It can actually be a bargain though. CPU time I can buy with money, life time is a lot more difficult to extend (not to mention the frustration a second of startup time can cause).

f0e4c2f7 2y ago

Building using LFS has been on my list of "I really should probably do that just to learn" for about 20 years now. I'll get around to it! This year I'm finally learning Lisp (and really enjoying it).

theanonymousone 2y ago
Isn't an LLM making such things more approachable?
- f0e4c2f7 2y ago
  For sure. It's been amazing honestly and I feel like it has really accelerated my learning. Enough that I'm trying to get really serious about making the most out of this new leverage seemingly out of nowhere. I use it a lot for mapping out of the initial concepts but I find one of the best use cases is after understanding the basics, explaining where I need to learn more and asking for a book recommendation. The quality of my reading list has gone up 10x this way and I find myself working through multiple books a week. Great for code too obviously, though still feels like early days there to me.
mekster 2y ago
If you'd done that 20 years ago, we'd been on the same page :) Though back then, I learnt well from Gentoo by setting the system up from stage 1, which was a good knowledge to keep when trying to fix things such as handling partitions in single user mode or modprobe'ing to detect hardware and such. If you can't do anything beyond relying on GUI, you could get stuck real fast.

rbattula 2y ago

tried lfs back in high school really fun experience and teaches you a lot

andrewmcwatters 2y ago

I remember trying to automate non-distribution builds of a functional GNU/Linux operating system and trying to not read Linux from Scratch but just official kernel.org documentation.[1] Unfortunately, the state of Linux documentation is so poor, you can't do it. You need to reference a number of third-party articles that kernel.org itself sometimes links you to. I believe kernel.org might also mention Linux from Scratch, but LFS does a very poor job of explaining why you need particular dependencies straight out of initramfs. You need a functional shell, and you need to get your installation on to an actual drive. Neither of those things are explained in sufficient detail with the current documentation available today. LFS at best says "throw in these ingredients," and leaves you with no other information. You can probably read man pages and piece this stuff together, but it requires that you, at the very least, hardcode where to install files with fsck/mount/cp, I think. Edit: LFS also does a really poor job of explaining why it chooses its "Basic System Software," much of which isn't actually required for a common GNU/Linux system. [1]: https://github.com/andrewmcwatters/linux-workflow https://github.com/andrewmcwatters/linux-workflow

machinestops 2y ago
> why you need particular dependencies straight out of initramfs. It came as a shock to me to learn that initramfs is mostly optional. It can be skipped, and you can boot straight into userland.
- andrewmcwatters 2y ago
  Yes, but if I understand correctly, you’d need to configure for this specifically. I think standard configurations and installations use them.
  - machinestops 2y ago
    Yes, it requires specific configuration. The reason most distributions of Linux use them is it allows you to dynamically discover and load kernel modules at boot time, and provide a fallback for if mounting the disk fails.

ngneer 2y ago

This brings back memories. I built LFS on a Pentium based machine a few times and it was quite fun. I took a similar approach to building a custom bare metal system for a client that needed extra security. I sometimes wish that there was a well maintained bare bones distribution inspired by the LFS philosophy, that might help with managing complexity on the cloud. Scanning the thread, NixOS came up. I used to love Slack, too. Any other recommendations? Something with minimal dependencies yet capable of being a LAMP-like server?

throwaway2037 2y ago
Alpine isn't good enough as a base system?
- _joel 2y ago
  muslc isn't as fast as glibc (last time I checked), but I guess it's fine if you're ok with that.
greggyb 2y ago
Alpine Linux is a very minimal base built around busybox and musl libc. Void is another minimalist distro offering a choice of glibc or musl, and using runit as an init. If you want more choices of init and a bleeding edge rolling release, Artix is a derivative of Arch with multiple init options in the core distribution. Arch and Gentoo are the classics of do-it-yourself minimalism.

krylon 2y ago

Back in the day, I made an attempt at LFS but gave up halfway through. In my defense, I had done a gentoo stage 1 emerge earlier that year. ;-) I never intended to end up with a usable system, I was just in it for the learning experience. And I did end up learning a lot.

deleted 2y ago

[deleted]

ocean_moist 2y ago

I did LFS after I started dailying gentoo with a custom kernel (~2017), it was actually fairly easy, if not time consuming. People on IRC are extremely helpful. These days I write code more than infra, but I still think it was valuable. That combined with my general interest in Linux probably saved me thousands in cloud costs. Linux is a dying art among young software engineers, but alive among young nerds. Even among infra/devops people, maybe if they learned more about linux they wouldn't reach for k8s to solve the simplest problems.

Razengan 2y ago

Anyone else annoyed by “online books” sites like these where you have to play a convoluted game find-the-breadcrumbs from the website landing page to the beginning of the actual book content..

greggyb 2y ago
Two clicks from the landing page. 1. Choose the book you want to read (they call them sub-projects) 2. "Download" or "Read online" Of course, they could reverse the nav order and provide first a "read online"/"download" and then let you pick which of the books you want to read, yielding ... two clicks to get to the one you want to read.

hi-v-rocknroll 2y ago

I wrote an (unreleased) Bash-based framework for Docker container- and exo-Docker VM-based LFS builds. Dir listing: https://pastebin.com/JrPkftgr https://pastebin.com/JrPkftgr Dockerfile (it's still missing throwing away intermediate layers b/c it's optimized for caching and dev speed): FROM quay.io/centos/centos:stream9 COPY files/bashrc /root/.bashrc COPY files/bash_profile /root/.bash_profile COPY --chmod=755 files/sysdiff /usr/bin/sysdiff WORKDIR /mnt/lfs COPY files/functions.inc files/ COPY --chmod=755 files/run files/ COPY scripts/0000-install-deps scripts/ RUN files/run install 0000-install-deps # Docker host runs a memcache instance to cache build artifacts in RAM COPY scripts/0010-install-sccache scripts/ COPY files/sccache-wrapper.c files/ RUN files/run install 0010-install-sccache COPY scripts/1000-setup scripts/ COPY files/hosts files/passwd files/group files/shells files/inputrc files/fstab.in files/locale.conf.in files/ RUN files/run install 1000-setup ...

jakogut 2y ago

Another project to check out if you enjoy LFS is buildroot. Buildroot uses Kconfig (and associated tools like menuconfig, config, etc.) same as the kernel, to generate a configuration for building an embedded Linux system. This configuration can be committed and patched in the repo for your project. Adding an application or library automatically pulls in dependencies. Configs can be merged with fragments that add specific features. Buildroot is capable of building its own cross compilation tool chain for many architectures, and enables developers to choose their own libc, and other toolchain configurations. It can also be configured to pull a prebuilt toolchain tarball to save time. Packages are written in GNU Make, and macros are available for pulling code from GitHub and building packages using autoconf, cmake, meson, golang, Rust, python, and many more. It works fantastically for anything from building an embedded Linux distro for something like digital signage, to a self-contained network bootable application for large scale automated provisioning of devices, to building minimal tarballs of filesystems to run as containers.

itomato 2y ago
It’s cool but adding more than a package or two in the buildroot way is where I would get into the weeds. I’m sometimes surprised a fork with better approach to package management doesn’t exist yet.

Scuds 2y ago

I wonder what an equivalent "BSD from scratch" is like? Linux was assembled from a collection of parts while BSD is (reputably) a lot more 'designed from the ground up' Even a modern system like Fuchsia - what's that like to build from the ground up? Or is it "You fool! Building your own kitbashed Gundam of an OS is the point."

joveian 2y ago
I'm not sure about the others but the full NetBSD base system can be cross compiled from Linux (and maybe some other unixy systems) using the included build script (I'm not sure what the current state of cross compiling pkgsrc is, likely not as good). You could add or remove things from the base system to learn more about how the build system works or create different install sets or something along those lines, although just looking at what the build system does might be as helpful.

cjk 2y ago

This, and its no-longer-maintained cousin, Cross Linux from Scratch[1], were instrumental in learning how to bootstrap a custom Linux distro for embedded devices (especially the CLFS Embedded variant[2]) at a previous job. [1]: https://clfs.org/ https://clfs.org/ [2]: https://clfs.org/view/clfs-embedded/arm/ https://clfs.org/view/clfs-embedded/arm/

keitmo 2y ago

I went through the LFS process back in 2003 or 2004. I didn't have a Linux system handy, so I built it with Fedora 3 running under VMWare on a Windows XP laptop. It was not speedy. I tried to automate the process as much as possible. There's a separate Automated Linux From Scratch project, but I choose to roll my own. This was my first experience with Bash scripts -- I developed skills (and bad habits) that I continue to use this day.

tapanjk 2y ago

I always thought Slackware [1] was the basic Linux distro which people could use to 'understand' Linux. How does Slackware compare to LFS in terms of understanding the Linux fundamentals? [1]: http://www.slackware.com/ http://www.slackware.com/

SuperNinKenDo 2y ago
Slackware is more like a BSD of Linux, to stretch an analogy. A very solid base system, all packaged up as a complete operating system. LFS is utterly different. There is no installer, there is no base system, there is nothing even approaching the power of Slackware's package manager, as barebones as it seems today. This is compile everything from source in a separate environment and put everything in the right place essentially manually*. Of course, `make`-and-friends will usually do the actual lifting, but you're in charge of giving direction, there isn't a distro community patching packages and build-scripts to adhere to a consistent FSH or the like.

janniks 2y ago

Is this still the best resource? Looking for something first principles, simple, dumbed down, stripping away the unnecessary.

trevjud000 2y ago

what's lfs and nix

garfieldnate 2y ago

I would love to see this updated with information on building a realtime Linux (which is now officially supported). I have in mind some hand-held devices that would be a lot more interesting to build with realtime support.

nineteen999 2y ago

My last LFS build was a couple of years ago and took 3-4 weeks, I decided to do a SVR4-themed distro for fun. This meant using the heirloom packaging tools (pkginfo/pkgadd/pkgrm etc) to package everything and I made the default desktop CDE with antialiased fonts. Loads of fun and nostalgia.