9 ms·

Ask HN: Recommended resources to learn the Linux kernel and OS theory?

• lignux 7y ago
  This looks like a good start: https://0xax.gitbooks.io/linux-insides/content/index.html https://0xax.gitbooks.io/linux-insides/content/index.html
• rapphil 7y ago
  In the past I did this: http://eudyptula-challenge.org/ http://eudyptula-challenge.org/ Not sure if they are accepting new applicants, but you can try. Also, you can try to find the assignments online.
  • neuland 7y ago
    The assignments are online in various places. Here is a copy I made from someone's Github (not linking GitHub b/c it had answers too): https://pastebin.com/cUXuHcDc https://pastebin.com/cUXuHcDc
• yourbandsucks 7y ago
  The O'Reilly book Understanding The Linux Kernel is fantastic. Starts you off in the deep end with memory management/addressing and fans out from there. Notably excludes networking, as the book is long enough without it.
  • DanAtC 7y ago
    Any suggestions for Linux networking? I have Understanding Linux Network Internals but it’s a bit dated.
    • yourbandsucks 7y ago
      I think it comes down to just understanding IP and TCP in general. Which I don't really (IP) or do sorta (TCP). As far as kernel architecture, the rest of the system can work with the assumption "writes go into a buffer, reads come from a buffer, and the networking subsystem does the right thing in between", plus some stuff about DMA optimizations.
    • slinger 7y ago
      not exactly focused on Linux but a good resource on networking nevertheless: http://beej.us/guide/bgnet/ http://beej.us/guide/bgnet/
• weinzierl 7y ago
  Not very Linux specific but Tanenbaum's "Modern Operating Systems" is an excellent source for OS theory and very readable as well. It's an expensive book, but I found it worth its money.
  • jascii 7y ago
    I agree! I took Andy's class that he wrote the book for and it was one of the most influential classes I have ever taken. It is also one of the few books I have kept over the years and throughout all my travels.
  • Noe2097 7y ago
    Seconded. I would add "Structured Computer Organization", as a preamble book. I read both as if they were novels - they really are interesting and written in such a way that can hook you just as a good story
  • slipwalker 7y ago
    +1 for Andrew Tanenbaum
• lorenzfx 7y ago
  If you are still interested in FreeBSD, "The Design and Implementation of the Freebsd Operating System" is a fantastic book. It's a great guide in how a UNIX system and especially FreeBSD works and written by some of FreeBSD's most respected developers.
  • non-entity 7y ago
    I actually did get a copy of that! Unfortunately, I only have the ebook at the moment, which I find difficult to read
    • ashafer 7y ago
      If you want something more hands on you can try "FreeBSD Device Drivers: A Guide for the Intrepid" by Joseph Kong. It is much easier to read, and my guess is you ran into it while making those BSD patches. He also has a short book on building rootkits for BSD, which is a very fun read and demonstrates how things like system calls work. If you want a guide to reading actual kernel code, here is a great resource explaining where some of the foundational pieces are. With a tool like cscope, reading the FreeBSD kernel is actually pretty easy. http://www.watson.org/~robert/freebsd/reading/ http://www.watson.org/~robert/freebsd/reading/ Hope that helps!
  • mjb 7y ago
    This is a great book, and while FreeBSD and Linux differ in a lot of ways, there are enough similarities to make it worth reading even if you're only interested in Linux.
• bytematic 7y ago
  OSTEP is the best book on Operating systems easily. http://pages.cs.wisc.edu/~remzi/OSTEP/ http://pages.cs.wisc.edu/~remzi/OSTEP/
  • apacheCamel 7y ago
    Not sure if it is the best book overall, but I used it for one of my college classes and I think it is a great book for learning much more about operating systems. And it is available for free online so my college student wallet loved it!
• yodsanklai 7y ago
  Not Linux, but I'd recommend this lab from MIT. https://pdos.csail.mit.edu/6.828/2014/overview.html https://pdos.csail.mit.edu/6.828/2014/overview.html I completed this one, but they may have more recent versions. They give you some initial code that you need to complete, as well as tests to check your work before going to the next step. It covers a lot of material. It takes some patience and it's not for complete beginners but it's a great fun project. Not sure, but I'd say it took me 1 or 2 weeks full time.
• geofft 7y ago
  Yes, there is just so much, and it is not thoroughly documented. "BPF maps" in particular are a) a special feature of a special feature that's Linux-specific, not anything that generalizes across OSes, and b) a feature for userspace anyway, not for kernel internals. But even longstanding kernel-internal features aren't well documented (e.g., the other day I was trying to figure out what struct file's f_version does, and I think there's genuinely no docs for it.) So honestly I think the answer there is to not feel bad about not knowing everything. The basic trick of dealing with the kernel is becoming comfortable working in a large codebase most of which you don't understand, and figuring out how to find what you need. Honestly, git grep is one of the best tools here. Get some practice finding some specific thing and where it's implemented, e.g., find a syscall (git grep SYSCALL.*foo) and trace what it calls. Find the definition of a structure inside include/ and see who uses it. Get comfortable with the kernel's OO-ish system of operations structs, and get some practice tracing both "this function makes a generic call, here's a sample driver that implements it" and "this is an implementation of a generic function, here's the syscall that calls it." Beyond that, reading https://lwn.net https://lwn.net 's articles is invaluable, partly for the clear prose coverage and partly for the breadth of what they talk about. (You don't need to pay unless you care strongly about this week's updates - you'll learn plenty from reading articles a week behind - but support them if you can, they're an important resource.) Again, you're not going to follow exactly why e.g. Google wants a new syscall for "restartable sequences" on your first read, but you'll get a sense of what is involved in adding a syscall, how various concurrency models work, what other kernel features are relevant, etc. What sort of OS concepts are you finding yourself Googling? I will say that actually doing a college OS class is what made things like virtual memory management click in my head. It's an intensive approach but writing code in a much smaller kernel than Linux is a valuable way to understand concepts without being drowned in real-world optimizations and edge cases and portability.
  • non-entity 7y ago
    Perhaps I should've said more. I do often end up googling basic OS stuff, especially related to memory (software interrupts still confuse me though, I understand the concepts, but never the implementation), but even more so have been my ignorance of hardware level concepts. For example trying to patch the code for a network card the other week, I learned what watchdog timer is, I've never heard of that prior.
• hatsubai 7y ago
  One important part about learning how the Linux kernel works is understanding the details of the system's interface the kernel provides. In my opinion, there is no better book out there than the "Linux Programming Interface" by Michael Kerrisk: http://man7.org/tlpi/ http://man7.org/tlpi/ It provides extremely detailed information about everything going on in Linux, as well as example programs and exercises to help you further your knowledge. While it doesn't get deep into kernel theory like Tanenbaum's books tend to do, it will provide you with a greater understanding of how things work, IMO.
  • tbrock 7y ago
    This book is great. Seconded.
  • whowhatwhy 7y ago
    This book is great, so is Advanced Programming in the Unix Environment, which covers much of the same material but details the differences in standards and implementation. I'd also add `Linux Kernel Development` by Robert Love.
  • aidos 7y ago
    £50 on kindle! Looks like a great book, but that’s punchy. (Maybe that’s standard for a text book like this, not something I would normally buy)
    • geofft 7y ago
      It's a huge book (1552 pages), much larger than even your average OS textbook. It's more like a Linux encyclopedia.
  • sruffell 7y ago
    I was going to suggest this book when I see I'm late to the party, so I'll just have to add my voice to the chorus. "Linux Programming Interface" is one of the best technical books I've ever read.
  • koolba 7y ago
    That’s a fantastic book.
• gravypod 7y ago
  These are some good resources on the things that go into creating your own kernel: - http://www.brokenthorn.com/Resources/OSDevIndex.html - http://www.osdever.net/bkerndev/Docs/intro.htm For learning about Linux specifically: - https://0xax.gitbooks.io/linux-insides/content/index.html
• ndesaulniers 7y ago
  Join us in /r/kernel. As a mod I'm always helping folks with questions like this, and our community frequently beats me to the punch.
  • non-entity 7y ago
    Didn't know that existed. Will definitely check it out.
• qntty 7y ago
  If you want to understand Linux systems programming, I've never found anything better than the book The Linux Programming Interface. For Linux internals, Linux Kernel Development by Robert Love is good. For something more removed from current systems, The Design of the UNIX Operating System and Operating Systems: Design and Implementation are good.
• n_t 7y ago
  From my previous comment - https://news.ycombinator.com/item?id=18451260 https://news.ycombinator.com/item?id=18451260
• duckqlz 7y ago
  Beyond the exceptional mentioned texts dedicated to the subject; my favorite resources are: lwn.net github.com/torvalds/linux
• pm215 7y ago
  My somewhat off-beat recommendation is "Lions' Commentary on UNIX 6th Edition with Source Code". The sources to a 1976 version of Unix written in an archaic dialect of C and targeting a long-dead CPU architecture are obviously not of immediate relevance to the modern world, and may well not be to your taste unless you already have some sympathy with the idea of retrocomputing. But if you do: 6th Edition is small enough that you really can read through the whole set of sources and understand pretty much how the whole thing is put together; the basic bones of the design are not so far away from Linux, so it gives you a workable conceptual model of what the shape of the Linux sources are and what the various important parts are; the commentary is really good, explaining the inscrutable but nudging you to figure things out yourself too; and overall it is practice in reading a large volume of somebody else's code, which is something you're going to be doing all the time if you work on Linux or any modern kernel. You probably also want a more theoretical and general book too, of course.
  • jiveturkey 7y ago
    And you can witness the glory of swtch() and the original "You are not expected to understand this" comment. Highly recommended.
  • alfiedotwtf 7y ago
    I've commented below... there's an old book called "Linux Core Kernel Commentary" which is based on the Lion's book. It's a great read. There's a Xinu book which is similar, but without the accompanying source code.
• throwawaypls 7y ago
  Linux Kernel Development By Robert Love and Operating Systems Concepts by Silberschatz are great resources.
• waddlesplash 7y ago
  The OSDev wiki can probably teach you anything you need to know about "OS theory" (and practice, mostly on x86): https://wiki.osdev.org/Expanded_Main_Page https://wiki.osdev.org/Expanded_Main_Page Depending on your style of programming, I'd recommend maybe not working on Linux if you are trying to learn kernel development and OS internals. Have you looked at Haiku (https://www.haiku-os.org/ https://www.haiku-os.org/)? We have a very well organized and commented modular-monolitic kernel, and a pretty active development team with a wide range of experience levels. I'm more than happy to help you (or anyone else) learn OS development! Some example Haiku kernel code: * "load_image_internal()", which is responsible for creating processes: https://github.com/haiku/haiku/blob/master/src/system/kernel/team.cpp#L1679 https://github.com/haiku/haiku/blob/master/src/system/kernel... * a recent change by me to replace a global lock with two local ones: https://github.com/haiku/haiku/commit/37eda488be1c9fee242e8e4bf6ca644dd13441d8 https://github.com/haiku/haiku/commit/37eda488be1c9fee242e8e...
  • non-entity 7y ago
    I have heard of Haiku,but haven't looked at it in a while. However, I have heard great things about it and would certainly be interesting in learning and contributing
    • waddlesplash 7y ago
      I and most of the other developers hang out on Freenode#haiku, and of course on our mailing lists. Come say hello and get involved, indeed. More than happy to teach :)
      • jordz 7y ago
        I really love to see this kind of stuff within the community. I'd really love to take you up on this offer sometime :)
        • waddlesplash 7y ago
          Hey, please do! Onboarding new people and teaching skills is fun, and learning them while working on a project people actually use for day-to-day work is even more fun. I idle in #haiku on Freenode, so ping me there and I'll usually reply soon enough. :)
• apples_oranges 7y ago
  From not too long ago: https://news.ycombinator.com/item?id=19225268 https://news.ycombinator.com/item?id=19225268
• kernyan 7y ago
  https://github.com/s-matyukevich/raspberry-pi-os https://github.com/s-matyukevich/raspberry-pi-os Matyukevich page on GitHub has lessons with accompanying code to develop a Linux OS for raspberry Pi. The author goes into kernel, processor initialization, interrupt handling, scheduler, implementing syscall, and virtual memory. I like his approach for these reasons 1. Minimal workable code, 2. Points you to entry function in linux repo 3. Line commentary of those entry function Note that it's also a somewhat short read as he focuses on the practical implementations instead of the theory. I myself was hoping that the author continue developing the chapters on file system, drivers, and networking but seemed to have been on hiatus.
• gamescodedogs 7y ago
  Years ago in school, I decided to figure out how the Linux kernel works. So I've downloaded the sources and open them with a notepad++. After an hour I've figured out that I have no idea what am I reading in the code, close the notepad++ and never open it back. Buy the sources are still awaiting in some old directory at my home's PC :)
• aeyes 7y ago
  The Linux kernel is unfortunately pretty far from UNIX nowadays but if you want to understand the basic concepts behind UNIX I think Lions' Commentary on UNIX is still the best resource out there: https://www.amazon.com/Lions-Commentary-Unix-John/dp/1573980137 https://www.amazon.com/Lions-Commentary-Unix-John/dp/1573980...
• whydoyoucare 7y ago
  "Operating Systems: Three Easy Pieces" is a more modern book to OS concepts, explained with real source code as opposed to algorithms, and is regularly updated. http://pages.cs.wisc.edu/~remzi/OSTEP/ http://pages.cs.wisc.edu/~remzi/OSTEP/ (Online version above, you can also order a print copy).
  • 0x262d 7y ago
    I took this class (although not from him). it's a good textbook and was pretty much sufficient.
  • snoocat 7y ago
    This is what I learned from, along with some xv6 labs. Just wanted to say that both Remzi and Andrea were exceptional professors and their book is clear and concise.
  • thinkmassive 7y ago
    ostep.org is an easy-to-remember shortcut
• molteanu 7y ago
  Operating Systems: Three Easy Pieces [1] XinuOS [2] MIT's xv6 OS [3] [1] http://pages.cs.wisc.edu/~remzi/OSTEP/ http://pages.cs.wisc.edu/~remzi/OSTEP/ [2] https://xinu.cs.purdue.edu/ https://xinu.cs.purdue.edu/ [3] https://github.com/mit-pdos/xv6-public https://github.com/mit-pdos/xv6-public
  • apengwin 7y ago
    +1 for xv6. The commentary book is really well written, and is a straightforward introduction to a lot of important ideas. https://pdos.csail.mit.edu/6.828/2014/xv6/book-rev8.pdf https://pdos.csail.mit.edu/6.828/2014/xv6/book-rev8.pdf
• jiveturkey 7y ago
  The old standards. Stevens to start. If you don't already know Stevens' thoroughly first, I wouldn't go deeper. The kernel is not separable from the userland-facing API. I'm not convinced you actually need to / want to understand actual internals. But assuming you do, next is Tanenbaum (maybe start with the newer "Modern" book). "Operating Systems: Three Easy Pieces" also seems well regarded, although I haven't read it myself. The FreeBSD book is great, if you're specifically interested in BSD. There's also a video lecture you can buy on McKusick's site. I had the pleasure of an on-site presentation and it is absolutely fantastic.
• ekr 7y ago
  Start with something smaller, such as lk.
• filereaper 7y ago
  Many folks here are recommending reading up on production grade operating systems which might be a very steep learning curve. I'd recommend starting with an academic operating system to nail the fundamentals down, once you have a solid foundation you can then use all the excellent links provided by everyone else and ramp up on production grade kernels. I have linked to multiple course syllabus'es below from excellent institutes, pick whichever feels most comfortable. Harvard with MIPS based OS/161 (I learned this personally): http://www.eecs.harvard.edu/~cs161/syllabus.html http://www.eecs.harvard.edu/~cs161/syllabus.html Berkeley EECS 162 (John Kubiatowicz is amazing and is behind RISC V) https://inst.eecs.berkeley.edu/~cs162/sp19/ https://inst.eecs.berkeley.edu/~cs162/sp19/ MIT OCW: https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-828-operating-system-engineering-fall-2012/lecture-notes-and-readings/ https://ocw.mit.edu/courses/electrical-engineering-and-compu...
• djhworld 7y ago
  I've been making my way through this course https://classroom.udacity.com/courses/ud923 https://classroom.udacity.com/courses/ud923 It's about 9 years old I think so might be too out of date for you, but I've been really enjoying it. I think they have an advanced version of the course as well if you're already familiar with the basics
  • Mxtetris 7y ago
    That appears to be the Intro to Operating Systems course from the Georgia Tech OMSCS program. Udacity also hosts the GT Advanced Operating Systems course: https://www.udacity.com/course/advanced-operating-systems--ud189 https://www.udacity.com/course/advanced-operating-systems--u.... AOS is not a "how to" course, but rather a tour of the challenges operating systems may face (distribution, fault tolerance, scale) and of academic papers attempting to solve them (mostly from the 90s and 00s). As such, the material might not be a good fit for the original post, but this commenter found it fascinating. The reading list is here: https://www.udacity.com/wiki/ud156-readings https://www.udacity.com/wiki/ud156-readings
• Jach 7y ago
  Zhao Jiong released this excellent work in English not too long ago: http://www.oldlinux.org/download/ECLK-5.0-WithCover.pdf http://www.oldlinux.org/download/ECLK-5.0-WithCover.pdf If you know Chinese it was originally done in that language, details are on http://www.oldlinux.org/ http://www.oldlinux.org/ It heavily comments / explains everything in the linux kernel v0.12. Quite some time ago, but I agree with the assessment on its relevance for an "inductive understanding". The growth since then is largely in new subsystems, new drivers, but maybe not so much in core theory... If you're more into the core theory and less about Linux in particular, a fun project-oriented approach to get more familiar with the basics would be to get a little development board for an ARM CPU and try to make a multi-tasking (real-time) OS for it from scratch.
  • gabrielblack 7y ago
    Thanks, I haven't this one, it seems interesting.
• peterkelly 7y ago
  I recommend starting with something smaller. Here's something I put together some years ago - it's about 5,000 lines of code and supports a number of key Unix features. There's a PDF of lecture notes accompanying it. http://adelaideos.sourceforge.net/ http://adelaideos.sourceforge.net/ Once you've gone through this you'll be in a better position to tackle the Linux kernel and books on OS architecture. My personal favorite is "The Design of the UNIX Operating System" by Maurice J. Bach. It's an old book but explains things well, and I believe it was one of the references Linux Torvalds relied on when creating Linux.
• Diederich 7y ago
  This is a somewhat different take, and quite possibly impractical. The main thing I've done over the years to get reasonably 'good' at the Linux kernel (and quite a few other things) has been to read every technical article appearing in https://www.lwn.net/ https://www.lwn.net/, every week, in full, even if I didn't really understand the material. I've been doing that since 1997, and it's paid off quite well. Also: please buy a membership, even though you don't have to. It's well worth supporting.
• mesarvagya 7y ago
  In addition to OSTEP, Linux From Scratch [1] is also a good resource. [1] http://www.linuxfromscratch.org http://www.linuxfromscratch.org
• rubyn00bie 7y ago
  This book has been immensely helpful to me, it's "Writing an OS in Rust" https://os.phil-opp.com/ https://os.phil-opp.com/ and I can't recommend it enough. Also, Redox OS is an OS written in Rust and has a ton of wonderful documentation on it that's been helping me get into systems development: https://www.redox-os.org/docs/ https://www.redox-os.org/docs/
• rpark 7y ago
  A classic: The Design of the UNIX Operating System by Maurice Bach. https://www.amazon.com/Design-UNIX-Operating-System/dp/0132017997 https://www.amazon.com/Design-UNIX-Operating-System/dp/01320... I learned more about OS from this book than any academic material used at the university and gave me a good foundational handle on *NIX's.
• theonewolf 7y ago
  The Minix OS was designed as an academic, teaching OS: https://www.minix3.org/ https://www.minix3.org/ There is a textbook that goes along with it: https://www.pearson.com/us/higher-education/program/Tanenbaum-Operating-Systems-Design-and-Implementation-3rd-Edition/PGM228096.html https://www.pearson.com/us/higher-education/program/Tanenbau... The author famously had interactions with Linus Torvalds when he was starting Linux! And, Minix is inside every modern Intel CPU (https://www.zdnet.com/article/minix-intels-hidden-in-chip-operating-system/ https://www.zdnet.com/article/minix-intels-hidden-in-chip-op...).
  • non-entity 7y ago
    Oh yes, I've been curious about breaking into the Minix in intel chips.
• darkr 7y ago
  If you’re looking for something specific to Linux, that is less about the kernel, and more about how the whole operating system is strung together, I can highly recommend: http://www.linuxfromscratch.org/ http://www.linuxfromscratch.org/
• hd4 7y ago
  https://github.com/0xAX/linux-insides https://github.com/0xAX/linux-insides
• alfiedotwtf 7y ago
  For the Linux kernel, there is an excellent book called "Linux Core Kernel Commentary" (which is based off the SVR4 Lion's Commentary book), which has the source of the kernel (an old version) printed out and then a commentary on all the bits (code and data structures). It was an eye opener for me. As for OS theory, "Operating Systems Vade Macum" and "Operating Systems Design and Implementation" are my favorites.
• no-dr-onboard 7y ago
  In school we used Tannenbaum's MINIX book (Operating Systems Design and Implementation). It was generally a pretty good introduction to "OS Theory", directly references the source code in the back. Useful for a desk book if you're constantly elbows deep in writing syscalls.
• linguae 7y ago
  For learning the basics about how an operating system works, I recommend Remzi and Andrea Arpaci-Dusseau's "Three Easy Pieces", which is freely available at http://pages.cs.wisc.edu/~remzi/OSTEP/ http://pages.cs.wisc.edu/~remzi/OSTEP/. Both Remzi and Andrea Arpaci-Dusseau are great writers and are prolific researchers in the areas of operating systems and file systems, and their work is top-notch. For learning the basics about how the kernels of Unix-like systems are designed, I recommend the following books: - The Design of the UNIX Operating System (1986) by Maurice Bach. You could also compare this book to the source code of Version 7 Unix, which is freely available under a BSD-style license. Many of the basic design principles of Unix-like operating systems have not changed dramatically over the years, and so studying the design and implementation of older, smaller implementations of Unix-like operating systems can aid with figuring out the code base of Linux, which is huge. - UNIX Internals: The New Frontier (1996) by Uresh Vahalia. This discusses some more modern features that weren't part of AT&T Unix. - The Design and Implementation of the FreeBSD Operating System (2014) by Marshall Kirk McKusick et al. I've met "Kirk" in person a few times at USENIX FAST conferences; he's not only a great writer, but he is also very friendly and approachable. This is a well-written text describing FreeBSD 11, including modern features such as ZFS. There are some good Linux kernel books (I'm partial to "Linux Kernel Development" by Robert Love, which helped me out nine years ago when I was writing a device-mapper device), but what I've found during my experience working with the Linux kernel is that the Linux kernel changes much faster than the books describing it get updated. Nevertheless, the best thing to do is to get a solid understanding of how Unix works, then get a basic understanding of the Linux kernel, then start diving into the source code, which is ultimately the definitive documentation of the Linux kernel. When I was working on that device-mapper device, I found that perusing the Linux kernel mailing list was excellent for learning about how relevant portions of the kernel changed over time. Good luck and have fun! I haven't done much kernel-level work lately, but I miss working at that level!
• master_yoda_1 7y ago
  http://inst.eecs.berkeley.edu/~cs162/sp19/ http://inst.eecs.berkeley.edu/~cs162/sp19/
• moron4hire 7y ago
  I've semi-seriously been considering starting my own OS project to circumvent some of the IO latency issues with VR and AR applications. The big metric is motion-to-photon latency, which is a measure (in milliseconds) of how quickly a user's head motion is reflected on the display. Anything greater than ~10ms is considered insufficient, as you've missed a whole frame of rendering for VR headsets that run at 90hz refresh rate. Current XR applications need to wait for the USB or Bluetooth stack to return the head pose before modifying the view, then wait for the graphics stack to reflect that change. There are many ring transitions along the way. This leads to an absolute minimum motion-to-photon latency that can't be broken just because of OS overhead. This eats into your frame budget, as every ms of context-switching out of the IO drivers leaves you fewer ms to render a frame in time to get it back into the kernel for context-switching back to the graphics driver.