6 ms·

Ask HN: How to get into OS/systems programming in 2023?

40 points by newprint 3y ago

I graduated from college about decade+ ago, took few OS & systems programming courses, but didn't find them interesting back then. However, now, more than a decade later, I'm finding myself bored at my cookie cutter C#/Asp.net/SQL(+ cloud) job. What would be your suggestions getting into OS/System programming in 2023 ? For starters, I'm thinking about learning assembly, which I never learned.

markus_zhang 3y ago
I'm following the OSTEP website and the book: https://pages.cs.wisc.edu/~remzi/OSTEP/ https://pages.cs.wisc.edu/~remzi/OSTEP/ In the first step they ask you to build a simple shell with redirection and parallel execution which seems to be accessible to me (I don't have CS background). MIT also has their own OS courses exposed to audiences: https://pdos.csail.mit.edu/6.S081/2020/ https://pdos.csail.mit.edu/6.S081/2020/
gladiatr72 3y ago
I haven't been an active BSD operator for several years (the old hardware died), but the lack of corporate influence in the DragonFly/Free/Net/OpenBSD might be a space to check out. They've got non-trivial standards for what gets excepted into their source trees, but a significant number of contributors to those projects are professional or (at the very least) lifelong systems coders. OpenBSD would be the best choice if you're into what's current in the hardened systems arena. NetBSD has historically been the OS that runs on nearly every general computing platform that can still boot (I thought I had read something about Net sunsetting support for the vax architecture, but I just checked--still listed on their tier 2 list)
- gtirloni 3y ago
  > They've got non-trivial standards for what gets excepted into their source trees The Wireguard drama in FreeBSD debunks this myth.
  - gladiatr72 3y ago
    Eh. Okay. Like I said, I've been out of direct contact with the *BSD world for a while, so take it less as mythologizing as not being up-to-date :)
- newprint 3y ago
  OP here. A while ago, I saw a book and set of videos (if my memory serves me right) going over the design and source code for one of the BSD flavors. I was thinking about finding that book.
  - toast0 3y ago
    Not sure about videos, but there's a series of books on FreeBSD called The Design and Implementation of the FreeBSD Operating System; the 1996 edition with a mildy different name is freely available [1], but I recommend picking up a newer version if it resonates with you; the 2014 version covers up to FreeBSD 11, the 2004 covers up to FreeBSD 5; but there were a lot of important changes in that ten year period. It will be a little dated, but the broad strokes are still the same. OSTEP, linked elsewhere in the thread is good, too. [1] https://docs.freebsd.org/en/books/design-44bsd/ https://docs.freebsd.org/en/books/design-44bsd/
    - gtirloni 3y ago
      For videos, it might be this: https://www.mckusick.com/courses/advdescrip.html https://www.mckusick.com/courses/advdescrip.html
      - newprint 3y ago
        Yep this is what I saw years ago. Thank you for the link. Damn, those video are pricey and they are for outdated code as well.
deleted 3y ago
[deleted]
eimrine 3y ago
Nobody (except of hardware guys and compiler writers) needs to learn Asm, but definitely you are got to get a skill of reading it. There are a lot of asms (one per every computing architerture plus several possible syntaxes). Of course you need to be an expert in C, in my understanding this is way more important than Asm. One of the easiest way of starting system programming is to build some electronic toy based on microcontroller and write some code for it to do something useful. Bonus points will be earned if you will manage to pack some complicated behaviour so heavily that your controller will be having zero free memory.
- P_I_Staker 3y ago
  Depending what you mean by "hardware guys" this isn't completely accurate. Though, you may not need to "learn" it in the sense that you do other programming languages, or like you previously did. It's conceivable that they might have to write ASM at some point. However it will probably be very minimal and geared to a specific low level (hardware) goal. There's still some value to "learning" ASM, or at least having worked with it if you want to write low level code, though it might be minimal. I wouldn't understand C and low level software without having worked with ASM at some point, even though I don't use it. I probably wouldn't recommend it a good use of time.
  - baremetal 3y ago
    It is good to know how control flow works under the hood. Like C loops and ifs are made of jumps and compares in assembly.
    - eimrine 3y ago
      > jumps <irony>goto is considered harmful</irony>
    - P_I_Staker 3y ago
      If you do it correctly it can help demystify function call/return and pointers... although, as I understand it pointers are NOT just addresses in memory.
hnthrowaway0328 3y ago
One thing I found pretty useful is to port very early Linux command line programs to modern platforms. I just "ported" (nerfed the number of switches) the 1991 version of `ls.c` to modern Ubuntu and learned a lot of about `stat()`, `lstat()`, the `struct state`, file types in Linux and some programming tricks. It only has 1.2K LoC so it's easy to sort out the logic and write your own from scratch if needed.
_448 3y ago
* Start with C programming language. K&R book is still the best, even though outdated. * Take a course like Nand2Tetris (freely available online) * Read a good OS book, e.g. The Design of the UNIX Operating System by Maurice Bach or any Andrew Tanenbaum book on the subject. * Search for "unix systems programming" or "linux systems programming" and you will find some fantastic free books online. * Read the free Linux Device Driver book. It is outdated but will still provide enough information to help you self navigate Linux driver code. * Visit https://wiki.osdev.org/Expanded_Main_Page https://wiki.osdev.org/Expanded_Main_Page * And as they say, "Read the source, Luke"; read the Linux source code. First you will have to identify which subsystem you are interested in, and then start browsing that.
- anditherobot 3y ago
  If it's outdated is it still the best? Did you mean: even though it's old ?
  - linguae 3y ago
    “The C Programming Language, Second Edition” is an excellent book for learning the basics of ANSI C. However, there have been a few revisions of the C standard since then. While none of the changes are extreme (it’s not like learning Java 1.2 or Python 2 in 2023, and there are many ANSI C code bases still being maintained today), it’s still good to know of the newer features of the language. Additionally, the book does not cover certain pitfalls that affect the security and reliability of programs written in C, such as security problems caused by faulty uses of gets() and strcpy(). It’s unfortunate Dennis Ritchie is no longer with us; it would be nice to have a third edition of K&R that covers modern C standards and has information about best practices for avoiding situations that can lead to insecure code.
    - _448 3y ago
      There is one free book for modern C https://gustedt.gitlabpages.inria.fr/modern-c https://gustedt.gitlabpages.inria.fr/modern-c Did not check whether it covers security issues.
HeyLaughingBoy 3y ago
As a hobby or as a career switch? I've been programming embedded systems for the majority of my career and if you were to tell me that you just learned assembly, I'd wonder why you were wasting time like that. The last time I did more than glance at assembly language would have been around 1999. If it's for a hobby, grab an arduino clone board and go for it! I'd suggest something ESP32-based though.
- Ironlikebike 3y ago
  My experience has been the complete opposite. When I was in Uni I was required to take an assembly language course and was told it was foundational but I'd never use it in industry. Straight out of Uni in 2001, I became an operating systems programmer bringing up a new platform and pretty much immediately started debugging C++ code almost exclusively in an asm debugger (it's what we had for an OS level debugger). I also started implementing capabilities provided by a hypervisor directly available via asm instruction because there wasn't a C construct for doing so). After that I became a toolchain programmer (working mostly on new system libraries) and would write in C and Assembly language and debug mostly in Assembly language because, when you're working on system libraries and the toolchain for a new platform, you can't always trust the compiler, and the compiler can't always generate optimal code (or any code) based on the new instruction set. Oh, and you can't always trust the debugger either if it doesn't understand the instruction set. While there I would work on 9 new platform bringups and would often need to stub in assembly language routines for capabilities that were in the architecture but not yet available in the toolchain. Other times, new system call routines, locking intrinsics, unwinders, exception handlers, and some optimized routines would require implementation in asm as well. If you're only working on the non-platform part of an operating system, you might not need to learn assembly language, but if you touch platform code you will likely need to know it.
  - markus_zhang 3y ago
    Sounds like fun experience to me. I guess grass is greener everywhere else but what are places that hire for a large number of such positions (and don't require much industry experience)? But I'm in Canada so that could be an issue...
sonjat 3y ago
You can try working on the Linux kernel, perhaps with the kernel janitors project (https://code.google.com/archive/p/kernel-janitors/ https://code.google.com/archive/p/kernel-janitors/ ). I recommend hanging out on the mailing list a little while to get some idea of what to do, as well as reading the wikis on that page.
1letterunixname 3y ago
I would learn C and x86/arm assembly first enough to bootstrap. It's usually good to learn how to write ld linker scripts and how to make something bootable from grub. Then, I would look Rust because it's easier to create correct no_std kernels. EDIT: Link spew turned into a gist https://gist.github.com/5a1b94e8fde45e37c55c5a13d97c9b3f https://gist.github.com/5a1b94e8fde45e37c55c5a13d97c9b3f
mikewarot 3y ago
I'd dig into genode[1], which is a capability based operating system. You'll likely see an upsurge in interest in capability based systems in the next decade. [1] https://genode.org/ https://genode.org/
brapru 3y ago
Also worth mentioning SerenityOS https://github.com/SerenityOS/serenity https://github.com/SerenityOS/serenity. It’s a very friendly environment and easy to get involved.
dev_0 3y ago
[dead]
phendrenad2 3y ago
Assembly isn't very fun, but maybe reading an old book on assembly for an old 8- or 16-bit cpu will satisfy your curiosity.