5 ms·

I've got a CS degree and of all the things that I studied, including OS design, nothing gives me that feeling of black magic like Kernel hacking and driver buil

I've got a CS degree and of all the things that I studied, including OS design, nothing gives me that feeling of black magic like Kernel hacking and driver building. It's like being told your gonna learn how to draw an owl you start with some ovals and then you have a completed owl. Low level stuff has always been interesting to me, I just have no clue how one would even get started with it, or who would even pay me to work on it.

kupopuffs 5y ago
Well, continue classes and your studying. Have you implemented semaphores?
- ensan 5y ago
  What does implementing semaphores have to with this? When it comes to writing firmwares or an OS, nothing can substitute for hands-on experience with actual hardware.
  - shukantpal 5y ago
    Semaphores are important when adding support for SMT in an OS kernel.
    - user-the-name 5y ago
      Likewise, nails are important when building a house, but you do not learn how to build a house by studying nails.
      - kupopuffs 5y ago
        this is a terrible analogy. I learned implementing sempahores when building a kernel. It's absolutely required to learn, a necessary step to building the "house"
Galanwe 5y ago
The easiest way to get started is to just write a driver for whatever simple device you have. It may be easier if you do some electronics as a hobby since you can create the devices yourself with simple enough protocols. It gives you a good understanding of how the hardware works at a basic level. There are thousands of tutorials out there on how to get started writing simple drivers, as well as the excellent book "Linux device drivers". I don't do _complex_ drivers, but I've been able to write simple ones for my DIY electronic devices (keyboards, sensors, ...) and even a reverse engineered one for my BOSS RC5 guitar pedal.
- Ballas 5y ago
  I have written some audio drivers for my own hardware and I still feel lost every time I start with a driver. Admittedly less so than the first time, but if there were decent up-to-date documentation it would really help a lot. (Perhaps my problem is more specific to ALSA documentation)
adamnemecek 5y ago
You setup a table of functions that the HW invokes when things happen. These functions add some sort of event onto a queue. At some point in the future, the kernel processes these events. Rinse, repeat.
- 10000truths 5y ago
  That, and MMIO. Lots and lots of mapping memory ranges/addresses to peripherals.
ensan 5y ago
IMO the best way to get into this type of low-level tinkering is by writing a simple operating system. https://github.com/isometimes/rpi4-osdev https://github.com/isometimes/rpi4-osdev There are other courses/projects for other boards. The keyword is usually “baremetal”. For Linux drivers specifically there are training material from Bootlin etc. They definitely give you a kickstart—the kernel is so complex that most of the device driver knowledge is organizational and cannot be found on YouTube.
tkinom 5y ago
Start by tracing exist open source Intel/AMD driver in linux: Can be for old laptop (T420, etc with Intel GPU/Driver.) Use FTRACE in kernel. Setup ebpf trace for GPU acitivies. Write some doc/blog/medium pages on the process and show off your works. Understand, document and improve some opensource GPU API related utilities Understand and document interaction between GPU/GUI App and OpenSource driver. In term of jobs: AMD has 289 opening for intern positions: https://jobs.amd.com/go/Internships-&-Co-op-Opportunities/2567200/ https://jobs.amd.com/go/Internships-&-Co-op-Opportunities/25... A lot of them are graphic related.
numlock86 5y ago
> Low level stuff has always been interesting to me, I just have no clue how one would even get started with it, or who would even pay me to work on it. One keyword is embedded programming. You can start out bare metal basically doing everything in a while(1) loop until you get to a point where different tasks (in the literal way) and peripherals get so much that you will end up doing exactly this, but on a much smaller (and comprehensible) scale.
ckastner 5y ago
I also have a CS degree and felt the same black magic during my studies, but then I took some microcontroller classes, and they were eye-opening. In the sense that all of the CS operating system stuff suddenly made so much more sense. For example, take something like (1) the simplest AVR microcontroller (the type used in Arduinos, hence there is a ton of third-party documentation and code floating around) and (2) your typical Hitachi HD44780 LED display. To control the display, one needs to pull certain pins high and low, for specific intervals -- this is all in the HD44780's data sheet. Of course, you don't do this every time you send a character; you abstract this away into functions setchar(x, y, c), clearDisplay(), and so on. Presto, your first driver! Then you might think: let's print stuff that we receive via a serial connection. So how do you react to that data? Then you learn about interrupts and interrupt handling. Then you might think: I could be doing stuff while I wait for I/O. Actually, I'd like to do a lot of stuff, in parallel, but the AVRs are single-core (so to speak). So you do what you've heard other OSes do: take available compute time, divide it into slices, and distribute slices to processes. And thus, you have learned about multi-tasking. This was relatively simple to do on AVRs, and great fun.
- chris37879 5y ago
  Yeah, people don't realize that the operating systems we have today arise from devices about as powerful as some of these microcomputers. Removing the complexities that have been added to the x86 platform itself and being able to see "oh when this pin goes high cause I pressed that momentary button the CPU runs the code at this memory address" things can really start clicking. A great youtube creator I recommend for those types of explanations (he's making his own CPU out of base components) is Ben Eater and specifically his 6502 series, but the series on how USB keyboards work is also quite instructive for people looking at the hardware at that level of detail.
  - seanw444 5y ago
    +1 to Ben Eater. His breadboard computer series, when I first watch it a few years ago, is when computers as a whole finally started to click for me, and not just the high-level programming languages.
bitwize 5y ago
Try running NetBSD. You don't have to marry it, just put it on a spare machine or even in a VM. The thing about NetBSD (and I can vouch for OpenBSD being much the same) is that it's so stinking well documented, if you go in knowing C you can learn how to write a simple device driver using just the man pages as your guide. Section 9 of the manual in particular documents every kernel API and most if not all internal types. It's a great OS to practice developing low-level code with. Linux is a bit harder.

kupopuffs 5y ago

Well, continue classes and your studying. Have you implemented semaphores?

ensan 5y ago
What does implementing semaphores have to with this? When it comes to writing firmwares or an OS, nothing can substitute for hands-on experience with actual hardware.
- shukantpal 5y ago
  Semaphores are important when adding support for SMT in an OS kernel.
  - user-the-name 5y ago
    Likewise, nails are important when building a house, but you do not learn how to build a house by studying nails.
    - kupopuffs 5y ago
      this is a terrible analogy. I learned implementing sempahores when building a kernel. It's absolutely required to learn, a necessary step to building the "house"

Galanwe 5y ago

The easiest way to get started is to just write a driver for whatever simple device you have. It may be easier if you do some electronics as a hobby since you can create the devices yourself with simple enough protocols. It gives you a good understanding of how the hardware works at a basic level. There are thousands of tutorials out there on how to get started writing simple drivers, as well as the excellent book "Linux device drivers". I don't do _complex_ drivers, but I've been able to write simple ones for my DIY electronic devices (keyboards, sensors, ...) and even a reverse engineered one for my BOSS RC5 guitar pedal.

Ballas 5y ago
I have written some audio drivers for my own hardware and I still feel lost every time I start with a driver. Admittedly less so than the first time, but if there were decent up-to-date documentation it would really help a lot. (Perhaps my problem is more specific to ALSA documentation)

adamnemecek 5y ago

You setup a table of functions that the HW invokes when things happen. These functions add some sort of event onto a queue. At some point in the future, the kernel processes these events. Rinse, repeat.

10000truths 5y ago
That, and MMIO. Lots and lots of mapping memory ranges/addresses to peripherals.

ensan 5y ago

IMO the best way to get into this type of low-level tinkering is by writing a simple operating system. https://github.com/isometimes/rpi4-osdev https://github.com/isometimes/rpi4-osdev There are other courses/projects for other boards. The keyword is usually “baremetal”. For Linux drivers specifically there are training material from Bootlin etc. They definitely give you a kickstart—the kernel is so complex that most of the device driver knowledge is organizational and cannot be found on YouTube.

tkinom 5y ago

Start by tracing exist open source Intel/AMD driver in linux: Can be for old laptop (T420, etc with Intel GPU/Driver.) Use FTRACE in kernel. Setup ebpf trace for GPU acitivies. Write some doc/blog/medium pages on the process and show off your works. Understand, document and improve some opensource GPU API related utilities Understand and document interaction between GPU/GUI App and OpenSource driver. In term of jobs: AMD has 289 opening for intern positions: https://jobs.amd.com/go/Internships-&-Co-op-Opportunities/2567200/ https://jobs.amd.com/go/Internships-&-Co-op-Opportunities/25... A lot of them are graphic related.

numlock86 5y ago

> Low level stuff has always been interesting to me, I just have no clue how one would even get started with it, or who would even pay me to work on it. One keyword is embedded programming. You can start out bare metal basically doing everything in a while(1) loop until you get to a point where different tasks (in the literal way) and peripherals get so much that you will end up doing exactly this, but on a much smaller (and comprehensible) scale.

ckastner 5y ago

I also have a CS degree and felt the same black magic during my studies, but then I took some microcontroller classes, and they were eye-opening. In the sense that all of the CS operating system stuff suddenly made so much more sense. For example, take something like (1) the simplest AVR microcontroller (the type used in Arduinos, hence there is a ton of third-party documentation and code floating around) and (2) your typical Hitachi HD44780 LED display. To control the display, one needs to pull certain pins high and low, for specific intervals -- this is all in the HD44780's data sheet. Of course, you don't do this every time you send a character; you abstract this away into functions setchar(x, y, c), clearDisplay(), and so on. Presto, your first driver! Then you might think: let's print stuff that we receive via a serial connection. So how do you react to that data? Then you learn about interrupts and interrupt handling. Then you might think: I could be doing stuff while I wait for I/O. Actually, I'd like to do a lot of stuff, in parallel, but the AVRs are single-core (so to speak). So you do what you've heard other OSes do: take available compute time, divide it into slices, and distribute slices to processes. And thus, you have learned about multi-tasking. This was relatively simple to do on AVRs, and great fun.

chris37879 5y ago
Yeah, people don't realize that the operating systems we have today arise from devices about as powerful as some of these microcomputers. Removing the complexities that have been added to the x86 platform itself and being able to see "oh when this pin goes high cause I pressed that momentary button the CPU runs the code at this memory address" things can really start clicking. A great youtube creator I recommend for those types of explanations (he's making his own CPU out of base components) is Ben Eater and specifically his 6502 series, but the series on how USB keyboards work is also quite instructive for people looking at the hardware at that level of detail.
- seanw444 5y ago
  +1 to Ben Eater. His breadboard computer series, when I first watch it a few years ago, is when computers as a whole finally started to click for me, and not just the high-level programming languages.

bitwize 5y ago

Try running NetBSD. You don't have to marry it, just put it on a spare machine or even in a VM. The thing about NetBSD (and I can vouch for OpenBSD being much the same) is that it's so stinking well documented, if you go in knowing C you can learn how to write a simple device driver using just the man pages as your guide. Section 9 of the manual in particular documents every kernel API and most if not all internal types. It's a great OS to practice developing low-level code with. Linux is a bit harder.