4 ms·

Can someone explain how such a HUGE codebase is maintained and new people get acquainted with it? I understand the distributed nature of development, but still,

by karterk 15y ago

Can someone explain how such a HUGE codebase is maintained and new people get acquainted with it? I understand the distributed nature of development, but still, this seems like LOTS of code especially for someone new and who wants to get started in terms of contributing.

exDM69 15y ago
Seven easy steps to get acquainted with the Linux kernel: Step 1: Build gnu binutils, gcc and gdb from source for the correct target architecture Step 2: Setup qemu, some kind of virtualization or get a gadget that can run linux so that you can run and debug the kernel without repeated rebooting. (in the early days, Linus actually used to reboot to test the kernel, booting from floppy disks. I wouldn't have the patience) Step 3: Configure and build the linux kernel from git master branch Step 4: Make an initrd root filesystem, e.g. uClibC + busybox. Step 5: If applicable, install a bootloader. Boot to your new kernel. Step 6: Hack with the source and/or compile time or runtime kernel configuration Step 7: Start over from the beginning but do something differently. Steps 1-5 takes about one saturday afternoon. The faster your CPU, the less there will be waiting. You have to start somewhere, so pick an area of interest and work your way around from there. Don't start in the beginning of the boot code, that's arcane magic that's not very useful for learning. There are several books written about the Linux kernel, so you might want to pick one of those. Of course, printed material about the kernel gets old fast but the basic foundations have been pretty solid.
- 4ad 15y ago
  > in the early days, Linus actually used to reboot to test the kernel, booting from floppy disks. I wouldn't have the patience. Still required all the time if you develop drivers for physical hardware. Still required at some point if you develop other kernel mode components because you can get bugs reproducible only on physical hardware because of different timings, cache coherency bugs (rare, but more often than you think) or broken ACPI routing tables (Every motherboard in existence has a broken ACPI table and the kernel has to deal with it).
  - tuukkah 15y ago
    You're not required to boot from floppy disks these days :-) You can even test on another machine without installing the new kernel by using network boot.
    - 4ad 15y ago
      Of course, I was just saying the reboot - test cycle on a physical machine is still required, details are insignificant :-).
  - exDM69 15y ago
    > Still required all the time if you develop drivers for physical hardware. Thankfully, this is not quite true. Most Linux drivers are implemented as kernel modules, which can be loaded and unloaded at runtime. No rebooting necessary. The guys who develop drivers for hardware that cannot be implemented as a kernel module (like some motherboard chipsets) probably have another machine for debugging or have some kind of dev boards to make their workflow more efficient.
    - 4ad 15y ago
      Thanks for telling me how to do my job, I'm a kernel developer :-). A bug in kernel code means you panic the machine, so you have to do a reboot. Whether you load the modules at runtime, on demand, or the driver is statically compiled into the kernel has nothing to do with it. The decision of when to do linking does not influence development a bit. Also, having multiple machines is required, not optional, because you use the dev machine to connect with a kernel debugger through a serial/firewire link to the test machine.
obtu 15y ago
The development model has various people being responsible for integrating patches in various subsystems, with integration done in a tree-like fashion. A lot of people are contributing reviews. To get acquainted, there are some tutorials about writing minimal modules, and reference books like Linux Kernel Development, third edition, by Robert Love. If some area has changed since the book was out, LWN (Linux Weekly News) has an article about it, often quite detailed.
sp332 15y ago
Most of the code is system-specific, either hardware or filesystem drivers. And believe me, that stuff is heavily compartmentalized or the kernel would have collapsed under its own weight long ago.
jlarocco 15y ago
I don't know about the kernel specifically, but most large code bases are modular. There's probably a small set of core kernel functionality that provide services for everybody, and then specialized interfaces for each of the driver types. I would guess that few people are familiar with the entire structure. Most probably know the core interfaces and the design of the subsystem where they mostly work, like wireless drivers.

by karterk 15y ago

exDM69 15y ago

Seven easy steps to get acquainted with the Linux kernel: Step 1: Build gnu binutils, gcc and gdb from source for the correct target architecture Step 2: Setup qemu, some kind of virtualization or get a gadget that can run linux so that you can run and debug the kernel without repeated rebooting. (in the early days, Linus actually used to reboot to test the kernel, booting from floppy disks. I wouldn't have the patience) Step 3: Configure and build the linux kernel from git master branch Step 4: Make an initrd root filesystem, e.g. uClibC + busybox. Step 5: If applicable, install a bootloader. Boot to your new kernel. Step 6: Hack with the source and/or compile time or runtime kernel configuration Step 7: Start over from the beginning but do something differently. Steps 1-5 takes about one saturday afternoon. The faster your CPU, the less there will be waiting. You have to start somewhere, so pick an area of interest and work your way around from there. Don't start in the beginning of the boot code, that's arcane magic that's not very useful for learning. There are several books written about the Linux kernel, so you might want to pick one of those. Of course, printed material about the kernel gets old fast but the basic foundations have been pretty solid.

4ad 15y ago
> in the early days, Linus actually used to reboot to test the kernel, booting from floppy disks. I wouldn't have the patience. Still required all the time if you develop drivers for physical hardware. Still required at some point if you develop other kernel mode components because you can get bugs reproducible only on physical hardware because of different timings, cache coherency bugs (rare, but more often than you think) or broken ACPI routing tables (Every motherboard in existence has a broken ACPI table and the kernel has to deal with it).
- tuukkah 15y ago
  You're not required to boot from floppy disks these days :-) You can even test on another machine without installing the new kernel by using network boot.
  - 4ad 15y ago
    Of course, I was just saying the reboot - test cycle on a physical machine is still required, details are insignificant :-).
- exDM69 15y ago
  > Still required all the time if you develop drivers for physical hardware. Thankfully, this is not quite true. Most Linux drivers are implemented as kernel modules, which can be loaded and unloaded at runtime. No rebooting necessary. The guys who develop drivers for hardware that cannot be implemented as a kernel module (like some motherboard chipsets) probably have another machine for debugging or have some kind of dev boards to make their workflow more efficient.
  - 4ad 15y ago
    Thanks for telling me how to do my job, I'm a kernel developer :-). A bug in kernel code means you panic the machine, so you have to do a reboot. Whether you load the modules at runtime, on demand, or the driver is statically compiled into the kernel has nothing to do with it. The decision of when to do linking does not influence development a bit. Also, having multiple machines is required, not optional, because you use the dev machine to connect with a kernel debugger through a serial/firewire link to the test machine.

obtu 15y ago

The development model has various people being responsible for integrating patches in various subsystems, with integration done in a tree-like fashion. A lot of people are contributing reviews. To get acquainted, there are some tutorials about writing minimal modules, and reference books like Linux Kernel Development, third edition, by Robert Love. If some area has changed since the book was out, LWN (Linux Weekly News) has an article about it, often quite detailed.

sp332 15y ago

Most of the code is system-specific, either hardware or filesystem drivers. And believe me, that stuff is heavily compartmentalized or the kernel would have collapsed under its own weight long ago.

jlarocco 15y ago

I don't know about the kernel specifically, but most large code bases are modular. There's probably a small set of core kernel functionality that provide services for everybody, and then specialized interfaces for each of the driver types. I would guess that few people are familiar with the entire structure. Most probably know the core interfaces and the design of the subsystem where they mostly work, like wireless drivers.