4 ms·

> Also, this pinged a few neurons deep in my memory, so I found what it made me remember. Back in 2004, some random named Jeff V. Merkey tried to buy a special

> Also, this pinged a few neurons deep in my memory, so I found what it made me remember. Back in 2004, some random named Jeff V. Merkey tried to buy a special license to the then-current kernel for $50,000.00: > https://dwheeler.com/essays/linux-kernel-cost.html https://dwheeler.com/essays/linux-kernel-cost.html > In response, Molnar did some quick sloccount math and estimated the kernel as being worth $175,974,824 at that time. Absolutely. Just to clarify: Molnar first did the quick sloccount math and estimated $176 million USD. However, Molnar used the estimation values appropriate for an application. But in fact, operating systems kernels are known to be more difficult to develop than typical applications. So I used the same approach but refined it to use parameters appropriate for a kernel. The article https://dwheeler.com/essays/linux-kernel-cost.html https://dwheeler.com/essays/linux-kernel-cost.html is really a response to Molnar's work; he did a rough estimate, I did a slightly-more-refined estimate. Those additional tweaks resulted in a higher redevelopment cost of $612 million (USD). Which gave the $50,000 offer an even bigger contrast. The real point was that, even if it would have been possible accept $50K, it was absurdly low. You could argue about the estimate for a factor of 2, or 10, or even 100, and it still wouldn't change anything. Merkey was free to lowball a proposal, but that doesn't mean it should be accepted :-). There was a 2004 article about this in LWN.net. LWN.net, and many others, don't see how such an offer could have been legally enforcible anyway, since you'd have to get the agreement of all the kernel contributors: https://lwn.net/Articles/106353/ https://lwn.net/Articles/106353/ To me, the kernel offer was more of an opportunity to find a way to measure the size of a kernel in a way that matters to people. Lines of code, or bytes on disk, don't really mean much to most people. Money... does :-).

cperciva 6d ago
in fact, operating systems kernels are known to be more difficult to develop than typical applications. I'm not convinced. The core OS kernel code, sure; but more than half of the Linux kernel is drivers, and those are not just applications but separate applications; a change in an Intel wifi driver is unlikely to interact in any way with the nvme driver.
- dwheeler 6d ago
  > I'm not convinced. The core OS kernel code, sure; but more than half of the Linux kernel is drivers ... I agree that more than half of the Linux kernel is drivers. In fact, in my analysis of Red Hat 7.1 at https://dwheeler.com/sloc/redhat71-v1/redhat71sloc.html https://dwheeler.com/sloc/redhat71-v1/redhat71sloc.html I found that 57% of the Linux kernel was in the drivers subdirectory. Obviously that varies over time. So I completely agree with that part! > and those are not just applications but separate applications; a change in an Intel wifi driver is unlikely to interact in any way with the nvme driver. They aren't really separate applications, though. More importantly, that's not the primary effort (and cost) driver for this effort estimation model. I used the COCOMO model, an effort estimation model that was publicly available and widely used at the time. I then applied its rules to the Linux kernel of that time, as best I could. I discussed every parameter I set, and why, here: https://dwheeler.com/essays/linux-kernel-cost.html https://dwheeler.com/essays/linux-kernel-cost.html Here are some of the reasons why kernel code takes more effort, per the model: * It inherits some of the general challenges of writing embedded software. There's very little in the way of a safety net (this is the safety net), and it's closer to the bare metal. You're writing in C (or later in Rust), not a language that shields you from many challenges. In COCOMO parlence this code is "semidetached". * RELY: Required software reliability: High (1.15). The Linux kernel developers care a lot about reliability, which takes longer. * CPLX: Product complexity: Extra high (1.65). "The kernel must perform multiple resource handling with dynamically changing priorities: multiple processes/tasks running on potentially multiple processors, with multiple kinds of memory, accessing peripherals which also have various dynamic priorities. The kernel must deal with device timing-dependent coding, and with highly coupled dynamic data structures (some of whose structure is imposed by hardware). In addition, it implements routines for interrupt servicing and masking, as well as multi-processor threading and load balancing. And yes, that includes drivers; drivers must handle threading and other challenges that "normal" code doesn't, because the driver code is where those complexities are handled." * TIME: Execution time constraint: High (1.11). "Although it doesn’t need to stay at less than 70% resource use, performance is an important design criteria, and much effort has been spent on measuring and improving performance." * VIRT: Virtual machine volatility: High (1.15). "The most common processor (x86) doesn’t change that quickly, though new releases by Intel and AMD do need to be taken into account [but] the other components of underlying machines (such as motherboards, peripheral and bus interfaces, etc.) change on a weekly basis. Often the documentation is unavailable, and when available, it’s sometimes wrong (which from a developer’s point of view looks like a volatile interface, since it keeps changing). The Linux kernel developers spend a vast amount of time identifying hardware limitations/problems and working around them. What’s worse, there’s a variety of different hardware, and new ones keep arriving... the interface of the underlying machine is actually quite volatile." Not every factor makes things worse. The people analyzing it (ACAP) and developing the code (PCAP) are unusually capable, with high experience in the programming language (LEXP) and modern development practices (MODP). But these only partly compensate for the fundamental challenge of writing highly performant kernel code. Anyway, that's my rationale. I did this back in 2004, so I was necessarily using data and models available at the time. Most importantly, I think its key point was absolutely correct: it would have cost far more than $50,000 USD to re-develop the Linux kernel of that time.

cperciva 6d ago

in fact, operating systems kernels are known to be more difficult to develop than typical applications. I'm not convinced. The core OS kernel code, sure; but more than half of the Linux kernel is drivers, and those are not just applications but separate applications; a change in an Intel wifi driver is unlikely to interact in any way with the nvme driver.

dwheeler 6d ago
> I'm not convinced. The core OS kernel code, sure; but more than half of the Linux kernel is drivers ... I agree that more than half of the Linux kernel is drivers. In fact, in my analysis of Red Hat 7.1 at https://dwheeler.com/sloc/redhat71-v1/redhat71sloc.html https://dwheeler.com/sloc/redhat71-v1/redhat71sloc.html I found that 57% of the Linux kernel was in the drivers subdirectory. Obviously that varies over time. So I completely agree with that part! > and those are not just applications but separate applications; a change in an Intel wifi driver is unlikely to interact in any way with the nvme driver. They aren't really separate applications, though. More importantly, that's not the primary effort (and cost) driver for this effort estimation model. I used the COCOMO model, an effort estimation model that was publicly available and widely used at the time. I then applied its rules to the Linux kernel of that time, as best I could. I discussed every parameter I set, and why, here: https://dwheeler.com/essays/linux-kernel-cost.html https://dwheeler.com/essays/linux-kernel-cost.html Here are some of the reasons why kernel code takes more effort, per the model: * It inherits some of the general challenges of writing embedded software. There's very little in the way of a safety net (this is the safety net), and it's closer to the bare metal. You're writing in C (or later in Rust), not a language that shields you from many challenges. In COCOMO parlence this code is "semidetached". * RELY: Required software reliability: High (1.15). The Linux kernel developers care a lot about reliability, which takes longer. * CPLX: Product complexity: Extra high (1.65). "The kernel must perform multiple resource handling with dynamically changing priorities: multiple processes/tasks running on potentially multiple processors, with multiple kinds of memory, accessing peripherals which also have various dynamic priorities. The kernel must deal with device timing-dependent coding, and with highly coupled dynamic data structures (some of whose structure is imposed by hardware). In addition, it implements routines for interrupt servicing and masking, as well as multi-processor threading and load balancing. And yes, that includes drivers; drivers must handle threading and other challenges that "normal" code doesn't, because the driver code is where those complexities are handled." * TIME: Execution time constraint: High (1.11). "Although it doesn’t need to stay at less than 70% resource use, performance is an important design criteria, and much effort has been spent on measuring and improving performance." * VIRT: Virtual machine volatility: High (1.15). "The most common processor (x86) doesn’t change that quickly, though new releases by Intel and AMD do need to be taken into account [but] the other components of underlying machines (such as motherboards, peripheral and bus interfaces, etc.) change on a weekly basis. Often the documentation is unavailable, and when available, it’s sometimes wrong (which from a developer’s point of view looks like a volatile interface, since it keeps changing). The Linux kernel developers spend a vast amount of time identifying hardware limitations/problems and working around them. What’s worse, there’s a variety of different hardware, and new ones keep arriving... the interface of the underlying machine is actually quite volatile." Not every factor makes things worse. The people analyzing it (ACAP) and developing the code (PCAP) are unusually capable, with high experience in the programming language (LEXP) and modern development practices (MODP). But these only partly compensate for the fundamental challenge of writing highly performant kernel code. Anyway, that's my rationale. I did this back in 2004, so I was necessarily using data and models available at the time. Most importantly, I think its key point was absolutely correct: it would have cost far more than $50,000 USD to re-develop the Linux kernel of that time.