Yes, this is what some people seem to be missing here. Much of the Linux code implements drivers or subsystems that you might not be interested in to begin with

Yes, this is what some people seem to be missing here. Much of the Linux code implements drivers or subsystems that you might not be interested in to begin with (such as the network packet filter). Kernel code is highly modular. The code that implements trivial things like context switching is still much more complex than in an '80s Unix, but it is not millions of lines.