9 ms·
Linus says Linux has become 'bloated and huge'
- staunch 17y agoOut of context link bait drivel.
- WalterGR 17y agoHow so? > "Uh, I'd love to say we have a plan," Torvalds replied to applause and chuckles from the audience. "I mean, sometimes it's a bit sad that we are definitely not the streamlined, small, hyper-efficient kernel that I envisioned 15 years ago...The kernel is huge and bloated, and our icache footprint is scary.
- paddy_m 17y agoDoes anyone know what icache is? instruction cache?
- joechung 17y agoinode cache - http://publib.boulder.ibm.com/infocenter/systems/index.jsp?topic=/com.ibm.aix.kdb/doc/kdb/icache.htm http://publib.boulder.ibm.com/infocenter/systems/index.jsp?t...
- kmavm 17y agoYes: instruction cache. Big programs are slow programs.
- swolchok 17y agoThat's too simplistic. Programs with hot code sections that don't fit in icache are slow programs. If your program is huge but you spend all your time in a small fraction of code (typical) that fits in icache (no idea if that's typical), it's fine from a cache performance perspective.
- mahmud 17y agoI think joechung is more right here, icache in this instance means inode cache; the CPU D-Cache and I-Cache (not to be confused the file-system related dcache and icache for dentry cache and inode cache respectively) are of fixed size and you couldn't have "bloat" there. The FS inode cache is more likely to be bloated given the larger upper limit for disk storage compared to CPU cache.
- gjm11 17y agoNo way, for three reasons. (1) Saying "icache" and meaning anything other than "instruction cache" would be an obvious recipe for confusion, and Linus is not stupid. (2) It makes excellent sense with the obvious "instruction cache" meaning: "our icache footprint" means "the number of instruction cache lines occupied by kernel code". (3) It makes no sense with the "inode cache" meaning: how could a growth in the amount of code in the kernel possibly have anything much to do with the size or the occupancy of the inode cache?
- mahmud 17y agoAnd you would be absolutely right. Thanks for the good analysis :-)
- stcredzero 17y ago...Windows usually gets the bad wrap for bloat... At the very least, badly edited.
- fnid 17y agoI think it just shows that open source isn't immune to the same kinds of problems that plague commercial software. As the market for a product expands, so too do the features people require for the software to work within an ecosystem of other software. More hardware to support requires more code, bigger systems.
- ramy_d 17y agoindeed, it will be interesting to see how this issue gets tackled, if at all.
- loup-vaillant 17y agoOff topic: You wrote "commercial", while the correct term was "proprietary". You're not the only one. If I recall correctly, "Open source" was designed specifically to avoid this mistake. Apparently, that failed.
- skwiddor 17y agoMy operating system is open source but I bought it. What kind of software is it?
- foole 17y agoscamware? ;)
- TomOfTTB 17y agoI think this shows the amount of bloat in an Open Source package increases with how widely used the project becomes. As an article posted a couple days ago on HN pointed out about 75% of Linux development is done by corporations these days. Those corporations often times create the same "good enough" code that most proprietary developers do. Now the theory behind Open Source is that developers around the world can see bloat and fix it. But with very few exceptions the corporate developers aren't going to make their code more efficient. So you have 75% of developers creating bloated code and at best 25% left to fix it. Meaning the amount of bloated code easily overwhelms those volunteers who are willing and able to fix it.
- 17y ago
- Gibbon 17y agoRoughly 11.5 million lines of code for the kernel.. That's way more than I would have imagined.. 5k lines a day are added and 2k are modified. (http://www.itworldcanada.com/a/IT-Workplace/68f20d4f-8ab9-402c-be30-df47e9812836.html http://www.itworldcanada.com/a/IT-Workplace/68f20d4f-8ab9-40...) IMHO it doesn't really jive with UNIX principles to have such a monolithic kernel.
- mahmud 17y agoIt's monolithic but modular 11.5M LOC for every platform it runs on, not to mention drivers for the devices it supports, and all the modules for other functionality, pluggable file-systems and networking stacks. Now throw in code for backward compatibility going further back at least to 2.4 and you have 11.5M lines. In reality, the core Linux kernel is just the stuff that compiles to vmlinux. You could read it all in a few weeks and understand it.
- skwiddor 17y agothats right, bloat filesystems dont belong in the kernel.
- agazso 17y agoYeah, there is the FUSE project: filesystem in a userspace. There are lots of interesting filesystems using this e.g. NTFS-3G, sshfs or ferrisfuse. These projects are valuable and most of them doesn't belong to the kernel as they use userspace libraries for network protocols and xml parsing. Unfortunately native disk file systems like NTFS-3G is lot more slower than their native counterparts.
- scythe 17y ago&totse users ran tests of zfs-fuse back in the day and found that it was actually noticeably slower than native ext3. From that, I'm inclined to argue the opposite: filesystems should almost definitely be in-kernel. Granted, their tests were on Linux, not on a modern microkernel.
- ecq 17y agoit's a monolithic kernel what did he expect. Remember the Tanenbaum vs. Torvalds debate? I wonder what Linus thinks about it now. btw, here they are together. http://lwn.net/images/conf/lca2007/lt-ast.jpg http://lwn.net/images/conf/lca2007/lt-ast.jpg
- Tuna-Fish 17y agoI'm sorry, but do you have any idea what you are talking about? How the kernel is organized has no meaning on how bloated it is. A microkernel can be just as bloated, in it the bloat is just not in the kernel itself, but in the processes providing the services a monolithic kernel would provide. If anything, a microkernel is always more bloated because of the extra code needed to do all the process sync. Linux being a microkernel would make the situation worse. (What we need to do is to start spending more time on cutting features.)
- zouhair 17y agoNow micro kernel doesn't seem that stupid after all?
- snprbob86 17y agoDespite the snarky tone of your comment, you hint at an interesting point: Unix is NOT the be-all, end-all definitive operating system. Linux is NOT even the be-all, end-all Unix (or Unix-like). There is A LOT of research left to be done in operating systems / kernels and many many years of evolution to come. It's a shame that it is SO HARD to build the ecosystem required to launch a kernel. New ideas are hard pressed to gain traction.
- pyre 17y agoYou realize that OS X uses a micro-kernel, right? And that GNU Hurd is a micro-kernel too?
- m0th87 17y agoI wouldn't say OS X is a microkernel. Much of its codebase was built off of Mach, but much was also built off of BSD. For example, its network stack logic resides in the kernel, which is certainly not microkernel-esque. See http://en.wikipedia.org/wiki/Darwin_(operating_system)#Kernel http://en.wikipedia.org/wiki/Darwin_(operating_system)#Kerne...
- vinutheraj 17y ago"Mac OS X is sort of microkernelish. Inside, it consists of Berkeley UNIX riding on top of a modified version of the Mach microkernel. Since all of it runs in kernel mode (to get that little extra bit of performance) it is not a true microkernel, but Carnegie Mellon University had Berkeley UNIX running on Mach in user space years ago, so it probably could be done again, albeit with a small amount of performance loss, as with L4Linux. Work is underway to port the Apple BSD code (Darwin) to L4 to make it a true microkernel system." - Andy Tanenbaum reference - http://www.cs.vu.nl/~ast/reliable-os/ http://www.cs.vu.nl/~ast/reliable-os/
- known 17y agoLinus is getting old.
- scythe 17y agoIs it really possible to implement the full functionality of the Linux kernel in such a way that it would be significantly smaller? Obviously, there's bloat from bad code, but would a redesign of the kernel be seriously useful, or just a waste of time? As a (possible) comparison, is NetBSD's kernel any smaller? It'd be nice to know if Linus had anything to say to that effect - it might lead to a more productive discussion than waxing lyrical about the wonders of microkernels.
- davidw 17y ago> Torvalds answered. "Acceptable and avoidable are two different things. It's unacceptable but it's also probably unavoidable." I think that's more or less where things are right now. You can't have tons of features without paying a price. They do a pretty good job of balancing things, but maybe someone will come along and improve things in the future.
- pmjordan 17y agoRather than bringing out the old monolithic vs microkernel chestnut, let's hear some realistic ideas on how to cut down the bloat. Does anyone know how much redundant code is in Linux right now? Would finding, isolating, generalising and merging it be a good way to start? I know of examples of this happening, e.g. libata taking over a lot of the old 'ide' drivers; the many attempts at unifying the various WLAN subsystems and drivers.
- rythie 17y agoWhilst libata and the unifying of the WLAN may make the code easier to maintain, it will only save a tiny amount of memory and in the short term, just like any new code, is going to increase the number of bugs. It sounds like the problem is that everyone is writing new features that probably not many people actually need (since they did fine with out it before - I'm not talking about drivers here) but it's slowing the whole kernel down and people need to go back and optimise that stuff.
- robryan 17y agoThat's the problem though isn't it, you start off light weight and you continue working, adding more and more new and useful features until you have it all but now have a giant slow bloated system.
- RyanMcGreal 17y agoThat's classic creeping featurism, and there doesn't appear to be an easy way out. Either your project is rich and full-featured or it's lean and lightweight. I don't see how it can be both.
- dkarl 17y agoThe approach that is inevitably taken by extremely large systems is to try to make all features optional except a sleek core. This is what OSGi is trying to do for Java. Unfortunately for the current debate, this is often called a "microkernel" approach. I'm not going to touch that one with a ten foot pole, but let's just say that hopefully Linux can be incrementally rearchitected to make more and more features optional without losing the benefits of its very successful "macrokernel" approach. After all, Linux is already designed to make most of its code optional via the module systems. The only problem is that its core is becoming not-so-sleek, and they're probably already getting as much mileage out of the module system as they can hope for. One interesting possibility is that Linux could avoid the performance hit of run-time dynamism by introducing a new kind of compile-time feature selection. It's obvious to me that C isn't up to the task, but it might not be obvious to someone much smarter than me. Or maybe they could extend C. These days we think of the Linux kernel community as old, conservative, and unlikely to surprise, but who knows? I'll keep my fingers crossed.
- ilitirit 17y agoI don't understand why he mentioned this. I understand software bloat to describe the situation that arises when unnecessary features added to a program increases it's size and/or decreases performance, without providing significant user benefits. In the article, Linus says "Acceptable and avoidable are two different things. It's unacceptable but it's also probably unavoidable." If it's unavoidable, then the features that were added aren't really bloat, because it implies that these features are necessary, and that they provide significant benefits. Necessary additions that provide significant benefits could be anything from security features to IO. IMO, Linus should either state what he thinks should be removed from the kernel, or he shouldn't even bother mentioning it at all. If on the other hand he's lamenting the fact that there's a cost associated with adding features (necessary or unnecessary), well then I think most people understand that you can't have your cake and eat it, so even from that perspective his point seems redundant.
- jsz0 17y agoIt's easy to make a tiny super efficient kernel. Just don't support stuff. WLAN? Who needs it? SMP? That'll never catch on. Take it out. Device driver support? Bleh. Let hardware makers post the source code on their websites and users can download and compile the code. No need to include it in the kernel source tree. And who uses anything but x86 anymore? Get all that PPC, ARM, SPARC non-sense out of there too. If we all work together we can make Linux completely useless but hey, it'll be tiny and efficient. We can sit around and congratulate each other for being such elegant programmers. We won't have to worry about those pesky end users anymore either.
- dkarl 17y agoDevice drivers and processor architectures might make the source tree big, but they don't make the kernel slower or its memory footprint larger. I believe you can even still compile the kernel without SMP support.
- known 17y agoWho should decide if a piece of code to run in kernel space or user space? user/developer/management/customer/congress/judiciary
- stinkytaco 17y agoHow about "Linux Lite"? I know, it's probably a silly idea, but is it possible that the kernel team could provide a "core" kernel, that runs on most modern hardware, most popular filesystems, etc. Shoot for some arbitrary support level (80% of systems). Also provide a "large" kernel with everything. Distro maintainers can use either. Actually, that brings up another point. Who, besides Slackware maybe, uses a stock kernel? Don't most distros customize it anyway? Am I being unfair in saying most Linux users don't really encounter the kernel at all?