11 ms·

Software bloat makes me sad

101 points by vanschelven 11y ago

tribaal 11y ago
That's pretty much the only reason I cringe every time somebody writes a piece of client code in golang. While I really like the language itself, statically linking everything is creating exactly what this article describes. Burn karma, burn. I don't care.
- Galanwe 11y ago
  I dont really understand the link between software bloat and static linking...
  - adrusi 11y ago
    When you statically link and executable, all the code from the libraries that it links to (at least the code that is directly or indirectly called by the application) is bundled into the binary. If you have 13 applications running that each statically link in 60% of a 2MB library, then you're using 15.6MB of memory where with dynamic linking you only need 2MB.
    - TheLoneWolfling 11y ago
      On the flip side, if you have a small tool that pulls in 1% of that 2MB library you only need 20KB of memory, whereas with dynamic linking you still need the full 2MB. Ditto, static linking leads to a lot more optimization opportunities, both in terms of performance and in terms of size. I would be very surprised if that 13 x 60% of a 2MB library was actually anywhere near 15.6MB.
  - deleted 11y ago
    [deleted]
- vezzy-fnord 11y ago
  Not gonna burn your karma, however: a) You are failing to account for the bloat caused by the presence of the dynamic linker, the dynamic loader, all the code that must operate under the assumptions of a dynamically linked environment, and the various auxiliaries used to treat shared library hell issues like WinSxS and libtool. b) Most modern static linking implementations do things like sharing of text segments across processes and other dedup techniques, so they're not actually that bloated at all.
  - tribaal 11y ago
    So maybe my comment stems from ignorance. Are you saying that two Golang projects sharing the same (large, for the sake of the argument) dependency would not take (size of the dependency * 2) on i.e. the Ubuntu live CD (as was the example in the fine article)?
    - ajuc 11y ago
      You have program X, and Y, both depending on big library L with 1000 symbols. X uses 1 symbol l1 from L, Y use another symbol l2 from L. With dynamic linking you need the whole library, no matter how much of it you use. With static linking (and --gc-sections -fdata-sections) you have parts of L compiled into X and Y binaries, but just the functions used (and their dependencies). It's possible that l1 and l2 share dependencies - and these will be duplicated. But still static linking is likely to be net win. Dynamic linking is mostly about maintanance.
    - orbifold 11y ago
      A statically linked binary can in principle only include code from the libraries it actually needs, so the overhead for some programs can be quite small and not as large as you might imagine. It also reduces the memory footprint at runtime.
    - TheLoneWolfling 11y ago
      Correct. I would not be surprised if those two Golang projects take less than the size of the dependency. 2x is a degenerate worst-case. With static linking, you only need to pull in things you actually need (and you can optimize from there!), whereas with dynamic linking you need to pull in the entire dependency regardless. (You only need to pull in one copy regardless of how many things are using the library, true. But you need to pull in one full copy always.) There are very few cases where you actually use the entire library in a project - and if you are indeed using an entire library in a project your project is necessarily large enough compared to the library that it's not much overhead percentage-wise.
- PythonicAlpha 11y ago
  I just don't agree. One of the best features I found in Golang was the statically linkage concept. There is just to much trouble with tools that come either in dozens of versions for different OS versions (Ubuntu, Debian, Mint, CentOS .... all Linux, but no Linux is the same Linux ..., it goes on: Ubuntu 12.04, 12.10, 14.x, ..., Debian Sid, Debian with and without systemd ...) While statically linking might add some bloat, I don't see, that the real bloat-factor is because of this. The real bloat-factor for example comes from programs that rely to much on template libraries (yes, I mean C++), that just unroll everything into new statements and because many even simple tasks have a multitude of solutions in the code ... each aside each other. When hundreds of developers are developing on one application, many of them invent the wheel again ...
- lsaferite 11y ago
  But doesn't static linking JUST link in the used features of the linked libs? Wouldn't that actually be better? Then I don't have 5 versions of a huge dynamic lib for 5 programs that all use slightly incompatible version and that only use 1 or 2 methods.
  - cbd1984 11y ago
    > But doesn't static linking JUST link in the used features of the linked libs? No, not if it works like it does in C and C++. Lisp used to do that, back when Symbolics still existed; it was called tree-shaking, as in "shake the call graph tree to see what falls out". The main function calls functions FOO, BAR, BAZ, which call FROTZ and QUUX, which call... and remove everything not in that tree. Of course, this gets to be equivalent to solving the Halting Problem the moment the EVAL function is called, so if the tree-shaker found any mention of EVAL anywhere in the code (probably after it had removed the provably dead code) it punted and linked in the whole Lisp runtime. It was the only viable solution, really, especially given how creative Lispers can get...
    - adrusi 11y ago
      In C and C++ you can conservatively eliminate unused code for statically linked libraries using --gc-sections and -fdata-sections. Go uses the Plan9 compiler architecture, which means that its linker knows more about the code and is responsible for more optimizations than its Unix equivalent, so it quite effectively eliminates dead code. Of course, libraries are often highly coupled within themselves, so not much code can be eliminated, and then they might pull in additional dependencies, etc., etc.
      - cbd1984 11y ago
        Interesting. I didn't realize there had been any substantive new work done on linkers in decades.
    - lispm 11y ago
      Lisp still does that. The commercial Lisps Allegro CL and LispWorks have that as a part of their respective delivery toolkit. See for example the LispWorks documentation: 4.6 How Delivery makes an image smaller, http://www.lispworks.com/documentation/lw70/DV/html/delivery-22.htm#pgfId-854332 http://www.lispworks.com/documentation/lw70/DV/html/delivery...
redcalx 11y ago
The notion of 'premature optimisation is evil' is partly to blame; It's not a terrible rule, but it's much better with conditions applied. i.e. apply some intelligence/judgement to where you use optimisation rather than just lazily following some rule.
- richmarr 11y ago
  Isn't that intelligence/judgement wrapped up in the interpretation of 'premature'?
  - redcalx 11y ago
    Fair point. I suspect that that subtlety to the rule is often lost to the reader. Perhaps a rewording is in order that puts emphasis on the conditional part (i.e. what constitutes premature)
    - vezzy-fnord 11y ago
      Or you could just reference the full quote, which reveals a remarkably different context than what is commonly interpreted.
      - redcalx 11y ago
        Ha! But that is the quote that gets repeated, i.e. it's been 'optimised' to aid viral/meme transfer, but not optimised in meaning :)
- kazinator 11y ago
  Firstly, lack of optimization for software size or execution time is a form of optimization: it's optimization of someone's time. Not writing a compiler for your scripting language saves you years. Not choosing carefully what features to include in a program, or what packages in a system image: design time saved again. It also saves considerable time if a user wants the program to do something, and by golly, "look, they thought of that already: there is a feature for it!" Seconds later, the user is just doing whatever they need instead of surfing the web for workarounds or calling support. Having all the features also increases the flexibility of customization: you have more choices about what you can remove to create smaller images which have specific feature sets. You can hardly remove anything from a minimal image to make a specialized custom image. I.e. by not making a program as minimal as possible, compact as possible or fast as possible, you are making some kind of trade off. Though the program is worse in some regard, there exist parameters (of something, not necessarily the program) which have gotten better: if you think about it a little, you can identify what those parameters are. The worsening in some of those parameters in a "bad trade" is what makes premature optimization "evil".
matthewmacleod 11y ago
I'm not really convinced. Leaving aside the argument about whether or not software is 'bloated' at all, or what 'bloat' actually means, this in particular stands out: “wouldn’t it have been easier not to create the problem in the first place?” It would seem obvious that the answer to this is indeed “yes”. It doesn't seem obvious at all, especially considering the implicit assumptions about whether or not bloat is actually a problem. There are three classic examples highlighted in the article of writing about this tradeoff: "Make it work, make it right, make it fast", "Premature optimization is the root of all evil", and "Bloatware and the 80/20 Myth" – but the argument about why these don't apply is a bit weak: "bloated software simply makes me sad". To be honest, I don't find we've got a problem, generally. My experience with my computer is that I can do many more things much faster than I could in the past. And I don't think that counts as bloat.
- scrollaway 11y ago
  > My experience with my computer is that I can do many more things much faster than I could in the past. Bloat is an invisible problem, but that doesn't mean it doesn't exist. Less bloat is more storage space, more performance, more battery life, etc. And the problem is you can't judge "bloat" standalone in one piece of software, you need to judge it as the sum of all the software the user is running. In the first equation, your numbers will always tell you "it's not worth it", unless things are really bad. In the latter, not so much. Maybe if the Bonbon Squash Series game I have on my android phone didn't link to liballthesocialthings, I'd shave 1MB off a game which is 50MB large. But maybe if every app on my phone did the same thing, I'd have 200MB more on my phone. My old HTC desire doesn't even have half that much memory total.
- TeMPOraL 11y ago
  I came to believe there are two types of programmers - those who get emotionally upset about ineficciencies, and those who couldn't care less. I'm in the first camp, so I understand the feelings of the author perfectly. You say your computer can do "more things much faster" than before - just imagine how many more things, how much faster you could do if the software didn't bloat itself up almost at the same pace as the hardware gets faster. And you see, I can understand abstraction layers. I can excuse coding for function first, speed second. But then there's the kind of bloat that can only be explained by programmers having no clue about what their software is doing. There's literally no reason why an article should lag in your browser, or Spotify should take 10+% of CPU on standby. There seems to be a lot of pointless code that's being executed, and nobody cares, because developers often have powerful machines. Not so the general population who then has to endure such creations.
yiyus 11y ago
You are not alone. See, for example, suckless.org.
prof_hobart 11y ago
>The usual counter to this is that the actual (as opposed to imagined) bottlenecks will only become apparent after intense usage. The usual counter is that "optimising" takes time. And all of the time that you're spending trying to optimise before release is time that no one is able to use your software at all. And the reality is that a lot of the bottlenecks can't be predicted in advance - who knows how may files the average user might want to load into your system, or whether your website is going to attract 100 or a million users a month? And many expected bottlenecks become largely irrelevant as technology moves on. Take his Ubuntu example - for what percentage of users does the fact that it doesn't fit on an actual CD really matter any more? Is it worth spending weeks or months fine-tuning the distro to a point that it can be fitted onto a medium that most people probably aren't even using anymore?
- bloateddevtards 11y ago
  I could edit a doc with a word processor that could fit onto a floppy, and it worked well. Now, why does the same task require a multi GB pos bloatfest to do the same job? And oftentimes, more slowly! Imagine for a second, the power of today's hardware running optimised lean code. It could be so good, but we accept bloated crappy bug ridden shitfests of OS and application software. We seem to care more about glossy and social then performance. It is a real problem.
  - FollowSteph3 11y ago
    The problem is you can't write that optimized lean assembly code in a lifetime anymore. Software today does a lot more than it used to under the seams now that most people forget about.
    - bloateddevtards 11y ago
      I am not sure we need to go as lean as hand-rolled asm, but we sure went down a wrong path when we just accept that word will get bigger and bigger with each release. No one asks why.
kkapelon 11y ago
"There are very little tools available to to help the user select unbloated software" Actually there is a whole community around this http://suckless.org/ http://suckless.org/
- Freaky 11y ago
  It's a funny idea of sucking less where any user configuration involves editing config.h and recompiling.
  - falcolas 11y ago
    Which is easier? Re-running make, or having to write and induce another library to read configuration files? Configuration files introduce a new point of failure (file not present, file corrupted, file not readable), a point of slowdown (how long does it take to read the file from disk, how long does it take to parse and load the file), and a point of complexity (parsing even ini files is pretty complicated, if you want to support the myriad of ways we write them). All of this for values which change once every... how long? Once a year? Once an install? Or for most configuration values, never? I still use configuration files, because I'm OK with including third party libraries... but I can certainly understand why some people may not be.
    - mondoshawan 11y ago
      Parsing config files is not hard. A simple key-value flat file is often more than enough. You don't need external libs for it, but it does add additional complexity. Suckless is about minimalism and speed, and if you want speed, you don't want extra junk like config files clogging up your code's mainline.
    - Freaky 11y ago
      > Which is easier? Re-running make, or having to write and induce another library to read configuration files? It's certainly easier for the programmer to just leave some of the work to the compiler, but it's at the cost of being a complete pain in the ass to packagers and users. Do you even have to do much work yourself? .Xresources is already parsed and loaded for you by xrdb if you can't afford the cost of an extra 100 microseconds doing it yourself. Is the API to interact with that mindblowing horrid or something? > All of this for values which change once every... how long? When I'm configuring software to taste, several times a minute. And considering this is likely to be my first exposure to the software, it better not suck completely.
kazinator 11y ago
Software bloat as such doesn't make me sad. Free open source software bloat makes me sad. However, that bloat is winning on its own merits. For instance, GNU/Linux distributions which are minimal are not popular. The popular ones tend to be the bloated ones. Vim has gotten a lot bigger in 20 years, yet I'm not going to jump ship to a smaller vi implementation. FOSS makes us confront the fact that we actually seem to like bloat. We inflict bloat upon ourselves --- there is no monopolistic software vendor there to blame.
- drabiega 11y ago
  Because bloat is like government. Multiple people might agree that there's too much of it, but they don't necessarily agree on which parts should be gotten rid of.
  - omegaham 11y ago
    This. As always, there's a relevant xkcd[1]. Features that I might decry as bloat are probably someone else's "necessary, to be preserved at all costs" features. If you want to cover a large number of use cases, you have to include all of them. And with that comes bloat. There's a reason why even a minimalist CLI-only Linux installation requires more than 24MB of RAM when Windows 3.1 runs on 3MB - the Linux kernel is capable of a hell of a lot more. [1]https://xkcd.com/1172/ https://xkcd.com/1172/
    - kazinator 11y ago
      A minimalist CLI-only Linux installation was still possible in around in 2 megs of RAM in 1994. A minimalist XWindow installation was possible with a 40 megabyte hard drive and 4 megs of RAM over a 386 CPU. (That's similar specs to what 68K-based Sun workstations had perhaps a decade before that.)
grandalf 11y ago
Atom is designed to harness community participation from a community that currently embraces some technology that can easily lead to bloat. So ti's a tradeoff. On the margin, many more people are likely to write a halfway decent Atom extension than learn elisp and write one for emacs.
aikah 11y ago
point 2 about atom: Well, writing a text editor is far more complicated than it seems at first place.In order to support big files (50mb logs for instance), you need to make all your code rely on complex memory management, streaming and caching. Maybe js is not the right tool for the job in order to implement these features. I always thought that atom should be build in Qt with an api that can interact with JS,a bit like Adobe products for instance. That way you still allow JS plugins, just like Photoshop while getting a max of perfs thanks to C++.
- outworlder 11y ago
  The problem with Atom is not due to javascript. Editing large files is a non-trivial problem. Editors such as Emacs (which has some C code, but a LOT of elisp) go to great lengths to make editing large files possible. The algorithms and data structures used are very clever. Atom does have C++ code, since is essentially Chromium. However, try a naive implementation in Assembly and you'll find that the Emacs Lisp buffer manipulation code will outperform it. In fact, if you try to use C strings (null-terminated), you're doomed from the beginning.
- runj__ 11y ago
  Atom is a super weird example, wouldn't it bloat the software to add support for files larger than 2MB? Wouldn't that be feature bloat? I mean, a code editor shouldn't have to deal with gigantic files since code shouldn't be written that way <sup>[citation needed]</sup>.
  - wongarsu 11y ago
    One obvious use case for opening large files in a code editor are XML files. XML files can be huge but can profit a lot from the features of a good code editor.
  - creichert 11y ago
    What if the code isn't written but generated? How would I inspect it with me editor?
emsy 11y ago
Atom and Vim (and imo even Ubuntu) aren't good example of software bloat. While they might be slow and use a lot of memory, they're usually shipped with basic functionality and more functions can be added and removed at any time. Good examples are iTunes, Office, iOS and Android (the latter two are usually shipped with non removable bloatware).
- falcolas 11y ago
  In my (controversial) opinion, Atom is a great example. It includes/requires Nodejs, an embedded web browser, and runs Javascript, to run a rather basic text editor. And in many cases, it's slow, CPU hungry, and memory intensive. Those three characteristics are the very definition of bloat to me.
  - emsy 11y ago
    I don't think Atom wants to be a basic text editor in the first place. It's shipped as a basic editor but the killer feature is that you can easily hack together extensions to suit your needs. It's a tradeoff: you pay for the html+js engine, but get a lot of potential developers. That being said, having worked with Atom on a larger project I can't confirm it's slow.
- smaili 11y ago
  > While they might be slow and use a lot of memory, they're usually shipped with basic functionality and more functions can be added and removed at any time. Isn't that what bloat is? By definition, something that's shipped with "basic functionality" should not consume so much.
  - emsy 11y ago
    The question I ask is: could it be built more lightweight? While there is certainly room for improvement, I really don't think you can have an editor that can be extended by most web developers with much less resources. You could write your own backend engines for js/html but now your using/wasting manpower instead of machine power. The same goes for Vim or Ubuntu: what could you cut to easily make it slimmer without hurting their purpose? Not much imo.On the other hand, the examples I stated have certainly many parts you could outsource to extensions or cut alltogether. To me, bloat means stuff that doesn't serve the software's purpose, and I still don't think the examples in the article have much of that.
maramono 11y ago
Related: great article on code inflation https://www.computer.org/cms/Computer.org/ComputingNow/issues/2015/04/mso2015020010.pdf https://www.computer.org/cms/Computer.org/ComputingNow/issue...
zimpenfish 11y ago
"Convenient though it would be if it were true, Mozilla is not big because it's full of useless crap. Mozilla is big because your needs are big. Your needs are big because the Internet is big. There are lots of small, lean web browsers out there that, incidentally, do almost nothing useful. If that's what you need, you've got options..." http://www.jwz.org/doc/easter-eggs.html http://www.jwz.org/doc/easter-eggs.html
- vezzy-fnord 11y ago
  That doesn't excuse resource inefficiency, which is really the main point. The issue of size can be further addressed by separation of mechanism and policy, and building for extensibility.
  - XorNot 11y ago
    Are you sure there's inefficiency? Do you have specific points in the code or behavior which are obviously inefficient, or does it just feel "too big"?
    - vezzy-fnord 11y ago
      The most obvious way it manifests is through progressive performance degradation, delayed response or output delivery times, locking up and an eventual necessity to kill/exec again for usable operation. It isn't about "feeling big" at all. You can have large, feature-packed programs that still respond quickly and don't fragment.
    - mondoshawan 11y ago
      ahem MorkDB. There's tons of code in Mozilla that isn't necessary to do the one job the browser was built for. I don't need code that manages bookmarks. At all. I don't need code that stores history for years. Honestly, I don't /want/ cookies persisted /at all/. I don't need themes to browse the web, and a reorganizeable UI is /never/ what I'm looking for in a browser. I don't need WebGL. I don't need Canvas. Hell, I definitely don't need syncing of all this extra bloated data around to services somewhere in the 'net, either. Frames? What about floating frames? Half this stuff is better left to tools that are built for it, the other half is junk I don't need, or is junk that actually creates more problems than it solves.
redcalx 11y ago
I've idly thought of writing a 'manifesto for quality software', and/or trying to start some kind of political-esque movement to that effect. I get annoyed by having to reboot my TV or radio when they freeze up, or by having delays to my channel changes, or having to navigate complex menus to do things that would have just been a physical button click before. (this is all part of the same problem as bloat IMO). This is where the likes of Apple or Dyson do well (or OK) I think. It's not necessarily that they're brilliant at what they do, it's just that the competition is pretty awful.
Kaali 11y ago
There is another side of extra resource use that I don't really see addressed except in the mobile space: ecology. Even though my computer can run all applications without a hitch, it is still very wasteful to constantly use CPU power because of technology choices or plain laziness. As an example, Spotify and Slack are two applications that seem to use most of my CPU after Chrome. Spotify and Slack combined seems to hover around 5-15% of total CPU (a two year old i7). When there is a lot of traffic in Slack I have seen it using 15-20% by itself, with multiple processes running and memory use going above 200 megs. Both applications work smoothly, but should they really use that much resources? A chat application? A music player? With modern CPU's I would expect them to be at the bottom of the process list when sorted by CPU usage. I used IRC on my Pentium 75Mhz and it ran fine. When simple applications are made so poorly that they use that much resources, what is the worldwide impact of that power use? And what about the users that don't have powerful and expensive CPU's?
- bloateddevtards 11y ago
  A very good point, we should have a metric that establishes how much carbon is released extra into the environment from cpu cycle wasting crap like atom and even more bloated versions of word. Clippy could pop up: "I can see you would like to release an extra ton of carbon into the atmos, why not upgrade word now!"
  - Kaali 11y ago
    Apple has started doing something about energy efficiency, probably because most of their computers are laptops where energy use is quite important. OSX tracks energy efficiency, which somehow calculates how much power a single application uses (CPU + GPU if I remember correctly). But I don't think they do anything with that info at the moment. I guess it would be a nice incentive for developers if OSX could notify you that an app that is on background is using a lot of energy at the moment, maybe even with a quit-button when on battery power. At least I wouldn't want my app to end up on that kind of a popup.
  - icebraining 11y ago
    Yeah, and then we could see how small the differences would actually be. A 15" Macbook Pro has a 99.5Wh battery and lasts about 8h; that's 12.5W. A program that decreases battery life by 20% would only mean an increase of 2-3W. Even if the machine and program are running 24/7 and all the energy comes from coal, that means an increase of less than 0.03T of CO2 per year. For comparison, the average carbon footprint of an US citizen is 20 Tons per year.
istvan__ 11y ago
There is a nice study in the subject: https://twitter.com/lix/status/589171043010412544/photo/1 https://twitter.com/lix/status/589171043010412544/photo/1
mwcampbell 11y ago
I've previously decried bloat too, but I find that my rants against software bloat are based on emotion, not reason. So, looking at this rationally, consider your favorite lightweight window manager or desktop environment for Linux. Is it fully internationalized, including support for CJK input methods? Is it fully accessible to users with disabilities, e.g. blind people and people with mobility impairments? Does it auto-mount USB thumb drives? Does it do all of the other things I'm not even aware of that are required for a fully usable desktop environment covering a wide variety of machines, users, and use cases? AFAIK, the only Linux desktop environments that come close are GNOME and Unity. Basically, the world is complicated, so real-world software has to be complicated too. It sounds like Atom could still use some optimization though.
- scrollaway 11y ago
  > AFAIK, the only Linux desktop environments that come close are GNOME and Unity. I'm the project lead of LXQt. All those things you listed can be supported without introducing much bloat at all. In DEs, most bloat comes from pure technical debt. Unnecessary libraries, duplicated code/libs, etc. Feature bloat is another matter and is more subjective. A lot of it is down to UX. Fun fact: KDE 4.x depends on ruby in most distros. Do you know why? Dolphin ships with a ruby script to update some folders (don't remember what it does). Nobody runs it. That's it.
  - Twirrim 11y ago
    > Fun fact: KDE 4.x depends on ruby in most distros. Do you know why? Dolphin ships with a ruby script to update some folders (don't remember what it does). Nobody runs it. That's it. Why did they allow an additional dependency on Ruby in? It seems like perl or python would be the more logical choice, given they'd already be on the system (heck.. maybe bash?)
    - exprL 11y ago
      Probably because a Ruby activist wrote a cool script and no-one cared about the implications of a new dependency at all.
Djonckheere 11y ago
this discussion reminds me of... http://s1.postimg.org/6tm9yeue7/invisible_software.jpg http://s1.postimg.org/6tm9yeue7/invisible_software.jpg
keedot 11y ago
Only experience with the same problem space can make you accurately predict where a bottleneck is going to be. Premature optimization will have you focusing on areas where optimization isn't required. YAGNI applies to all levels of development. I would rather see bloat and new ideas, than only experts working in a problem domain. My best ideas come from a marriage of understanding a single problem space, and a wonder what I can do in another. Bring on the bloat.
dpweb 11y ago
Over time, all software tends to get bigger and not necessarily better. Windows went from 7MB required (95) to 6GB required (Vista), 1000x bigger, in just 10 years. It's true in file formats as well and verbosity is one of the reasons XML was/is so hated. There is an irresitable urge to make things bigger bigger bigger, and huge cpu, mem, and disk resources are essentially free.
alkonaut 11y ago
Feature bloat is inevitable if you write an application in which thousands of users want different features. Most users want just a subset of the features, and think of the rest as "bloat" even though it might be a core feature for another user. The alternative is multiple lighter applications which would just reduce bloat at the cost of integration, a trade off almost no one seems to want.
- TheLoneWolfling 11y ago
  Or, you write the application as a plugin-centric architecture with very minimal core features, and go from there. Not perfect (especially as writing things as plugins tend to introduce a certain amount of bloat regardless), but it can be better regardless.
placebo 11y ago
As someone who is also saddened by software bloat, I also wondered a bit about where it stems from. There are of course the usual suspects (some of them mentioned in the article), but I think it boils down to another symptom of the modern rat-race culture, and it is not limited to software. When the goal of creating beauty and enjoying (as well as taking pride in) the creation process is replaced by the god of money and getting it out the door as fast as possible, the quality of the thing you're getting out the door has to deteriorate. I'd argue that a side effect of this is that the quality of life of both the creator and the receiver of the product decreases, but that is not immediately apparent when you're too busy trying to win a rat race... However, I'm optimistic. For one, I think that the tools we have today are superior in absolute terms (convenience, speed, ease of use etc.) than they ever were. Of course they are not exponentially better (like the hardware they run on), but they are better. Also, occasionally I see some gems here and there. There are still many developers who take pride in what they do and I enjoy finding that diamond once in a while when searching for an alternative technology to use in a new project.
vlad 11y ago
I don't care if an app is 20% bigger in file size as long as the user interface makes sense and the product actually works, especially if the bloat is due to a programming language or architecture that makes it easier to add new features or fix bugs.
- fullwedgewhale 11y ago
  But what if it's 200% bigger? I remember reading an article that looked at the time it takes to load MS office under Windows. Even though computers over the last 15-20 years or so are much, much faster, the wall-clock time to get Word or Excel up and running has pretty much stayed constant. Granted, disks haven't kept pace with CPUs, but when your computer is 100 times faster and it still takes just as long... Is that a good trade off?
fullwedgewhale 11y ago
One thing that strikes me is the explosion in dependencies in most software. I'm guilty of this, too. I've seen plenty of examples where an entire library or framework is added to a project just for a couple of features. Add a few libraries like that and suddenly you have a few megabytes of additional libraries, where maybe 90% or 95% of the features will never be used. A good article a while back looked at common unix utilities, comparing the size of commands like cp from the 1980's to the present. Most of the bloat had to do with features that almost nobody ever uses. It wouldn't be so bad if everyone used the same set of libraries. For example, almost all applications have a dependency on certain core libraries like libc. But we often use different libraries that do essentially the same thing or different versions of the same library, so instead of 1 copy of libfoo.jar, I have 2 copies of libbar.jar and 4 copies of libfoo.jar that may all do essentially the same thing. Then I have essentially the same functionality in C++ (some libraries that wrap collections), Python (where maybe one of they python versions wraps one of the C++ libraries, but a different version). And of course I have a version installed in each ruby environment. Add to that their dependencies, and the dependency's dependencies, and you have a perfect storm of craptastic. So libfoo.jar version 1.2.3 depends on libbaz.jar 2.3.4 which depends on libqux 1.5.7. Let's say each one is 250k, and all I ever used was some list sorting utility in libfoo. But I don't know what we could really do about it. You can't force everyone to program in C++ or limit them to a set of blessed libraries. I think maybe developers could be more judicious about when they could add a few lines of code and when they actually need to bring in a hard dependency an an external library. And it happens with commercial software as well. Maybe this is just the way the world will be.
- vezzy-fnord 11y ago
  A good article a while back looked at common unix utilities, comparing the size of commands like cp from the 1980's to the present. Most of the bloat had to do with features that almost nobody ever uses. That was actually a talk called "Bloat: How and Why UNIX Grew Up (and Out)": https://www.youtube.com/watch?v=Nbv9L-WIu0s https://www.youtube.com/watch?v=Nbv9L-WIu0s Well worth a viewing.
- adrusi 11y ago
  This is one of the biggest sources of bloat. In the node and ruby ecosystems especially, dependencies proliferate exponentially, where an application pulls in 12 libraries, each of which pull in 12 of their own, which each pull... Downloading the depedencies for Ghost, the node blogging platform with the explicit goal of simplicity and minimalism, takes me minutes. Compare this with the status quo when writing programs in C, where you might link to 4 libraries total, one of which pulls in 2 others as dependencies. I've come to suspect that the super convenient package managers that all the "modern" languages have are at fault for this.
rumcajz 11y ago
I guess devs don't necessarily believe that the bloated software is good. However, they believe that the project should be developed forever, which yields the same result. I've written about it here: http://250bpm.com/blog:50 http://250bpm.com/blog:50
al2o3cr 11y ago
"In fact, in my experience the process of finding such bottlenecks on running systems is itself quite time-consuming - time which cannot be spent actually reducing bloat." Wow. Talk about a self-refuting argument. If it's hard to figure out where to optimize a RUNNING system, how is it supposed to be easier to do so when the system is being created? Arguing "bloat" is "bad for performance"? No problem - graphs or GTFO. Otherwise all you're left with is the emotional "OMG NUMBERS ARE HIGHER NAOW THAN BEFORE" drivel.
deckiedan 11y ago
I agree in general, but the 'vim in combination with a few small plugins may lead to your whole computer locking up' is extremely misleading. It's the editor which has frozen, not the whole computer. And the plugin in question is a fairly complex autocompletion / analysis plugin for python. There's a bug listed in that thread, to do with it scanning all the files millions of times, but the big problem actually isn't bloat, in this instance, it's the non-async nature of vim. neovim should, I hope, solve the 'totally unresponsive' problem, and hopefully someone fixes the rope/vim interaction re-scanning all the files bug. But it's not bloat, per se. If your whole computer is locking up due to this, then that's a scheduling issue, and you could try switching scheduler, or lowering the 'nice' priority of that program. Not that we should have to care about such things - that's pretty terrible.
bryanlarsen 11y ago
Lotus 1-2-3 is the textbook example of the benefits and costs of the speed / feature trade-off curve. A major reason for 1-2-3 becoming dominant was its speed and memory efficiency; it was written in assembler. If you had a 640K machine, 1-2-3 let you write bigger spreadsheets than anybody else could. But 1-2-3 was killed by Excel because it lost the feature race. It's origins in assembly language gave it a significant disadvantage in this race.
- Shorel 11y ago
  > But 1-2-3 was killed by Excel because it lost the feature race. It's origins in assembly language gave it a significant disadvantage in this race. Yet the winner was written in C++ for many years, while the world was distracted with the less inefficient VB. Now some parts are .Net, but the crucial parts are still C++.
outworlder 11y ago
We can't all agree on the definition of 'software bloat'. Is using more memory resources for caching "bloat"? It irks me when someone pulls a task manager screenshot, orders by memory usage, and picks one application out of it. "See, program X is using XXXMBs of memory! Bloated!" Well, perhaps it is, perhaps it isn't. You are still using it, so it is probably doing its job well. Perhaps that "wasted" memory is being used for caching. This is specially true when people compare OS memory consumption. If an OS is not using all your memory, you are wasting it. It should be using it for something, be it caching, eager loading, whatever. As long as you can quickly reclaim it when needed. Same goes for CPU usage. "It's using 100% of my CPU!". Well, did you tell it do anything? If so, isn't that what you want? You want the task to complete quickly so it can go back to idle. Now, if it is supposed to be idling, and using a lot of CPU, that's a problem (Slack, I'm looking at you). From the article: > The same CD that hasn’t been able to fit Ubuntu since 2011 still fits approximately 150,000 pages of unformatted English text without any compression. Well, yeah. But one-dimensional metrics are useless. What about the number of packages? Did it increase? Is Ubuntu now packing high-resolution artwork, to be used with our 5k displays? What else has changed? I'm certain that the CD is not all source code, so the English text comparison is meaningless. >The burden of selecting software that is not bloated is entirely on the user. The default is bloated, if you want the unbloated version, you’ll have to work (search) for it yourself. And in many cases (e.g. anything that needs a web browser) such a search may not even be fruitful. Sometimes, the opposite is true. Take Vim and Atom, which are mentioned in the article. You can add packages to them, so increasing the perceived "bloat" is entirely up to the user. Unless the user takes the easy way and installs a "vim-full" package. Which, if we have memory and cpu to spare, isn't usually a problem. We are talking about VIM in the age of laptops with 16GB of RAM. > There are very little tools available to to help the user select unbloated software. Very few packages make any claims about their storage and runtime charactaristics at all Now, here I agree. It is something difficult to measure. > Over time, the battle against bloat is always lost. Even Ubuntu, which has traditionally presented itself (besides other things) as a method to extract a few extra life-years out of old hardware, is mentioned in the list above. In other words: it’s only less bloated than the alternatives. Yes. And yet it is inching closer and closer of being "ready for the desktop". It has to cater to a lot of people, which means bundling lots of features. Still less "bloated" than other commercial operating systems. Considering the number of available packages, Ubuntu offers a very good deal. > or even simply the introduction of flash screens. Oh, now we are in agreement. See, if you are waiting for a splash screen, you can't do your job. The software is not doing its job. Therefore, the amount of time being spent at the splash screen should be reduced, so that we can eliminate the need for one. In the bargain, eliminating the code and artwork for the splash screen, thus reducing bloat. Still, in some cases, this is unavoidable. Take games. They will often present splash screens when loading a level (if they have such a concept). Some of them will even go as far as present animated 3D geometry, using the GPU (and CPU). Which are mostly idle anyway, waiting for I/O. Sometimes, one can devise a better way of packing the files, to reduce loading times. Even more so when the actual serialization mechanism is inefficient(hi, Kerbal Space Program). But what they are usually trading off is a lag-free game play, in exchange of a loading screen (think of it as warming a huge cache). Is that bloat? The assets are huge, because we want them to be. Are you developing for an embedded platform? No, mobile doesn't count, they are effectively shrinked PCs from a few years ago (with non-mechanical storage even). If so, then worry about every CPU cycle you are using, as well as storage. Build your own linux distribution if you have to, compile with the exact flags for your platform so you can generate optimal code. Running in a battery powered device (no matter the size)? Then try not to use too much CPU, and certainly not constantly. Batch stuff, run in bursts, get it over quickly. Disable any non-essential tasks. Is a background task? Try not to disturb the rest of the system, please. A foreground, interactive desktop application? You are likely the focus of the user's attention, do whatever it takes to minimize the latency! The only reason not to gobble all RAM is that the user may be running other stuff too. And the reason for minimizing CPU usage is heat and power. Other than that, I bet a user will rather have an application that is making use of all resources available, if it means getting work done quickly. And feeling... "snappy"! If the user is not being impacted, then I don't see the problem. My issue with Slack, for instance, is that it uses a lot of system resources and doesn't feel fast in return.
legulere 11y ago
I think lots of the bloat is actually cruft that piles up over the years. Why can't we remove it? Because somebody will use the feature and complain about it getting removed.
a15971 11y ago
The bloat is simply a side effect of the evolution. Software is not as much designed as evolved. You make a name for yourself in SW development by adding features that others deem useful. You get money (in commercial setting) or prestige (in freely distributed software) if you seem to be producing something useful. It's trivially easy to recognize a contribution that adds code, but it's hard to recognize a contribution that means absence of something (e.g. absence of bloat, absence of memory hogging, ...). You get inclusion in newer, larger or more important projects easier the more you make yourself recognized. So any useful software gains more contributors that add things than those who remove things. Commercial code can and does gain developers if it earns money, open source software gains developers if it is deemed useful by programmers. This is a force that shapes both the group of involved developers and the resulting software in a process akin to evolution. In both cases the selection is biased towards adders of code. There's also always possible to add improvements that help some use cases and audiences. On the other hand, arguing for removing something or limiting might make you unpopular (you are seen as an obstacle to everlasting progress) and removed from the group of developers. Rare people have enough recognition and clout to prevent inclusion of something (Linus Torvalds is one of them - he can get away with rejecting patches to Linux kernel and he can play the role of the kernel guardian). On the whole, software expands until it fills the resources available.
chrismcb 11y ago
A blog that has a full screen image, with no text and adds no value to the story asks why there is software bloat?

101 points by vanschelven 11y ago

That's pretty much the only reason I cringe every time somebody writes a piece of client code in golang. While I really like the language itself, statically linking everything is creating exactly what this article describes. Burn karma, burn. I don't care.

Galanwe 11y ago
I dont really understand the link between software bloat and static linking...
- adrusi 11y ago
  When you statically link and executable, all the code from the libraries that it links to (at least the code that is directly or indirectly called by the application) is bundled into the binary. If you have 13 applications running that each statically link in 60% of a 2MB library, then you're using 15.6MB of memory where with dynamic linking you only need 2MB.
  - TheLoneWolfling 11y ago
    On the flip side, if you have a small tool that pulls in 1% of that 2MB library you only need 20KB of memory, whereas with dynamic linking you still need the full 2MB. Ditto, static linking leads to a lot more optimization opportunities, both in terms of performance and in terms of size. I would be very surprised if that 13 x 60% of a 2MB library was actually anywhere near 15.6MB.
- deleted 11y ago
  [deleted]
vezzy-fnord 11y ago
Not gonna burn your karma, however: a) You are failing to account for the bloat caused by the presence of the dynamic linker, the dynamic loader, all the code that must operate under the assumptions of a dynamically linked environment, and the various auxiliaries used to treat shared library hell issues like WinSxS and libtool. b) Most modern static linking implementations do things like sharing of text segments across processes and other dedup techniques, so they're not actually that bloated at all.
- tribaal 11y ago
  So maybe my comment stems from ignorance. Are you saying that two Golang projects sharing the same (large, for the sake of the argument) dependency would not take (size of the dependency * 2) on i.e. the Ubuntu live CD (as was the example in the fine article)?
  - ajuc 11y ago
    You have program X, and Y, both depending on big library L with 1000 symbols. X uses 1 symbol l1 from L, Y use another symbol l2 from L. With dynamic linking you need the whole library, no matter how much of it you use. With static linking (and --gc-sections -fdata-sections) you have parts of L compiled into X and Y binaries, but just the functions used (and their dependencies). It's possible that l1 and l2 share dependencies - and these will be duplicated. But still static linking is likely to be net win. Dynamic linking is mostly about maintanance.
  - orbifold 11y ago
    A statically linked binary can in principle only include code from the libraries it actually needs, so the overhead for some programs can be quite small and not as large as you might imagine. It also reduces the memory footprint at runtime.
  - TheLoneWolfling 11y ago
    Correct. I would not be surprised if those two Golang projects take less than the size of the dependency. 2x is a degenerate worst-case. With static linking, you only need to pull in things you actually need (and you can optimize from there!), whereas with dynamic linking you need to pull in the entire dependency regardless. (You only need to pull in one copy regardless of how many things are using the library, true. But you need to pull in one full copy always.) There are very few cases where you actually use the entire library in a project - and if you are indeed using an entire library in a project your project is necessarily large enough compared to the library that it's not much overhead percentage-wise.
PythonicAlpha 11y ago
I just don't agree. One of the best features I found in Golang was the statically linkage concept. There is just to much trouble with tools that come either in dozens of versions for different OS versions (Ubuntu, Debian, Mint, CentOS .... all Linux, but no Linux is the same Linux ..., it goes on: Ubuntu 12.04, 12.10, 14.x, ..., Debian Sid, Debian with and without systemd ...) While statically linking might add some bloat, I don't see, that the real bloat-factor is because of this. The real bloat-factor for example comes from programs that rely to much on template libraries (yes, I mean C++), that just unroll everything into new statements and because many even simple tasks have a multitude of solutions in the code ... each aside each other. When hundreds of developers are developing on one application, many of them invent the wheel again ...
lsaferite 11y ago
But doesn't static linking JUST link in the used features of the linked libs? Wouldn't that actually be better? Then I don't have 5 versions of a huge dynamic lib for 5 programs that all use slightly incompatible version and that only use 1 or 2 methods.
- cbd1984 11y ago
  > But doesn't static linking JUST link in the used features of the linked libs? No, not if it works like it does in C and C++. Lisp used to do that, back when Symbolics still existed; it was called tree-shaking, as in "shake the call graph tree to see what falls out". The main function calls functions FOO, BAR, BAZ, which call FROTZ and QUUX, which call... and remove everything not in that tree. Of course, this gets to be equivalent to solving the Halting Problem the moment the EVAL function is called, so if the tree-shaker found any mention of EVAL anywhere in the code (probably after it had removed the provably dead code) it punted and linked in the whole Lisp runtime. It was the only viable solution, really, especially given how creative Lispers can get...
  - adrusi 11y ago
    In C and C++ you can conservatively eliminate unused code for statically linked libraries using --gc-sections and -fdata-sections. Go uses the Plan9 compiler architecture, which means that its linker knows more about the code and is responsible for more optimizations than its Unix equivalent, so it quite effectively eliminates dead code. Of course, libraries are often highly coupled within themselves, so not much code can be eliminated, and then they might pull in additional dependencies, etc., etc.
    - cbd1984 11y ago
      Interesting. I didn't realize there had been any substantive new work done on linkers in decades.
  - lispm 11y ago
    Lisp still does that. The commercial Lisps Allegro CL and LispWorks have that as a part of their respective delivery toolkit. See for example the LispWorks documentation: 4.6 How Delivery makes an image smaller, http://www.lispworks.com/documentation/lw70/DV/html/delivery-22.htm#pgfId-854332 http://www.lispworks.com/documentation/lw70/DV/html/delivery...

redcalx 11y ago

The notion of 'premature optimisation is evil' is partly to blame; It's not a terrible rule, but it's much better with conditions applied. i.e. apply some intelligence/judgement to where you use optimisation rather than just lazily following some rule.

richmarr 11y ago
Isn't that intelligence/judgement wrapped up in the interpretation of 'premature'?
- redcalx 11y ago
  Fair point. I suspect that that subtlety to the rule is often lost to the reader. Perhaps a rewording is in order that puts emphasis on the conditional part (i.e. what constitutes premature)
  - vezzy-fnord 11y ago
    Or you could just reference the full quote, which reveals a remarkably different context than what is commonly interpreted.
    - redcalx 11y ago
      Ha! But that is the quote that gets repeated, i.e. it's been 'optimised' to aid viral/meme transfer, but not optimised in meaning :)
kazinator 11y ago
Firstly, lack of optimization for software size or execution time is a form of optimization: it's optimization of someone's time. Not writing a compiler for your scripting language saves you years. Not choosing carefully what features to include in a program, or what packages in a system image: design time saved again. It also saves considerable time if a user wants the program to do something, and by golly, "look, they thought of that already: there is a feature for it!" Seconds later, the user is just doing whatever they need instead of surfing the web for workarounds or calling support. Having all the features also increases the flexibility of customization: you have more choices about what you can remove to create smaller images which have specific feature sets. You can hardly remove anything from a minimal image to make a specialized custom image. I.e. by not making a program as minimal as possible, compact as possible or fast as possible, you are making some kind of trade off. Though the program is worse in some regard, there exist parameters (of something, not necessarily the program) which have gotten better: if you think about it a little, you can identify what those parameters are. The worsening in some of those parameters in a "bad trade" is what makes premature optimization "evil".

matthewmacleod 11y ago

I'm not really convinced. Leaving aside the argument about whether or not software is 'bloated' at all, or what 'bloat' actually means, this in particular stands out: “wouldn’t it have been easier not to create the problem in the first place?” It would seem obvious that the answer to this is indeed “yes”. It doesn't seem obvious at all, especially considering the implicit assumptions about whether or not bloat is actually a problem. There are three classic examples highlighted in the article of writing about this tradeoff: "Make it work, make it right, make it fast", "Premature optimization is the root of all evil", and "Bloatware and the 80/20 Myth" – but the argument about why these don't apply is a bit weak: "bloated software simply makes me sad". To be honest, I don't find we've got a problem, generally. My experience with my computer is that I can do many more things much faster than I could in the past. And I don't think that counts as bloat.

scrollaway 11y ago
> My experience with my computer is that I can do many more things much faster than I could in the past. Bloat is an invisible problem, but that doesn't mean it doesn't exist. Less bloat is more storage space, more performance, more battery life, etc. And the problem is you can't judge "bloat" standalone in one piece of software, you need to judge it as the sum of all the software the user is running. In the first equation, your numbers will always tell you "it's not worth it", unless things are really bad. In the latter, not so much. Maybe if the Bonbon Squash Series game I have on my android phone didn't link to liballthesocialthings, I'd shave 1MB off a game which is 50MB large. But maybe if every app on my phone did the same thing, I'd have 200MB more on my phone. My old HTC desire doesn't even have half that much memory total.
TeMPOraL 11y ago
I came to believe there are two types of programmers - those who get emotionally upset about ineficciencies, and those who couldn't care less. I'm in the first camp, so I understand the feelings of the author perfectly. You say your computer can do "more things much faster" than before - just imagine how many more things, how much faster you could do if the software didn't bloat itself up almost at the same pace as the hardware gets faster. And you see, I can understand abstraction layers. I can excuse coding for function first, speed second. But then there's the kind of bloat that can only be explained by programmers having no clue about what their software is doing. There's literally no reason why an article should lag in your browser, or Spotify should take 10+% of CPU on standby. There seems to be a lot of pointless code that's being executed, and nobody cares, because developers often have powerful machines. Not so the general population who then has to endure such creations.

yiyus 11y ago

You are not alone. See, for example, suckless.org.

prof_hobart 11y ago

>The usual counter to this is that the actual (as opposed to imagined) bottlenecks will only become apparent after intense usage. The usual counter is that "optimising" takes time. And all of the time that you're spending trying to optimise before release is time that no one is able to use your software at all. And the reality is that a lot of the bottlenecks can't be predicted in advance - who knows how may files the average user might want to load into your system, or whether your website is going to attract 100 or a million users a month? And many expected bottlenecks become largely irrelevant as technology moves on. Take his Ubuntu example - for what percentage of users does the fact that it doesn't fit on an actual CD really matter any more? Is it worth spending weeks or months fine-tuning the distro to a point that it can be fitted onto a medium that most people probably aren't even using anymore?

bloateddevtards 11y ago
I could edit a doc with a word processor that could fit onto a floppy, and it worked well. Now, why does the same task require a multi GB pos bloatfest to do the same job? And oftentimes, more slowly! Imagine for a second, the power of today's hardware running optimised lean code. It could be so good, but we accept bloated crappy bug ridden shitfests of OS and application software. We seem to care more about glossy and social then performance. It is a real problem.
- FollowSteph3 11y ago
  The problem is you can't write that optimized lean assembly code in a lifetime anymore. Software today does a lot more than it used to under the seams now that most people forget about.
  - bloateddevtards 11y ago
    I am not sure we need to go as lean as hand-rolled asm, but we sure went down a wrong path when we just accept that word will get bigger and bigger with each release. No one asks why.

kkapelon 11y ago

"There are very little tools available to to help the user select unbloated software" Actually there is a whole community around this http://suckless.org/ http://suckless.org/

Freaky 11y ago
It's a funny idea of sucking less where any user configuration involves editing config.h and recompiling.
- falcolas 11y ago
  Which is easier? Re-running make, or having to write and induce another library to read configuration files? Configuration files introduce a new point of failure (file not present, file corrupted, file not readable), a point of slowdown (how long does it take to read the file from disk, how long does it take to parse and load the file), and a point of complexity (parsing even ini files is pretty complicated, if you want to support the myriad of ways we write them). All of this for values which change once every... how long? Once a year? Once an install? Or for most configuration values, never? I still use configuration files, because I'm OK with including third party libraries... but I can certainly understand why some people may not be.
  - mondoshawan 11y ago
    Parsing config files is not hard. A simple key-value flat file is often more than enough. You don't need external libs for it, but it does add additional complexity. Suckless is about minimalism and speed, and if you want speed, you don't want extra junk like config files clogging up your code's mainline.
  - Freaky 11y ago
    > Which is easier? Re-running make, or having to write and induce another library to read configuration files? It's certainly easier for the programmer to just leave some of the work to the compiler, but it's at the cost of being a complete pain in the ass to packagers and users. Do you even have to do much work yourself? .Xresources is already parsed and loaded for you by xrdb if you can't afford the cost of an extra 100 microseconds doing it yourself. Is the API to interact with that mindblowing horrid or something? > All of this for values which change once every... how long? When I'm configuring software to taste, several times a minute. And considering this is likely to be my first exposure to the software, it better not suck completely.

kazinator 11y ago

Software bloat as such doesn't make me sad. Free open source software bloat makes me sad. However, that bloat is winning on its own merits. For instance, GNU/Linux distributions which are minimal are not popular. The popular ones tend to be the bloated ones. Vim has gotten a lot bigger in 20 years, yet I'm not going to jump ship to a smaller vi implementation. FOSS makes us confront the fact that we actually seem to like bloat. We inflict bloat upon ourselves --- there is no monopolistic software vendor there to blame.

drabiega 11y ago
Because bloat is like government. Multiple people might agree that there's too much of it, but they don't necessarily agree on which parts should be gotten rid of.
- omegaham 11y ago
  This. As always, there's a relevant xkcd[1]. Features that I might decry as bloat are probably someone else's "necessary, to be preserved at all costs" features. If you want to cover a large number of use cases, you have to include all of them. And with that comes bloat. There's a reason why even a minimalist CLI-only Linux installation requires more than 24MB of RAM when Windows 3.1 runs on 3MB - the Linux kernel is capable of a hell of a lot more. [1]https://xkcd.com/1172/ https://xkcd.com/1172/
  - kazinator 11y ago
    A minimalist CLI-only Linux installation was still possible in around in 2 megs of RAM in 1994. A minimalist XWindow installation was possible with a 40 megabyte hard drive and 4 megs of RAM over a 386 CPU. (That's similar specs to what 68K-based Sun workstations had perhaps a decade before that.)

grandalf 11y ago

Atom is designed to harness community participation from a community that currently embraces some technology that can easily lead to bloat. So ti's a tradeoff. On the margin, many more people are likely to write a halfway decent Atom extension than learn elisp and write one for emacs.

aikah 11y ago

point 2 about atom: Well, writing a text editor is far more complicated than it seems at first place.In order to support big files (50mb logs for instance), you need to make all your code rely on complex memory management, streaming and caching. Maybe js is not the right tool for the job in order to implement these features. I always thought that atom should be build in Qt with an api that can interact with JS,a bit like Adobe products for instance. That way you still allow JS plugins, just like Photoshop while getting a max of perfs thanks to C++.

outworlder 11y ago
The problem with Atom is not due to javascript. Editing large files is a non-trivial problem. Editors such as Emacs (which has some C code, but a LOT of elisp) go to great lengths to make editing large files possible. The algorithms and data structures used are very clever. Atom does have C++ code, since is essentially Chromium. However, try a naive implementation in Assembly and you'll find that the Emacs Lisp buffer manipulation code will outperform it. In fact, if you try to use C strings (null-terminated), you're doomed from the beginning.
runj__ 11y ago
Atom is a super weird example, wouldn't it bloat the software to add support for files larger than 2MB? Wouldn't that be feature bloat? I mean, a code editor shouldn't have to deal with gigantic files since code shouldn't be written that way <sup>[citation needed]</sup>.
- wongarsu 11y ago
  One obvious use case for opening large files in a code editor are XML files. XML files can be huge but can profit a lot from the features of a good code editor.
- creichert 11y ago
  What if the code isn't written but generated? How would I inspect it with me editor?

emsy 11y ago

Atom and Vim (and imo even Ubuntu) aren't good example of software bloat. While they might be slow and use a lot of memory, they're usually shipped with basic functionality and more functions can be added and removed at any time. Good examples are iTunes, Office, iOS and Android (the latter two are usually shipped with non removable bloatware).

falcolas 11y ago
In my (controversial) opinion, Atom is a great example. It includes/requires Nodejs, an embedded web browser, and runs Javascript, to run a rather basic text editor. And in many cases, it's slow, CPU hungry, and memory intensive. Those three characteristics are the very definition of bloat to me.
- emsy 11y ago
  I don't think Atom wants to be a basic text editor in the first place. It's shipped as a basic editor but the killer feature is that you can easily hack together extensions to suit your needs. It's a tradeoff: you pay for the html+js engine, but get a lot of potential developers. That being said, having worked with Atom on a larger project I can't confirm it's slow.
smaili 11y ago
> While they might be slow and use a lot of memory, they're usually shipped with basic functionality and more functions can be added and removed at any time. Isn't that what bloat is? By definition, something that's shipped with "basic functionality" should not consume so much.
- emsy 11y ago
  The question I ask is: could it be built more lightweight? While there is certainly room for improvement, I really don't think you can have an editor that can be extended by most web developers with much less resources. You could write your own backend engines for js/html but now your using/wasting manpower instead of machine power. The same goes for Vim or Ubuntu: what could you cut to easily make it slimmer without hurting their purpose? Not much imo.On the other hand, the examples I stated have certainly many parts you could outsource to extensions or cut alltogether. To me, bloat means stuff that doesn't serve the software's purpose, and I still don't think the examples in the article have much of that.

maramono 11y ago

Related: great article on code inflation https://www.computer.org/cms/Computer.org/ComputingNow/issues/2015/04/mso2015020010.pdf https://www.computer.org/cms/Computer.org/ComputingNow/issue...

zimpenfish 11y ago

"Convenient though it would be if it were true, Mozilla is not big because it's full of useless crap. Mozilla is big because your needs are big. Your needs are big because the Internet is big. There are lots of small, lean web browsers out there that, incidentally, do almost nothing useful. If that's what you need, you've got options..." http://www.jwz.org/doc/easter-eggs.html http://www.jwz.org/doc/easter-eggs.html

vezzy-fnord 11y ago
That doesn't excuse resource inefficiency, which is really the main point. The issue of size can be further addressed by separation of mechanism and policy, and building for extensibility.
- XorNot 11y ago
  Are you sure there's inefficiency? Do you have specific points in the code or behavior which are obviously inefficient, or does it just feel "too big"?
  - vezzy-fnord 11y ago
    The most obvious way it manifests is through progressive performance degradation, delayed response or output delivery times, locking up and an eventual necessity to kill/exec again for usable operation. It isn't about "feeling big" at all. You can have large, feature-packed programs that still respond quickly and don't fragment.
  - mondoshawan 11y ago
    ahem MorkDB. There's tons of code in Mozilla that isn't necessary to do the one job the browser was built for. I don't need code that manages bookmarks. At all. I don't need code that stores history for years. Honestly, I don't /want/ cookies persisted /at all/. I don't need themes to browse the web, and a reorganizeable UI is /never/ what I'm looking for in a browser. I don't need WebGL. I don't need Canvas. Hell, I definitely don't need syncing of all this extra bloated data around to services somewhere in the 'net, either. Frames? What about floating frames? Half this stuff is better left to tools that are built for it, the other half is junk I don't need, or is junk that actually creates more problems than it solves.

redcalx 11y ago

I've idly thought of writing a 'manifesto for quality software', and/or trying to start some kind of political-esque movement to that effect. I get annoyed by having to reboot my TV or radio when they freeze up, or by having delays to my channel changes, or having to navigate complex menus to do things that would have just been a physical button click before. (this is all part of the same problem as bloat IMO). This is where the likes of Apple or Dyson do well (or OK) I think. It's not necessarily that they're brilliant at what they do, it's just that the competition is pretty awful.

Kaali 11y ago

There is another side of extra resource use that I don't really see addressed except in the mobile space: ecology. Even though my computer can run all applications without a hitch, it is still very wasteful to constantly use CPU power because of technology choices or plain laziness. As an example, Spotify and Slack are two applications that seem to use most of my CPU after Chrome. Spotify and Slack combined seems to hover around 5-15% of total CPU (a two year old i7). When there is a lot of traffic in Slack I have seen it using 15-20% by itself, with multiple processes running and memory use going above 200 megs. Both applications work smoothly, but should they really use that much resources? A chat application? A music player? With modern CPU's I would expect them to be at the bottom of the process list when sorted by CPU usage. I used IRC on my Pentium 75Mhz and it ran fine. When simple applications are made so poorly that they use that much resources, what is the worldwide impact of that power use? And what about the users that don't have powerful and expensive CPU's?

bloateddevtards 11y ago
A very good point, we should have a metric that establishes how much carbon is released extra into the environment from cpu cycle wasting crap like atom and even more bloated versions of word. Clippy could pop up: "I can see you would like to release an extra ton of carbon into the atmos, why not upgrade word now!"
- Kaali 11y ago
  Apple has started doing something about energy efficiency, probably because most of their computers are laptops where energy use is quite important. OSX tracks energy efficiency, which somehow calculates how much power a single application uses (CPU + GPU if I remember correctly). But I don't think they do anything with that info at the moment. I guess it would be a nice incentive for developers if OSX could notify you that an app that is on background is using a lot of energy at the moment, maybe even with a quit-button when on battery power. At least I wouldn't want my app to end up on that kind of a popup.
- icebraining 11y ago
  Yeah, and then we could see how small the differences would actually be. A 15" Macbook Pro has a 99.5Wh battery and lasts about 8h; that's 12.5W. A program that decreases battery life by 20% would only mean an increase of 2-3W. Even if the machine and program are running 24/7 and all the energy comes from coal, that means an increase of less than 0.03T of CO2 per year. For comparison, the average carbon footprint of an US citizen is 20 Tons per year.

istvan__ 11y ago

There is a nice study in the subject: https://twitter.com/lix/status/589171043010412544/photo/1 https://twitter.com/lix/status/589171043010412544/photo/1

mwcampbell 11y ago

I've previously decried bloat too, but I find that my rants against software bloat are based on emotion, not reason. So, looking at this rationally, consider your favorite lightweight window manager or desktop environment for Linux. Is it fully internationalized, including support for CJK input methods? Is it fully accessible to users with disabilities, e.g. blind people and people with mobility impairments? Does it auto-mount USB thumb drives? Does it do all of the other things I'm not even aware of that are required for a fully usable desktop environment covering a wide variety of machines, users, and use cases? AFAIK, the only Linux desktop environments that come close are GNOME and Unity. Basically, the world is complicated, so real-world software has to be complicated too. It sounds like Atom could still use some optimization though.

scrollaway 11y ago
> AFAIK, the only Linux desktop environments that come close are GNOME and Unity. I'm the project lead of LXQt. All those things you listed can be supported without introducing much bloat at all. In DEs, most bloat comes from pure technical debt. Unnecessary libraries, duplicated code/libs, etc. Feature bloat is another matter and is more subjective. A lot of it is down to UX. Fun fact: KDE 4.x depends on ruby in most distros. Do you know why? Dolphin ships with a ruby script to update some folders (don't remember what it does). Nobody runs it. That's it.
- Twirrim 11y ago
  > Fun fact: KDE 4.x depends on ruby in most distros. Do you know why? Dolphin ships with a ruby script to update some folders (don't remember what it does). Nobody runs it. That's it. Why did they allow an additional dependency on Ruby in? It seems like perl or python would be the more logical choice, given they'd already be on the system (heck.. maybe bash?)
  - exprL 11y ago
    Probably because a Ruby activist wrote a cool script and no-one cared about the implications of a new dependency at all.

Djonckheere 11y ago

this discussion reminds me of... http://s1.postimg.org/6tm9yeue7/invisible_software.jpg http://s1.postimg.org/6tm9yeue7/invisible_software.jpg

keedot 11y ago

Only experience with the same problem space can make you accurately predict where a bottleneck is going to be. Premature optimization will have you focusing on areas where optimization isn't required. YAGNI applies to all levels of development. I would rather see bloat and new ideas, than only experts working in a problem domain. My best ideas come from a marriage of understanding a single problem space, and a wonder what I can do in another. Bring on the bloat.

dpweb 11y ago

Over time, all software tends to get bigger and not necessarily better. Windows went from 7MB required (95) to 6GB required (Vista), 1000x bigger, in just 10 years. It's true in file formats as well and verbosity is one of the reasons XML was/is so hated. There is an irresitable urge to make things bigger bigger bigger, and huge cpu, mem, and disk resources are essentially free.

alkonaut 11y ago

Feature bloat is inevitable if you write an application in which thousands of users want different features. Most users want just a subset of the features, and think of the rest as "bloat" even though it might be a core feature for another user. The alternative is multiple lighter applications which would just reduce bloat at the cost of integration, a trade off almost no one seems to want.

TheLoneWolfling 11y ago
Or, you write the application as a plugin-centric architecture with very minimal core features, and go from there. Not perfect (especially as writing things as plugins tend to introduce a certain amount of bloat regardless), but it can be better regardless.

placebo 11y ago

As someone who is also saddened by software bloat, I also wondered a bit about where it stems from. There are of course the usual suspects (some of them mentioned in the article), but I think it boils down to another symptom of the modern rat-race culture, and it is not limited to software. When the goal of creating beauty and enjoying (as well as taking pride in) the creation process is replaced by the god of money and getting it out the door as fast as possible, the quality of the thing you're getting out the door has to deteriorate. I'd argue that a side effect of this is that the quality of life of both the creator and the receiver of the product decreases, but that is not immediately apparent when you're too busy trying to win a rat race... However, I'm optimistic. For one, I think that the tools we have today are superior in absolute terms (convenience, speed, ease of use etc.) than they ever were. Of course they are not exponentially better (like the hardware they run on), but they are better. Also, occasionally I see some gems here and there. There are still many developers who take pride in what they do and I enjoy finding that diamond once in a while when searching for an alternative technology to use in a new project.

vlad 11y ago

I don't care if an app is 20% bigger in file size as long as the user interface makes sense and the product actually works, especially if the bloat is due to a programming language or architecture that makes it easier to add new features or fix bugs.

fullwedgewhale 11y ago
But what if it's 200% bigger? I remember reading an article that looked at the time it takes to load MS office under Windows. Even though computers over the last 15-20 years or so are much, much faster, the wall-clock time to get Word or Excel up and running has pretty much stayed constant. Granted, disks haven't kept pace with CPUs, but when your computer is 100 times faster and it still takes just as long... Is that a good trade off?

fullwedgewhale 11y ago

One thing that strikes me is the explosion in dependencies in most software. I'm guilty of this, too. I've seen plenty of examples where an entire library or framework is added to a project just for a couple of features. Add a few libraries like that and suddenly you have a few megabytes of additional libraries, where maybe 90% or 95% of the features will never be used. A good article a while back looked at common unix utilities, comparing the size of commands like cp from the 1980's to the present. Most of the bloat had to do with features that almost nobody ever uses. It wouldn't be so bad if everyone used the same set of libraries. For example, almost all applications have a dependency on certain core libraries like libc. But we often use different libraries that do essentially the same thing or different versions of the same library, so instead of 1 copy of libfoo.jar, I have 2 copies of libbar.jar and 4 copies of libfoo.jar that may all do essentially the same thing. Then I have essentially the same functionality in C++ (some libraries that wrap collections), Python (where maybe one of they python versions wraps one of the C++ libraries, but a different version). And of course I have a version installed in each ruby environment. Add to that their dependencies, and the dependency's dependencies, and you have a perfect storm of craptastic. So libfoo.jar version 1.2.3 depends on libbaz.jar 2.3.4 which depends on libqux 1.5.7. Let's say each one is 250k, and all I ever used was some list sorting utility in libfoo. But I don't know what we could really do about it. You can't force everyone to program in C++ or limit them to a set of blessed libraries. I think maybe developers could be more judicious about when they could add a few lines of code and when they actually need to bring in a hard dependency an an external library. And it happens with commercial software as well. Maybe this is just the way the world will be.

vezzy-fnord 11y ago
A good article a while back looked at common unix utilities, comparing the size of commands like cp from the 1980's to the present. Most of the bloat had to do with features that almost nobody ever uses. That was actually a talk called "Bloat: How and Why UNIX Grew Up (and Out)": https://www.youtube.com/watch?v=Nbv9L-WIu0s https://www.youtube.com/watch?v=Nbv9L-WIu0s Well worth a viewing.
adrusi 11y ago
This is one of the biggest sources of bloat. In the node and ruby ecosystems especially, dependencies proliferate exponentially, where an application pulls in 12 libraries, each of which pull in 12 of their own, which each pull... Downloading the depedencies for Ghost, the node blogging platform with the explicit goal of simplicity and minimalism, takes me minutes. Compare this with the status quo when writing programs in C, where you might link to 4 libraries total, one of which pulls in 2 others as dependencies. I've come to suspect that the super convenient package managers that all the "modern" languages have are at fault for this.

rumcajz 11y ago

I guess devs don't necessarily believe that the bloated software is good. However, they believe that the project should be developed forever, which yields the same result. I've written about it here: http://250bpm.com/blog:50 http://250bpm.com/blog:50

al2o3cr 11y ago

"In fact, in my experience the process of finding such bottlenecks on running systems is itself quite time-consuming - time which cannot be spent actually reducing bloat." Wow. Talk about a self-refuting argument. If it's hard to figure out where to optimize a RUNNING system, how is it supposed to be easier to do so when the system is being created? Arguing "bloat" is "bad for performance"? No problem - graphs or GTFO. Otherwise all you're left with is the emotional "OMG NUMBERS ARE HIGHER NAOW THAN BEFORE" drivel.

deckiedan 11y ago

I agree in general, but the 'vim in combination with a few small plugins may lead to your whole computer locking up' is extremely misleading. It's the editor which has frozen, not the whole computer. And the plugin in question is a fairly complex autocompletion / analysis plugin for python. There's a bug listed in that thread, to do with it scanning all the files millions of times, but the big problem actually isn't bloat, in this instance, it's the non-async nature of vim. neovim should, I hope, solve the 'totally unresponsive' problem, and hopefully someone fixes the rope/vim interaction re-scanning all the files bug. But it's not bloat, per se. If your whole computer is locking up due to this, then that's a scheduling issue, and you could try switching scheduler, or lowering the 'nice' priority of that program. Not that we should have to care about such things - that's pretty terrible.

bryanlarsen 11y ago

Lotus 1-2-3 is the textbook example of the benefits and costs of the speed / feature trade-off curve. A major reason for 1-2-3 becoming dominant was its speed and memory efficiency; it was written in assembler. If you had a 640K machine, 1-2-3 let you write bigger spreadsheets than anybody else could. But 1-2-3 was killed by Excel because it lost the feature race. It's origins in assembly language gave it a significant disadvantage in this race.

Shorel 11y ago
> But 1-2-3 was killed by Excel because it lost the feature race. It's origins in assembly language gave it a significant disadvantage in this race. Yet the winner was written in C++ for many years, while the world was distracted with the less inefficient VB. Now some parts are .Net, but the crucial parts are still C++.

outworlder 11y ago

We can't all agree on the definition of 'software bloat'. Is using more memory resources for caching "bloat"? It irks me when someone pulls a task manager screenshot, orders by memory usage, and picks one application out of it. "See, program X is using XXXMBs of memory! Bloated!" Well, perhaps it is, perhaps it isn't. You are still using it, so it is probably doing its job well. Perhaps that "wasted" memory is being used for caching. This is specially true when people compare OS memory consumption. If an OS is not using all your memory, you are wasting it. It should be using it for something, be it caching, eager loading, whatever. As long as you can quickly reclaim it when needed. Same goes for CPU usage. "It's using 100% of my CPU!". Well, did you tell it do anything? If so, isn't that what you want? You want the task to complete quickly so it can go back to idle. Now, if it is supposed to be idling, and using a lot of CPU, that's a problem (Slack, I'm looking at you). From the article: > The same CD that hasn’t been able to fit Ubuntu since 2011 still fits approximately 150,000 pages of unformatted English text without any compression. Well, yeah. But one-dimensional metrics are useless. What about the number of packages? Did it increase? Is Ubuntu now packing high-resolution artwork, to be used with our 5k displays? What else has changed? I'm certain that the CD is not all source code, so the English text comparison is meaningless. >The burden of selecting software that is not bloated is entirely on the user. The default is bloated, if you want the unbloated version, you’ll have to work (search) for it yourself. And in many cases (e.g. anything that needs a web browser) such a search may not even be fruitful. Sometimes, the opposite is true. Take Vim and Atom, which are mentioned in the article. You can add packages to them, so increasing the perceived "bloat" is entirely up to the user. Unless the user takes the easy way and installs a "vim-full" package. Which, if we have memory and cpu to spare, isn't usually a problem. We are talking about VIM in the age of laptops with 16GB of RAM. > There are very little tools available to to help the user select unbloated software. Very few packages make any claims about their storage and runtime charactaristics at all Now, here I agree. It is something difficult to measure. > Over time, the battle against bloat is always lost. Even Ubuntu, which has traditionally presented itself (besides other things) as a method to extract a few extra life-years out of old hardware, is mentioned in the list above. In other words: it’s only less bloated than the alternatives. Yes. And yet it is inching closer and closer of being "ready for the desktop". It has to cater to a lot of people, which means bundling lots of features. Still less "bloated" than other commercial operating systems. Considering the number of available packages, Ubuntu offers a very good deal. > or even simply the introduction of flash screens. Oh, now we are in agreement. See, if you are waiting for a splash screen, you can't do your job. The software is not doing its job. Therefore, the amount of time being spent at the splash screen should be reduced, so that we can eliminate the need for one. In the bargain, eliminating the code and artwork for the splash screen, thus reducing bloat. Still, in some cases, this is unavoidable. Take games. They will often present splash screens when loading a level (if they have such a concept). Some of them will even go as far as present animated 3D geometry, using the GPU (and CPU). Which are mostly idle anyway, waiting for I/O. Sometimes, one can devise a better way of packing the files, to reduce loading times. Even more so when the actual serialization mechanism is inefficient(hi, Kerbal Space Program). But what they are usually trading off is a lag-free game play, in exchange of a loading screen (think of it as warming a huge cache). Is that bloat? The assets are huge, because we want them to be. Are you developing for an embedded platform? No, mobile doesn't count, they are effectively shrinked PCs from a few years ago (with non-mechanical storage even). If so, then worry about every CPU cycle you are using, as well as storage. Build your own linux distribution if you have to, compile with the exact flags for your platform so you can generate optimal code. Running in a battery powered device (no matter the size)? Then try not to use too much CPU, and certainly not constantly. Batch stuff, run in bursts, get it over quickly. Disable any non-essential tasks. Is a background task? Try not to disturb the rest of the system, please. A foreground, interactive desktop application? You are likely the focus of the user's attention, do whatever it takes to minimize the latency! The only reason not to gobble all RAM is that the user may be running other stuff too. And the reason for minimizing CPU usage is heat and power. Other than that, I bet a user will rather have an application that is making use of all resources available, if it means getting work done quickly. And feeling... "snappy"! If the user is not being impacted, then I don't see the problem. My issue with Slack, for instance, is that it uses a lot of system resources and doesn't feel fast in return.

legulere 11y ago

I think lots of the bloat is actually cruft that piles up over the years. Why can't we remove it? Because somebody will use the feature and complain about it getting removed.

a15971 11y ago

The bloat is simply a side effect of the evolution. Software is not as much designed as evolved. You make a name for yourself in SW development by adding features that others deem useful. You get money (in commercial setting) or prestige (in freely distributed software) if you seem to be producing something useful. It's trivially easy to recognize a contribution that adds code, but it's hard to recognize a contribution that means absence of something (e.g. absence of bloat, absence of memory hogging, ...). You get inclusion in newer, larger or more important projects easier the more you make yourself recognized. So any useful software gains more contributors that add things than those who remove things. Commercial code can and does gain developers if it earns money, open source software gains developers if it is deemed useful by programmers. This is a force that shapes both the group of involved developers and the resulting software in a process akin to evolution. In both cases the selection is biased towards adders of code. There's also always possible to add improvements that help some use cases and audiences. On the other hand, arguing for removing something or limiting might make you unpopular (you are seen as an obstacle to everlasting progress) and removed from the group of developers. Rare people have enough recognition and clout to prevent inclusion of something (Linus Torvalds is one of them - he can get away with rejecting patches to Linux kernel and he can play the role of the kernel guardian). On the whole, software expands until it fills the resources available.

chrismcb 11y ago

A blog that has a full screen image, with no text and adds no value to the story asks why there is software bloat?