11 ms·

I never understand where this increasing size comes from. For videos or hi-res photographs, I understand. There is however no reason that code, either compiled

• kutkloon7 9y ago
  While I was typing this comment ghostly pointed out some great blog entries: https://news.ycombinator.com/item?id=14901602 https://news.ycombinator.com/item?id=14901602
• rlv-dan 9y ago
  Could it be that whenever coders today need some fairly trivial functionality, they tend to go out and find a library that contains it. So you end up with lots and lots of libraries where only a tiny bits of them are used. Just a hypothesis though.
  • deft 9y ago
    I'm pretty sure this is part of it. Remember leftpad? For electron based apps they're just bundling all of chrome which explains the size too.
    • allover 9y ago
      Leftpad is not an example of the issue mentioned. Leftpad is an example of a small module that was being used by a lot of projects. GPP is talking about pulling in larger libraries which contain code not actually used by the app. (Also Electron bundles 'just the rendering library from Chromium' [1], not 'all of Chrome'). [1] https://electron.atom.io/docs/tutorial/about/#core-philosophy https://electron.atom.io/docs/tutorial/about/#core-philosoph...
      • deft 9y ago
        Sorry for not being specific. As for leftpad I just meant the concept of using libraries when you really don't need to. Not the size.
        • allover 9y ago
          Well it wasn't that you weren't being specific, more that you were casually maligning the JS community unfairly, when the article isn't even about web/JS :) (I also don't agree with 'don't need to'. The main takeaway from the leftpad debarcle was the fixes to the npm module deletion policy, and hopefully people learning they shouldn't rely on an 'npm install' for production deployments! Whether people should use small modules is still up for debate, there are trade-offs [1] [2]). [1] https://github.com/sindresorhus/ama/issues/10#issuecomment-117766328 https://github.com/sindresorhus/ama/issues/10#issuecomment-1... [2] https://medium.com/@Rich_Harris/small-modules-it-s-not-quite-that-simple-3ca532d65de4 https://medium.com/@Rich_Harris/small-modules-it-s-not-quite...
  • AndrewStephens 9y ago
    There are many reasons why apps are so big, but I think you are right about a major part of the problem. Development environments these days make it very easy to add in third party libraries for very little effort. At a previous job we had a monolithic java server that ended up at over 350Mb of compiled code simply because each development team had imported whatever libraries they thought they needed. In some cases, 2 or 3 versions of the same library were included.
    • lsv1 9y ago
      This is why code coverage is so important as part of the general development cycle.
      • djKianoosh 9y ago
        Wait, code coverage? How does that address the number of libraries used or duplicate versions of the same library?
    • wooter 9y ago
      i don't see why any of this is a problem
    • slaymaker1907 9y ago
      How did they get multiple versions of the same library to work together? Java loads things via the class path so I would have thought that would cause some sort of error.
      • swsieber 9y ago
        Some libraries change the namespace (package names) between major versions, specifically to allow transition from one to another in a gradual manor, or to start following namespace guidelines better.
      • tedunangst 9y ago
        jarjar?
  • potatolicious 9y ago
    I think this is likely the biggest culprit. Another user pointed out an analysis of the Facebook app: http://blog.timac.org/?p=1707 http://blog.timac.org/?p=1707 Most of it is actual code - not assets. For some types of apps (see: games) assets do take up a significant portion of total size, but for most everyday apps bloat by code over-inclusion is likely a bigger problem than asset-bloat. I wonder if it's possible to get major open-source libs to move towards more fine-grained build targets and internal dependency management, so that devs don't pull in a gigantic binary when they're only using a small slice of the functionality.
    • Swizec 9y ago
      > I wonder if it's possible to get major open-source libs to move towards more fine-grained build targets and internal dependency management, so that devs don't pull in a gigantic binary when they're only using a small slice of the functionality. Fwiw, this is already a trend in JavaScript land. Libraries are moving towards many small packages so you can import only the parts you need either manually or using a tool like Webpack to throw away what isn't used.
      • cyphar 9y ago
        I don't think that's the right solution to the problem. Nobody wants libleftpad.so, and as someone who works on a distribution the very concept is horrific (making distribution packages for every three-line package does not make anyone happy). What I think GP was arguing for is that you have libstring.so which you can strip down to just having leftpad or w/e with Kconfig or similar configurations (preferably at link time not build time -- otherwise you still have the same problem).
        • RodericDay 9y ago
          I interpreted the comment in exactly the same way, and once again I'm baffled to see a perfectly polite and correct comment downvoted and greyed out.
          • 9y ago
  • ryandrake 9y ago
    This would be true if the entirety of the library was used by the application. Wouldn't the linker throw out anything unused?
    • teamhappy 9y ago
      > Wouldn't the linker throw out anything unused? There (usually) is no dead code elimination for libraries.
    • mikeash 9y ago
      Depends on the language. If you're using something like C++, you can probably remove a lot of dead code. But with more dynamic languages, like Objective-C or Ruby or JavaScript, it becomes difficult or impossible to prove that a given chunk of code is never used. In Objective-C (which I'm most familiar with), the linker will keep all Objective-C classes and methods because it has no idea if you might be looking things up by name at runtime and invoking them that way. Even if you can throw away dead code, you can run into bloat because a library has a massive set of foundational APIs that the rest is built on, and using one little feature of the library ends up bringing in half the library code because it's used everywhere.
      • Retric 9y ago
        You could use static analysis to find out if anything calls objects dynamically by name at run time which could be a useful optimization. To be safe this would be one bool value for the entire code base, but with some care it would help. A larger issue is size is simply not considered a significant issue.
        • mikeash 9y ago
          With Objective-C, that will always come back "true." Even if you never use dynamic lookups, Apple's frameworks do. I suspect the same is true of other languages.
          • Retric 9y ago
  • joshvm 9y ago
    This happens a lot, especially with dynamic linking. I have a project that uses Qt, OpenCV, CUDA, PCL and VTK - fairly standard stack for 3D imaging and visualisation. Since you normally need to bundle dependencies to account for different versions, this adds up quite fast. Qt adds 20MB for OpenGL, 15MB for the VC++ redist, about 30MB for other core libraries. Some stuff in OpenCV requires Nvidia's performance primitives (NPP), so in goes nppi64_80.dll - that's 100MB alone. opencv_cudaarithm310.dll is 70MB and even opencv_imgproc310.dll weighs in at 30MB. And on and on. So yes, one little call to cv::cuda::remap adds in a boatload of dependencies when all the algorithm is doing is using a lookup table to sample an image. https://github.com/opencv/opencv/blob/master/modules/cudawarping/src/cuda/remap.cu https://github.com/opencv/opencv/blob/master/modules/cudawar...
    • maxxxxx 9y ago
      It's pretty ironic that dynamic linking's main motivation was to eliminate duplicate code and reduce code size. Deploy a DLL once, every software uses it and doesn't have to include it. Now it has turned out exactly the opposite way. If we went back to linking object code together we would get smaller sizes. Instead we have to include huge DLLs.
      • dfox 9y ago
        IIRC Windows has pretty involved infrastructure for deduplicating same executable images both in memory and on disk.
      • verytrivial 9y ago
        > dynamic linking's main motivation was to eliminate duplicate code and reduce code size I'm pretty sure the main motivation is/was to allow patching of dependencies independently from applications. Very popular shared libraries might save space overall, but that is a secondary effect.
        • maxxxxx 9y ago
          When I started out the main motivation was to get install sizes down. At least on Windows patching never really worked and ended in "DLL hell".
  • tylersmith 9y ago
    I think this is exactly it. If you start vendoring dependencies you begin to notice how much of other's people's code your software is really using.
  • dingo_bat 9y ago
    As I understand it, when you link your binary, the linker will only include the used parts of the libraries. Linking to a 10M library but calling only one function will not increase your binary size by 10M, probably just a few bytes.
• vec 9y ago
  A huge chunk of that probably goes to, well, videos or hi-res photographs to be used in building the UI. Hi-res splash screens plus a bunch of hero images plus 2+ prerendered sizes of each display element for different pixel densities plus a dozen 30 second tutorial videos can easily add up to a couple hundred megs of assets alone.
  • silassales 9y ago
    I think a large factor is the ever increasing need to have different display sizes for different pixel densities, developers essentially have to create and package a couple versions of the same application into one package and that is alot of waste.
    • masklinn 9y ago
      If the application is properly packaged, the appstore should be able to generate configuration-specific variants which only include the assets relevant to your device class (https://developer.apple.com/library/content/documentation/IDEs/Conceptual/AppDistributionGuide/AppThinning/AppThinning.html https://developer.apple.com/library/content/documentation/ID...)
      • silassales 9y ago
        This makes sense, a much better way to do things.
    • potatolicious 9y ago
      At least in the context of this article (iOS app bloat) this is no longer the case - developers upload all assets for all pixel densities, but Apple repackages for each specific device, so that each device only gets one set of assets. Same goes for binaries for multiple architectures - each device only gets the binary for its specific CPU architecture. Also to the GP's point - Apple also now no longer supports splash screen images, so that element of bloat is no longer a factor (though some legacy apps have retained them pointlessly). I think for non-game apps assets are not the primary driver of bloat.
• titzer 9y ago
  Too many programmers, too many templates and frameworks, poor factoring, too much code reuse, bad build configurations, and lack of LTO.
• joeblau 9y ago
  Lots of things. Most companies now have huge development teams and that means that you're going to get a lot of duplicate and triplicate media assets. Duplicate and triplicate static libraries and frameworks (which Apple is trying to address with it's new dyld improvements[1]). As you also can't forget all of the analytics and A/B testing frameworks that are put in place. Each one of these apps can actually probably run 100^2 permutations of the app. [1] - https://developer.apple.com/videos/play/wwdc2017/413/ https://developer.apple.com/videos/play/wwdc2017/413/
• sliverstorm 9y ago
  When I was developing a few pet apps, the reason was pretty simple. I was pulling in entire libraries for one or two functions. They were good libraries and good functions. I think multiple examples were Google-provided app development libraries that you roll into your app. Like, appcompat. Nobody can get by without appcompat anymore it seems, but nobody needs all of its functionality either. Anyway, probably millions of lines of code, 99.9% of which I didn't call. There is a tool for stripping code that will not be called, and I reduced my apk's from ~20MB to ~1MB, although I wound up turning it off in the end because it was not trivial to enable correctly. (I was linking 3rd party binary libraries into my app which complicated things)
• jayd16 9y ago
  >Surely the 275 MB isn't all useful data (I wonder what compression ratios you get on 'apps'), and it should be possible to cut it down to a few MB. This is pretty tone def. The bulk of apps are audio and ui assets and the compression rates on those are quite good. That said, the compression rate for iOS apps is horrible as Apple decides to encrypt and then compress the binaries, completely blowing apart the compression ratio of duplicate data.
  • richardwhiuk 9y ago
    Compress then encrypt is continually a source of vulnerabilities.
    • sebcat 9y ago
      For protocols yes, but you need to look at the full picture. e.g., what would the oracle be in this case? Encrypt then compress makes no sense at all. Compress then encrypt, or don't compress at all.