5 ms·

How big of a problem is dependency duplication on 1TB drives?

indrora 2y ago
The issue is memory. Every library in an app image has a unique memory space and so you have a bunch of copies of sometimes very large libraries sitting in memory rather than one copy from disk mmapped into memory and page duplicated all over the place.
- eklavya 2y ago
  Linux has had support for not doing duplicate pages for a long time now. I am forgetting the name of the feature but essentially this duplication is a solved problem.
  - e12e 2y ago
    For different minor versions and builds of libraries?
  - raverbashing 2y ago
    This is basic paging and CoW (Copy on Write) behaviour. I agree, it's mostly a non-issue
  - rcxdude 2y ago
    That's only the case if the libraries loaded are identical, It won't work with slightly different versions of the same library (unless the differences are small and only replacements, so the pages remain aligned between different versions), and that case is very unlikely to be solvable
    - eklavya 2y ago
      The parent comment doesn't talk about different versions and I wasn't either.
    - deleted 2y ago
      [deleted]
e12e 2y ago
In addition to memory, there's the ability to patch a libz bufferoverflow once, and be reasonably sure you don't have any stale vulnerable copies still in use.
- deleted 2y ago
  [deleted]
abenga 2y ago
Could be big, depending on how much room you give to /. All my Linux life, I have allocated about 50GB to the root partition and it's been adequate, leaving enough room for my data (on a 512GB drive). Now I install one flatpak and I start getting low disk space warnings.
- jcelerier 2y ago
  That's also a big reason why I prefer appimages. ossia score's AppImage is 100 megabytes: https://github.com/ossia/score/releases/tag/v3.2.0 https://github.com/ossia/score/releases/tag/v3.2.0 Inside, there's: - Qt 6 (core, widgets, gui, network, qml, qtquick, serial port, websockets and a few others) and all its dependencies excluding xcb (so freetype, harfbuzz, etc. which I build with fairly more recent versions than many distros provide) - ffmpeg 6 - libllvm & libclang - the faust compiler (https://faust.grame.fr https://faust.grame.fr) - many random protocol & hardware bindings / implementations and their dependencies (sdl) - portaudio - ysfx with Flatpak I'd be looking at telling my users to install a couple GB (which is not acceptable, I was already getting comments that "60 MB are too much" when it was 60 MB a few years ago).
- curt15 2y ago
  How do you manage software with large assets? A single game can take 10-100GB of storage.
  - abenga 2y ago
    My steam library is on another drive (I have multiple O(TBs) large spinning-rust drives on the desktop). The nvme is strictly the base system + apt packages + build tools + /home/.
- semi-extrinsic 2y ago
  I also shun the snap bullshit. But TBH I haven't divided my disks into more than one partition (except from /boot and EFI stuff) for many many years now.

The issue is memory. Every library in an app image has a unique memory space and so you have a bunch of copies of sometimes very large libraries sitting in memory rather than one copy from disk mmapped into memory and page duplicated all over the place.

eklavya 2y ago
Linux has had support for not doing duplicate pages for a long time now. I am forgetting the name of the feature but essentially this duplication is a solved problem.
- e12e 2y ago
  For different minor versions and builds of libraries?
- raverbashing 2y ago
  This is basic paging and CoW (Copy on Write) behaviour. I agree, it's mostly a non-issue
- rcxdude 2y ago
  That's only the case if the libraries loaded are identical, It won't work with slightly different versions of the same library (unless the differences are small and only replacements, so the pages remain aligned between different versions), and that case is very unlikely to be solvable
  - eklavya 2y ago
    The parent comment doesn't talk about different versions and I wasn't either.
  - deleted 2y ago
    [deleted]

e12e 2y ago

In addition to memory, there's the ability to patch a libz bufferoverflow once, and be reasonably sure you don't have any stale vulnerable copies still in use.

deleted 2y ago
[deleted]

abenga 2y ago

Could be big, depending on how much room you give to /. All my Linux life, I have allocated about 50GB to the root partition and it's been adequate, leaving enough room for my data (on a 512GB drive). Now I install one flatpak and I start getting low disk space warnings.

jcelerier 2y ago
That's also a big reason why I prefer appimages. ossia score's AppImage is 100 megabytes: https://github.com/ossia/score/releases/tag/v3.2.0 https://github.com/ossia/score/releases/tag/v3.2.0 Inside, there's: - Qt 6 (core, widgets, gui, network, qml, qtquick, serial port, websockets and a few others) and all its dependencies excluding xcb (so freetype, harfbuzz, etc. which I build with fairly more recent versions than many distros provide) - ffmpeg 6 - libllvm & libclang - the faust compiler (https://faust.grame.fr https://faust.grame.fr) - many random protocol & hardware bindings / implementations and their dependencies (sdl) - portaudio - ysfx with Flatpak I'd be looking at telling my users to install a couple GB (which is not acceptable, I was already getting comments that "60 MB are too much" when it was 60 MB a few years ago).
curt15 2y ago
How do you manage software with large assets? A single game can take 10-100GB of storage.
- abenga 2y ago
  My steam library is on another drive (I have multiple O(TBs) large spinning-rust drives on the desktop). The nvme is strictly the base system + apt packages + build tools + /home/.
semi-extrinsic 2y ago
I also shun the snap bullshit. But TBH I haven't divided my disks into more than one partition (except from /boot and EFI stuff) for many many years now.