7 ms·

Keep in mind that UPX needs to decompress the whole executable to memory and thus cannot make use of OS-level memory sharing. Normally, when you load the same e

Keep in mind that UPX needs to decompress the whole executable to memory and thus cannot make use of OS-level memory sharing. Normally, when you load the same executable twice, its base image will only consume memory once. But it with UPX, you cannot make use of that. So in the end, it may use more resources than you thought!

paulfurtado 5y ago
Definitely important to note, I wonder if upx could solve this via execing an memfd_create in some mode. Also wish upx came with a trivial extraction tool so you could use it for distribution but not runtime.
- prova1 5y ago
  ok.
- mikepurvis 5y ago
  Aren’t your container layers compressed in transit anyway?
  - paulfurtado 5y ago
    UPX does significantly more compression on binaries than gzip can on the layers. It's not just simple compression, it makes a lot of modifications to the binary to make it more compressible.
    - mikepurvis 5y ago
      Perhaps this is what you were asking for in the GP, but it would be interesting then if UPX could apply those transformations without actually doing the final compression step itself. Perhaps the win isn't as great without the ability to smuggle specific hints to the compressor, but it would be interesting to see. On the other hand, if it could be demonstrated that modest increases to binary size correlated to significantly improved compressability, then maybe that's something that should just be considered by the compilers themselves.
dividuum 5y ago
I was initially a bit skeptical if the page cache would work with all layering docker does. But OverlayFS, perhaps unsurprisingly, does indeed support that: [1] "OverlayFS supports page cache sharing. Multiple containers accessing the same file share a single page cache entry for that file. This makes the overlay and overlay2 drivers efficient with memory and a good option for high-density use cases such as PaaS." TIL [1] https://docs.docker.com/storage/storagedriver/overlayfs-driver/ https://docs.docker.com/storage/storagedriver/overlayfs-driv...
- Hackbraten 5y ago
  UPX creates anonymous pages that contain the decompressed payload, right? Those aren’t file-backed pages so how would the filesystem even know those pages exist?
  - dividuum 5y ago
    That’s correct and I guess only page deduplication might help in that case. What I meant in my post was just to state that docker can use the page cache despite the massive indirection caused by overlays. I was wondering if that was even possible and it indeed is.
synergy20 5y ago
that's true, plus on low-end devices upx will take forever to decompress and is essentially unusable.
- paulfurtado 5y ago
  Is UPX actually that much slower than extracting a gzipped docker layer? I didn't realize UPX decompression was much slower than other compression algorithms
amscanne 5y ago
The base would consume “up to once”. You will only load pages touched, and they are all reclaimable under memory pressure. Memory used by UPX for decompression is guaranteed to be used, and is not reclaimable! So in fact, it’s not might use more resources, it’s guaranteed to use more resources. The only benefit of UPX is saving disk space, which is unlikely to be super useful to most people these days.
- paulfurtado 5y ago
  The other use case is to distribute images more quickly over the network and improve image pull times, at the expense of slightly more memory use. Similar to how saving disk is less useful to most people these days, RAM on servers in big docker/kubernetes clusters is also likely not much of a concern for binaries that are well under 100MB if you're not running a ton of copies on each server. In particular, it seems like this tradeoff is most useful for static go binaries, which don't benefit from shared system libraries and aren't likely to have many copies per server, but of course that definitely depends on the use case and I definitely do not recommend blindly using UPX without considering the tradeoffs. It would be an amazing integration though if it were possible to integrate UPX with docker's layer compression such that the binary is extracted to the FS before running. But given how it all works, that's pretty challenging to implement and the network is only getting faster...
  - shawnz 5y ago
    Are you sure UPX actually beats the standard layer compression which is already provided? After applying UPX, the result should be nearly uncompressable, so if UPX performs worse than layer compression then it might be better for network usage to not use it at all. In this scenario where you are unpacking it in advance, that eliminates the main benefit of UPX which is that it is self-extracting.
  - deleted 5y ago
    [deleted]

paulfurtado 5y ago

Definitely important to note, I wonder if upx could solve this via execing an memfd_create in some mode. Also wish upx came with a trivial extraction tool so you could use it for distribution but not runtime.

prova1 5y ago
ok.
mikepurvis 5y ago
Aren’t your container layers compressed in transit anyway?
- paulfurtado 5y ago
  UPX does significantly more compression on binaries than gzip can on the layers. It's not just simple compression, it makes a lot of modifications to the binary to make it more compressible.
  - mikepurvis 5y ago
    Perhaps this is what you were asking for in the GP, but it would be interesting then if UPX could apply those transformations without actually doing the final compression step itself. Perhaps the win isn't as great without the ability to smuggle specific hints to the compressor, but it would be interesting to see. On the other hand, if it could be demonstrated that modest increases to binary size correlated to significantly improved compressability, then maybe that's something that should just be considered by the compilers themselves.

dividuum 5y ago

I was initially a bit skeptical if the page cache would work with all layering docker does. But OverlayFS, perhaps unsurprisingly, does indeed support that: [1] "OverlayFS supports page cache sharing. Multiple containers accessing the same file share a single page cache entry for that file. This makes the overlay and overlay2 drivers efficient with memory and a good option for high-density use cases such as PaaS." TIL [1] https://docs.docker.com/storage/storagedriver/overlayfs-driver/ https://docs.docker.com/storage/storagedriver/overlayfs-driv...

Hackbraten 5y ago
UPX creates anonymous pages that contain the decompressed payload, right? Those aren’t file-backed pages so how would the filesystem even know those pages exist?
- dividuum 5y ago
  That’s correct and I guess only page deduplication might help in that case. What I meant in my post was just to state that docker can use the page cache despite the massive indirection caused by overlays. I was wondering if that was even possible and it indeed is.

synergy20 5y ago

that's true, plus on low-end devices upx will take forever to decompress and is essentially unusable.

paulfurtado 5y ago
Is UPX actually that much slower than extracting a gzipped docker layer? I didn't realize UPX decompression was much slower than other compression algorithms

amscanne 5y ago

The base would consume “up to once”. You will only load pages touched, and they are all reclaimable under memory pressure. Memory used by UPX for decompression is guaranteed to be used, and is not reclaimable! So in fact, it’s not might use more resources, it’s guaranteed to use more resources. The only benefit of UPX is saving disk space, which is unlikely to be super useful to most people these days.

paulfurtado 5y ago
The other use case is to distribute images more quickly over the network and improve image pull times, at the expense of slightly more memory use. Similar to how saving disk is less useful to most people these days, RAM on servers in big docker/kubernetes clusters is also likely not much of a concern for binaries that are well under 100MB if you're not running a ton of copies on each server. In particular, it seems like this tradeoff is most useful for static go binaries, which don't benefit from shared system libraries and aren't likely to have many copies per server, but of course that definitely depends on the use case and I definitely do not recommend blindly using UPX without considering the tradeoffs. It would be an amazing integration though if it were possible to integrate UPX with docker's layer compression such that the binary is extracted to the FS before running. But given how it all works, that's pretty challenging to implement and the network is only getting faster...
- shawnz 5y ago
  Are you sure UPX actually beats the standard layer compression which is already provided? After applying UPX, the result should be nearly uncompressable, so if UPX performs worse than layer compression then it might be better for network usage to not use it at all. In this scenario where you are unpacking it in advance, that eliminates the main benefit of UPX which is that it is self-extracting.
- deleted 5y ago
  [deleted]