4 ms·

Can you expand on this a bit? I use upx at work to ship binaries. Are you saying these binaries have different memory usage upx’d than they do otherwise?

by cced 3y ago

themerone 3y ago
Every instance of a program will use an amount of ram equal to the uncompressed size. If the application is uncompressed, the uncompressed executable will be loaded into ram 1 time and be reused by every instance of the application.
- ashishb 3y ago
  Any source that you can cite for this? I'm not saying you are wrong. I'm just curious to see more proof of this.
  - themerone 3y ago
    https://stackoverflow.com/questions/9219244/why-does-my-application-use-more-memory-after-i-compress-it-with-upx-and-what-c https://stackoverflow.com/questions/9219244/why-does-my-appl...
    - natmaka 3y ago
      On Linux isn't KSM able to compensate for this?
      - vardump 3y ago
        KSM has some security implications, possibility for side channel attacks.
    - ashishb 3y ago
      Wow. Thanks.
- mikepurvis 3y ago
  Also impacts startup time. Really it's only appropriate for situations like games where you're very confident there will be just one instance of it, and it'll be long-running. And even then, it's of dubious value when game install footprints are overwhelmingly dominated by assets rather than executable code.
- foobiekr 3y ago
  Containers really bust this model.
vardump 3y ago
Normally operating system simply maps binaries, executables and loadable libraries (.dylib, .so, .dll, etc.) into memory. The cost is approximately same whether you do this once or 1000 times. The code is executed from the mapped area as-is. However, when a binary is compressed, this cannot work, because in the file the binary is represented as a compressed data. The only way you can work around is to allocate some memory, decompress the binary there, map the region as executable and run it from there. This results a non-shareable copy of the data for each running instance. A random link about this issue in practice: https://github.com/nushell/nushell/issues/4131 https://github.com/nushell/nushell/issues/4131
Karrot_Kream 3y ago
I'm curious, was the practice of using upx there before you got there? We generally A/B test changes like this pretty thoroughly by running load tests against our traffic and looking at things like CPU and Memory pressure in our deploys.

Can you expand on this a bit? I use upx at work to ship binaries. Are you saying these binaries have different memory usage upx’d than they do otherwise?

by cced 3y ago

Can you expand on this a bit? I use upx at work to ship binaries. Are you saying these binaries have different memory usage upx’d than they do otherwise?

themerone 3y ago

Every instance of a program will use an amount of ram equal to the uncompressed size. If the application is uncompressed, the uncompressed executable will be loaded into ram 1 time and be reused by every instance of the application.

ashishb 3y ago
Any source that you can cite for this? I'm not saying you are wrong. I'm just curious to see more proof of this.
- themerone 3y ago
  https://stackoverflow.com/questions/9219244/why-does-my-application-use-more-memory-after-i-compress-it-with-upx-and-what-c https://stackoverflow.com/questions/9219244/why-does-my-appl...
  - natmaka 3y ago
    On Linux isn't KSM able to compensate for this?
    - vardump 3y ago
      KSM has some security implications, possibility for side channel attacks.
  - ashishb 3y ago
    Wow. Thanks.
mikepurvis 3y ago
Also impacts startup time. Really it's only appropriate for situations like games where you're very confident there will be just one instance of it, and it'll be long-running. And even then, it's of dubious value when game install footprints are overwhelmingly dominated by assets rather than executable code.
foobiekr 3y ago
Containers really bust this model.

vardump 3y ago

Normally operating system simply maps binaries, executables and loadable libraries (.dylib, .so, .dll, etc.) into memory. The cost is approximately same whether you do this once or 1000 times. The code is executed from the mapped area as-is. However, when a binary is compressed, this cannot work, because in the file the binary is represented as a compressed data. The only way you can work around is to allocate some memory, decompress the binary there, map the region as executable and run it from there. This results a non-shareable copy of the data for each running instance. A random link about this issue in practice: https://github.com/nushell/nushell/issues/4131 https://github.com/nushell/nushell/issues/4131

Karrot_Kream 3y ago

I'm curious, was the practice of using upx there before you got there? We generally A/B test changes like this pretty thoroughly by running load tests against our traffic and looking at things like CPU and Memory pressure in our deploys.