26 ms·

Not convinced making the code hard to debug is a very useful trade-off. What's the point of making the so smaller to begin with?

benj111 3y ago
How many people will actually be debugging though? 99.9% of users will never touch all this, in return they get faster downloads of the browser and faster startup. Yes, I suppose for pre release versions the calculation is different, but for a stable release there 'shouldn't' be any bugs.
- mgaunard 3y ago
  How would a smaller shared object lead to faster startup? The kernel isn't loading pages to memory unless they're actually used. You need some of that debug information to send backtraces back to the developers.
  - jcelerier 3y ago
    > How would a smaller shared object lead to faster startup? The kernel isn't loading pages to memory unless they're actually used. IDK on linux but on windows opening a 600mb exe (e.g. release with debug info) and a 15mb one (same but stripped) have a VERY OBVIOUS startup time difference. I assume the OS is scanning the whole binary through some antivirus for instance, and I'd assume the same to be true for DLLs used by the app, so all these kernel loading pages optimizations go through the trash.
    - mgaunard 3y ago
      That's ridiculous. It's frequent for a process to map large amounts of data from the disk. Requiring that this data be eagerly loaded and be present in memory at all times would be disastrous for performance, if even possible. Even Windows wouldn't do something so silly.
    - slaymaker1907 3y ago
      Are you using symbol files? I thought most debug info on Windows was put in those, not in the binary itself. We include some debug info with SQL Server, but not symbol files, so that we can debug crashes from production deployments.
      - dblohm7 3y ago
        +1 - they're probably using the wrong compiler flags.
    - fweimer 3y ago
      I'd expect this to be related to virus scanning. It's hard to use demand paging for that.
  - toast0 3y ago
    There's several possibilities here. The obvious one is a smaller .so may have fewer pages of interest. If each function you call takes half a page, and you call 1/4th of the functions, chances are good you need to load more than 1/4the pages, because sometimes you'll call both functions in a page you load and sometimes you won't. In other words: if the .so is smaller, you get better packing. The other explanation is prefetching. Lets say all functions take a full page, so packing into a page doesn't matter. When you call a function, which demand pages, chances are the kernel is also going to load adjacent pages; if all the functions you call are adjacent, prefetching saves you significant i/o time. Otoh, if your functions are sparse in the backing file, you will have to wait for i/o; which may be (marginally) delayed because of prefetching on earlier loads.
    - mgaunard 3y ago
      debug information is not interleaved with the code.
      - preseinger 3y ago
        where do debug symbols exist, if not "interleaved" with the code?
        erik_seaberg 3y ago
        To avoid always paging in debug info, it’s kept in separate sections of an ELF binary, or even in a separate file (not always shipped or deployed).
- mschuster91 3y ago
  > 99.9% of users will never touch all this Users won't touch it directly, but I deem it a given that Chrome and its derivatives upload crash logs to the mothership by default, and IIRC Firefox also offers that as an opt-in. Same for Android (Firebase Crashlytics) and iOS, and I think there's also ways to hook yourself into JS frontend code to transmit uncaught exceptions to a backend server.
bee_rider 3y ago
The biggest gain seems to come from removing unused zlib functionality which probably would have negligible impact on debugging (if anything it gives the person investigating a bug a list of which specific functions have been used in the library, which seems like it could be pretty helpful!) Although, thinking about it a bit more, are we sneaking up on the “why not use a static library” zone?
eschneider 3y ago
For this particular library, I'm not sure if it's worth it or not. But for some of us working in constrained environments, optimizing binaries for size can make the difference between a binary that'll fit on the box and one that won't, and there were some interesting techniques in the post.
secondcoming 3y ago
There are people who have to compile with -Osize
gpderetta 3y ago
You should be able to move the debug info to a separate file, to be installed on demand.
- preseinger 3y ago
  how? specifically?
  - gpderetta 3y ago
    for example: https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Fil...
    - preseinger 3y ago
      thanks! neat!
ormax3 3y ago
given the proliferation of electron-based desktop apps, which are known to be resource heavy, making various so libraries smaller is indeed a welcome improvement.

benj111 3y ago

How many people will actually be debugging though? 99.9% of users will never touch all this, in return they get faster downloads of the browser and faster startup. Yes, I suppose for pre release versions the calculation is different, but for a stable release there 'shouldn't' be any bugs.

mgaunard 3y ago
How would a smaller shared object lead to faster startup? The kernel isn't loading pages to memory unless they're actually used. You need some of that debug information to send backtraces back to the developers.
- jcelerier 3y ago
  > How would a smaller shared object lead to faster startup? The kernel isn't loading pages to memory unless they're actually used. IDK on linux but on windows opening a 600mb exe (e.g. release with debug info) and a 15mb one (same but stripped) have a VERY OBVIOUS startup time difference. I assume the OS is scanning the whole binary through some antivirus for instance, and I'd assume the same to be true for DLLs used by the app, so all these kernel loading pages optimizations go through the trash.
  - mgaunard 3y ago
    That's ridiculous. It's frequent for a process to map large amounts of data from the disk. Requiring that this data be eagerly loaded and be present in memory at all times would be disastrous for performance, if even possible. Even Windows wouldn't do something so silly.
  - slaymaker1907 3y ago
    Are you using symbol files? I thought most debug info on Windows was put in those, not in the binary itself. We include some debug info with SQL Server, but not symbol files, so that we can debug crashes from production deployments.
    - dblohm7 3y ago
      +1 - they're probably using the wrong compiler flags.
  - fweimer 3y ago
    I'd expect this to be related to virus scanning. It's hard to use demand paging for that.
- toast0 3y ago
  There's several possibilities here. The obvious one is a smaller .so may have fewer pages of interest. If each function you call takes half a page, and you call 1/4th of the functions, chances are good you need to load more than 1/4the pages, because sometimes you'll call both functions in a page you load and sometimes you won't. In other words: if the .so is smaller, you get better packing. The other explanation is prefetching. Lets say all functions take a full page, so packing into a page doesn't matter. When you call a function, which demand pages, chances are the kernel is also going to load adjacent pages; if all the functions you call are adjacent, prefetching saves you significant i/o time. Otoh, if your functions are sparse in the backing file, you will have to wait for i/o; which may be (marginally) delayed because of prefetching on earlier loads.
  - mgaunard 3y ago
    debug information is not interleaved with the code.
    - preseinger 3y ago
      where do debug symbols exist, if not "interleaved" with the code?
      - erik_seaberg 3y ago
        To avoid always paging in debug info, it’s kept in separate sections of an ELF binary, or even in a separate file (not always shipped or deployed).
mschuster91 3y ago
> 99.9% of users will never touch all this Users won't touch it directly, but I deem it a given that Chrome and its derivatives upload crash logs to the mothership by default, and IIRC Firefox also offers that as an opt-in. Same for Android (Firebase Crashlytics) and iOS, and I think there's also ways to hook yourself into JS frontend code to transmit uncaught exceptions to a backend server.

bee_rider 3y ago

The biggest gain seems to come from removing unused zlib functionality which probably would have negligible impact on debugging (if anything it gives the person investigating a bug a list of which specific functions have been used in the library, which seems like it could be pretty helpful!) Although, thinking about it a bit more, are we sneaking up on the “why not use a static library” zone?

eschneider 3y ago

For this particular library, I'm not sure if it's worth it or not. But for some of us working in constrained environments, optimizing binaries for size can make the difference between a binary that'll fit on the box and one that won't, and there were some interesting techniques in the post.

secondcoming 3y ago

There are people who have to compile with -Osize

gpderetta 3y ago

You should be able to move the debug info to a separate file, to be installed on demand.

preseinger 3y ago
how? specifically?
- gpderetta 3y ago
  for example: https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Fil...
  - preseinger 3y ago
    thanks! neat!

ormax3 3y ago

given the proliferation of electron-based desktop apps, which are known to be resource heavy, making various so libraries smaller is indeed a welcome improvement.