3 ms·

How easy would it be to have an entire distro (re)built with BOLT? Say for example Gentoo?

fishgoesblub 2y ago
It would be difficult as every package/program would need a step to generate the profile data by executing and running the program like the user would.
- metadat 2y ago
  Is it theoretically possible to perform the profile generation+apply steps dynamically at runtime?
  - cryptonector 2y ago
    It would be hard to trust the result.
  - tjalfi 2y ago
    I wouldn't want to support it, but similar things have been done before. Alexia Massalin's Synthesis[0] (pdf) operating system did JIT-like optimizations for system calls. Here's a LWN article[1] with a summary. Anyone who's interested in operating systems should read this thesis. HP's Dynamo[2] runtime optimizer did JIT-like optimizations on PA-RISC binaries; it was released in 2000. DynamoRIO[3] is an open source descendant. Also, DEC had a similar tool for the Alpha, but I've forgotten the name. [0] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=1467a614509e49995a213a138f86c81b43f4cd56 https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d... [1] https://lwn.net/Articles/270081/ https://lwn.net/Articles/270081/ [2] https://dl.acm.org/doi/pdf/10.1145/349299.349303 https://dl.acm.org/doi/pdf/10.1145/349299.349303 [3] https://dynamorio.org/ https://dynamorio.org/
    - hikarikuen 2y ago
      This is getting way outside the traditional compiler model, but I believe the .NET JIT has been adding more support for this in the last couple versions. One aspect of it is covered at https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-8/#tiering-and-dynamic-pgo https://devblogs.microsoft.com/dotnet/performance-improvemen...
    - EvanAnderson 2y ago
      Nat Freidman developed "GNU Rope"[1] from 1998 which, if memory serves, was inspired by a tool that did the same thing in IRIX (cord, I believe). [1] http://lwn.net/1998/1029/als/rope.html http://lwn.net/1998/1029/als/rope.html
  - pgaddict 2y ago
    I believe some JIT systems already do PGO / might be extended to do what BOLT does.
genewitch 2y ago
based on what "fishgoesblub" commented, building - read: `emerge -e @world` - a gentoo system with profiling forced, and then using it in that "degraded" state for a while ought to be able to inform PGO, right? if there's a really good speedup from putting hot code together, the hottest code after moderate use should suffice to speed up things, and this could continually be improved. I'm also certain that if there were a way to anonymously share profiling data upstream (or to the maintainers), that would decrease the "degradation" from the first step, above. I am 100% spitballing here. I'm a dedicated gentoo sysadmin, but i know only a small bit about optimization of the sort being discussed here. So it is possible that every user would have to do the "unprofiled profiler" build first, which, if one cares, is probably a net negative to the planet, unless the idea pans out, then it's a huge positive for the planet - man hours, electricity, wear/endurance on parts, etc.

It would be difficult as every package/program would need a step to generate the profile data by executing and running the program like the user would.

metadat 2y ago
Is it theoretically possible to perform the profile generation+apply steps dynamically at runtime?
- cryptonector 2y ago
  It would be hard to trust the result.
- tjalfi 2y ago
  I wouldn't want to support it, but similar things have been done before. Alexia Massalin's Synthesis[0] (pdf) operating system did JIT-like optimizations for system calls. Here's a LWN article[1] with a summary. Anyone who's interested in operating systems should read this thesis. HP's Dynamo[2] runtime optimizer did JIT-like optimizations on PA-RISC binaries; it was released in 2000. DynamoRIO[3] is an open source descendant. Also, DEC had a similar tool for the Alpha, but I've forgotten the name. [0] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=1467a614509e49995a213a138f86c81b43f4cd56 https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d... [1] https://lwn.net/Articles/270081/ https://lwn.net/Articles/270081/ [2] https://dl.acm.org/doi/pdf/10.1145/349299.349303 https://dl.acm.org/doi/pdf/10.1145/349299.349303 [3] https://dynamorio.org/ https://dynamorio.org/
  - hikarikuen 2y ago
    This is getting way outside the traditional compiler model, but I believe the .NET JIT has been adding more support for this in the last couple versions. One aspect of it is covered at https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-8/#tiering-and-dynamic-pgo https://devblogs.microsoft.com/dotnet/performance-improvemen...
  - EvanAnderson 2y ago
    Nat Freidman developed "GNU Rope"[1] from 1998 which, if memory serves, was inspired by a tool that did the same thing in IRIX (cord, I believe). [1] http://lwn.net/1998/1029/als/rope.html http://lwn.net/1998/1029/als/rope.html
- pgaddict 2y ago
  I believe some JIT systems already do PGO / might be extended to do what BOLT does.

genewitch 2y ago

based on what "fishgoesblub" commented, building - read: `emerge -e @world` - a gentoo system with profiling forced, and then using it in that "degraded" state for a while ought to be able to inform PGO, right? if there's a really good speedup from putting hot code together, the hottest code after moderate use should suffice to speed up things, and this could continually be improved. I'm also certain that if there were a way to anonymously share profiling data upstream (or to the maintainers), that would decrease the "degradation" from the first step, above. I am 100% spitballing here. I'm a dedicated gentoo sysadmin, but i know only a small bit about optimization of the sort being discussed here. So it is possible that every user would have to do the "unprofiled profiler" build first, which, if one cares, is probably a net negative to the planet, unless the idea pans out, then it's a huge positive for the planet - man hours, electricity, wear/endurance on parts, etc.