6 ms·
Compile enough traces with the accompanying optimizations and you have solid training data for an LLM that can suggest optimizations based on JIT traces.
by aantix 2y ago
Compile enough traces with the accompanying optimizations and you have solid training data for an LLM that can suggest optimizations based on JIT traces.
- ainoobler 2y agoHow would you verify semantic correctness of the optimizations?
- aantix 2y agoI think I envisioned traces being extracted from a series of open source projects and their automated test suites. Run the test suite, identify optimizations. One by one, make the the optimization change to the implementation as suggested by the LLM. Instrument the changed methods on the second test run and see if runtime performance has changed. Verify that the test still passes.
- ainoobler 2y agoI meant how do you make sure the optimization suggested by the AI is actually valid. If you're using AI to modify bytecode for faster execution then you have to make sure the optimized and unoptimized code are semantically equivalent. Neural networks can't do logic so how would you know the suggestions were not bogus?
- weebull 2y agoYou've asked the right question, and for those that think validation is as simpLe as "run it and see if it gets the right result", good start but instruction ordering can be critical around multi thread aware data structures. Taking a fence out, or an atomic operation might give a big performance gain. Trouble is the structure may now go wrong 1% of the time.
- aantix 2y agoA valid accompanying test would ensure this? You’d be extracting optimization candidates by running the test suite. You re-run the test suite after changes to ensure they still pass.
- ainoobler 2y agoJIT optimizers operate at runtime, there are no test suites to verify before/after. It's happening live as the code is running so if you use AI then you won't know if the optimization is actually valid or not. This is why the article is using Z3 instead of neural networks. Z3 can validate semantic equivalence, neural networks can't.
- fwip 2y agoYes, but this Z3 analysis is not done at runtime. It's done offline, based on JIT traces. A neural network could, in principal, suggest optimizations in the same way, which an expert would then review for possible inclusion into the Pypy JIT.
- ainoobler 2y agoYou'd still have to write a proof for verifying semantic equivalence before implementing the optimization so I don't see what the neural network gains you here unless it is actually supplying the proof of correctness along with the optimization.
- screcth 2y agoThe idea is that the LLM would provide "intuition" to guide the optimizer to find better optimizations, but a formal proof would be necessary to ensure that those optimizations are actually valid.
- fwip 2y agoI might be incorrect, but I don't believe that most compiler optimizations have formal proofs written out before implementation. Does Pypy do this?
- wolf550e 2y agoregehr et al use alive2 which uses z3
- Twirrim 2y agoBut.. But.. But.... This is HN. You must use AI / LLMs for everything! /s
- cfbolztereick 2y agoAll the minimized inefficiencies that are found by my script are already optimizations. They just happen to be rather specific patterns, so they need to be suitably generalized. There's another Regehr etal paper about how to do that automatically: https://dl.acm.org/doi/10.1145/3649837 https://dl.acm.org/doi/10.1145/3649837 (llm's aren't involved, it's all based on z3) I don't plan on implementing something like this for now, I'd rather take the inefficiencies and manually extract optimizations out of them and implement them in PyPy's jit.
- algo_trader 2y agoYou are not wrong AlphaZero-type systems were unable to significantly improve over near-optimal solvers such as z3.
- Zacharias030 2y agoCould you point me to some references to learn about his?
- bastawhiz 2y agoIf you already know what the optimizations are, you don't need AI to optimize them. Just write optimizations in the compiler. That's literally one of the biggest jobs of the compiler already.