3 ms·
Where do they benchmark code candidates? This is hard, noisy and expensive (I see example reward for code size). I am not an expert, but bayesian learning may
by algo_trader 6y ago
Where do they benchmark code candidates? This is hard, noisy and expensive (I see example reward for code size).
I am not an expert, but bayesian learning maybe more appropriate for such an expensive-sampling environment?
- sstangl 6y agoThey are optimizing only for code size, so they don't need to run the code: they only count bytes emitted. This works by changing around the order / interleaving of various LLVM optimization phases, so the learning process does not require knowledge of program timing or correctness.
- liuliu 6y agoI heard when they started to do this, LLVM folks were freaked out because they worried changing orders may affect correctness. But it seems to work so far :)