3 ms·

I know it is irrational - as compilers already perform optimizations that are non intuitive - but my lizard brain really doesn’t like the idea of ML possibly ch

by achr2 3y ago

I know it is irrational - as compilers already perform optimizations that are non intuitive - but my lizard brain really doesn’t like the idea of ML possibly changing semantics at a layer that is unobservable.

fulafel 3y ago
WE "just" need to get the models outputting machine checkable proofs that semantics are preserved. I wonder what like this are going on in the math scene.
- KirillPanov 3y ago
  You forgot valid machine checkable proofs. That's quite a tall order. Godel and all that.
bilekas 3y ago
Yeah, funny enough I was reading the “coding machines” short story yesterday after someone linked it and I had a bit of a giggle imagining the sentient compiler here.
UncleMeat 3y ago
This sort of work isn't going "i dunno, emit some new instructions based on a model and hope it is correct." ML techniques fall into one of two categories: 1. Decisions that have no semantic change in the program but affect performance. This is things like code layout or register allocation that will be more cache friendly. This is what I expect to show up in industrial optimizing compilers more and more over the next bunch of years. 2. Decisions where you can prove that the new code sequence is semantically identical to the original code sequence via some formal technique. This is basically the same as various superoptimization techniques but with a new search strategy and therefore no new concerns about correctness. I'd expect this to remain niche and only used offline to generate particularly optimized sequences in extremely hot and small code paths.
- gnramires 3y ago
  I think some functions with exhaustively enumerable could be good candidates as well (I guess somewhat rare though?). For example an Int->(...) or Float->(...) function (Doubles are not exhaustively enumerable though, I guess, since 2^64 ~= 10^19, 10,000-Peta items). If the programmer can give an error threshold, e.g. in bits of precision (maximum relative error), that should work quite well too.
- KirillPanov 3y ago
  > Decisions that have no semantic change Yeah the problem is that there are not a whole lot of those in PL.
  - UncleMeat 3y ago
    There are a huge number of these things in compilers. I listed two: code layout and register allocation. And you've got plenty of other things like inlining, outlining, and unrolling decisions.

by achr2 3y ago

fulafel 3y ago

WE "just" need to get the models outputting machine checkable proofs that semantics are preserved. I wonder what like this are going on in the math scene.

KirillPanov 3y ago
You forgot valid machine checkable proofs. That's quite a tall order. Godel and all that.

bilekas 3y ago

Yeah, funny enough I was reading the “coding machines” short story yesterday after someone linked it and I had a bit of a giggle imagining the sentient compiler here.

UncleMeat 3y ago

This sort of work isn't going "i dunno, emit some new instructions based on a model and hope it is correct." ML techniques fall into one of two categories: 1. Decisions that have no semantic change in the program but affect performance. This is things like code layout or register allocation that will be more cache friendly. This is what I expect to show up in industrial optimizing compilers more and more over the next bunch of years. 2. Decisions where you can prove that the new code sequence is semantically identical to the original code sequence via some formal technique. This is basically the same as various superoptimization techniques but with a new search strategy and therefore no new concerns about correctness. I'd expect this to remain niche and only used offline to generate particularly optimized sequences in extremely hot and small code paths.

gnramires 3y ago
I think some functions with exhaustively enumerable could be good candidates as well (I guess somewhat rare though?). For example an Int->(...) or Float->(...) function (Doubles are not exhaustively enumerable though, I guess, since 2^64 ~= 10^19, 10,000-Peta items). If the programmer can give an error threshold, e.g. in bits of precision (maximum relative error), that should work quite well too.
KirillPanov 3y ago
> Decisions that have no semantic change Yeah the problem is that there are not a whole lot of those in PL.
- UncleMeat 3y ago
  There are a huge number of these things in compilers. I listed two: code layout and register allocation. And you've got plenty of other things like inlining, outlining, and unrolling decisions.