6 ms·
Research papers on ML in Compilers
- StackOverlord 3y agoRelated C2 wik page: https://wiki.c2.com/?SufficientlySmartCompiler https://wiki.c2.com/?SufficientlySmartCompiler
- hetzenmat 3y agoNote that this refers to ML as in machine learning and not the ML language family, as I hoped.
- deleted 3y ago[deleted]
- choeger 3y agoMe too :(.
- devsegal 3y agoWhat a cool language!
- Findecanor 3y agoMy mind still unabbreviates "ML" into "Machine Language" before anything else, only because it was the first of these three for me.
- brookst 3y agoThere are so many flavors of machine learning that we should probably just refer to whole field as XML.
- TeMPOraL 3y agoAt some point we might need to cross-reference (X-ref) language models. This may involve using a dedicated data interchange format - perhaps an extensible markup language for language model cross-referencing. Or, in short, LMX-XML.
- achr2 3y agoI know it is irrational - as compilers already perform optimizations that are non intuitive - but my lizard brain really doesn’t like the idea of ML possibly changing semantics at a layer that is unobservable.
- fulafel 3y agoWE "just" need to get the models outputting machine checkable proofs that semantics are preserved. I wonder what like this are going on in the math scene.
- KirillPanov 3y agoYou forgot valid machine checkable proofs. That's quite a tall order. Godel and all that.
- bilekas 3y agoYeah, funny enough I was reading the “coding machines” short story yesterday after someone linked it and I had a bit of a giggle imagining the sentient compiler here.
- UncleMeat 3y agoThis sort of work isn't going "i dunno, emit some new instructions based on a model and hope it is correct." ML techniques fall into one of two categories: 1. Decisions that have no semantic change in the program but affect performance. This is things like code layout or register allocation that will be more cache friendly. This is what I expect to show up in industrial optimizing compilers more and more over the next bunch of years. 2. Decisions where you can prove that the new code sequence is semantically identical to the original code sequence via some formal technique. This is basically the same as various superoptimization techniques but with a new search strategy and therefore no new concerns about correctness. I'd expect this to remain niche and only used offline to generate particularly optimized sequences in extremely hot and small code paths.
- gnramires 3y ago
- lapinot 3y agoI'd be more interested in the reverse, that is compiler and PL techniques applied to machine learning.
- anlunx 3y agoYou might be interested in this: https://github.com/merrymercy/awesome-tensor-compilers https://github.com/merrymercy/awesome-tensor-compilers
- sjkoelle 3y agoThere is a mathematical description of programming compilers I read once (think it was a survey paper) that I found very clear but I've never been able to find since. Basically defined a compiler as a type of function. Any good suggestions?
- noblethrasher 3y agoPerhaps this one from Graham Hutton? “A parser for things Is a function from strings To lists of pairs Of things and strings”
- sjkoelle 3y agohmmmm maybe something like page 184 here https://apps.dtic.mil/sti/pdfs/ADA327435.pdf https://apps.dtic.mil/sti/pdfs/ADA327435.pdf
- AugustoCAS 3y agoThanks for sharing this. I became a bit of a fan of Dr Hutton after I followed his Functional Programming in Haskell classes on Youtube.
- hoosieree 3y agoMy research (ML for binary function recognition) uses obfuscation and diverse compilation for data augmentation. Intuitively, obfuscation is a form of anti-optimization. My hope is that it's differentiable, so perhaps given enough knowledge of obfuscation, an ML model can make corresponding optimizations to "undo" them, then for more performance, apply those same un-obfuscations to code that hasn't already been obfuscated.
- carom 3y agoI think this is a great line of research as it may solve the reasoning portion of AGI. The difficulty is that you would want to prove equivalence. LLMs can output stuff but for programs to execute you need to output correct stuff.
- hoosieree 3y agoYou might appreciate this survey paper: https://www.usenix.org/conference/usenixsecurity22/presentation/marcelli https://www.usenix.org/conference/usenixsecurity22/presentat... Most of the work I have read in this field is focused on finding potential malware or stolen copyrighted code, so false positives are no big deal. But you don't want anything less than 100% valid code coming out of your compiler.
- dzign 3y agoIt's interesting to add the work about Deepmind AlphaDev to this list.