5 ms·
Yeah I like this technique too. To put a name on it, it's a form of metaprogramming that could be called "staged programming" or "multi-stage programming", and
by chubot 8y ago
Yeah I like this technique too. To put a name on it, it's a form of metaprogramming that could be called "staged programming" or "multi-stage programming", and there are a bunch nascent systems that do things like it:
https://scala-lms.github.io/ https://scala-lms.github.io/ -- Scala LMS is similar in that you replace type T in your algorithms with Ref<T>, and you get a code generator (compiler) rather than an interpreter.
This links to a CACM article by the authors of Scala LMS:
https://cs.stackexchange.com/questions/2869/what-are-staged-functions-conceptually https://cs.stackexchange.com/questions/2869/what-are-staged-...
https://andrewkelley.me/post/zig-programming-language-blurs-line-compile-time-run-time.html https://andrewkelley.me/post/zig-programming-language-blurs-... -- Zig has a code generator for printf using compile-time metaprogramming.
http://terralang.org/ http://terralang.org/ -- You can write arbitrary Lua to generate Terra code, sort of like he's writing Python to generate LLVM code. I think this is more general and doesn't necessarily work by retaining the structure of say solve_linear_system(). I'm not quite sure. But the problem domains they tackle are very similar to this (numerical code).
TensorFlow also has two stages of computation -- you define a graph of objects in imperative Python, using Python operator overloading in the same way that LLVMCode in this example does. And then later the separate TensorFlow language is evaluated in parallel / distributed on GPUs.
In one case you're defining an SSA graph of ops in LLVM; in the other case you're defining a graph of TensorFlow ops.
Note that the semantics of TensorFlow language are distinct from the semantics of Python, e.g. https://ai.google/research/pubs/pub46196 https://ai.google/research/pubs/pub46196
One interesting language in the same vein is Spiral, with "first-class staging" for machine learning:
https://github.com/mrakgr/The-Spiral-Language https://github.com/mrakgr/The-Spiral-Language
You can also do a lot of this with constexpr / templates in C++, although the syntax is horrible.
As you allude to, these staged programming systems can have TWO languages or one. In the case of Scala and Zig it's one language, but in the case of Lua/Terra it's two. Arguably in the case of C++, it's two languages and not one.
Another thing to search for is "partial evaluation". I still have to understand the difference more precisely, as alluded to in the link above.
https://en.wikipedia.org/wiki/Multi-stage_programming https://en.wikipedia.org/wiki/Multi-stage_programming
The Python/RPython language pair in PyPy is also related:
https://morepypy.blogspot.com/2018/09/the-first-15-years-of-pypy.html#why-did-we-abandon-partial-evaluation https://morepypy.blogspot.com/2018/09/the-first-15-years-of-...
- akhilcacharya 8y agoSpeaking of LMS, the Tensorflow for Swift project is doing graph program extraction from LLVM for the same purpose. Super neat stuff. https://github.com/tensorflow/swift/blob/master/docs/GraphProgramExtraction.md https://github.com/tensorflow/swift/blob/master/docs/GraphPr...
- chubot 8y agoYes I think it's interesting, here's me trying to understand it better: https://www.reddit.com/r/ProgrammingLanguages/comments/9y4ehk/2018_llvm_dev_meeting_hong_lattner_graph_program/ https://www.reddit.com/r/ProgrammingLanguages/comments/9y4eh... At first I was a little critical but I get it a little more now. Of course the real test is trying it, but I'm not working in that area at the moment, though I've used TF before. What I find interesting is the recurring theme of two languages -- in Swift's case they say you can use a "static subset". I think that means all the imperative control flow, and then static dispatch (as in templates), but NOT dynamic dispatch (as in OOP). So in this example you have Python/LLVM, then there's Lua/Terra, Python/RPython, Python/TensorFlow, or Swift/static Swift, etc. Except in the latter case I guess it's all mixed together and you don't explicitly have a graph building stage.