4 ms·

Is it possible to hand the Virgil compiler a mix of source and binaries, or does the compiler demand that your whole program exist in source form? I don't doubt

by msla 5y ago

Is it possible to hand the Virgil compiler a mix of source and binaries, or does the compiler demand that your whole program exist in source form? I don't doubt that modern compilers running on modern computers can hold the IR of nontrivial programs in RAM at once, but that technical aspect isn't the only reason we build programs out of preexisting binaries: If you want your program to use the tuned algorithms in LAPACK, they're written in Fortran 90. Other libraries are written in tuned assembly language. And even that's ignoring the fact that, sometimes, source isn't available for legal or financial reasons. https://en.wikipedia.org/wiki/LAPACK https://en.wikipedia.org/wiki/LAPACK

titzer 5y ago
> Is it possible to hand the Virgil compiler a mix of source and binaries Not currently, no. I've thought about trying a completely transparent behind-the-scenes incremental compilation scheme, basically hiding customly-formatted object-file equivalents somewhere on disk (and/or in memory), checking source file modification times, and stitching them together like a linker would, but I did the cost/benefit analysis and since I am generally working on programs < 50KLOC in size, it's not a problem I have yet. (Relatedly, in my bug/debug/rebug cycle, aka development, I often run my programs in the compiler's built-in interpreter, which only parses and verifies the code, and then interprets IR directly, saving 90% of compilation time). As for linking with other languages, I think it's considerably easier to have the Virgil compiler generate a ".o" [1] that is relocatable (i.e. linkable) and describe the imports/exports of a Virgil module in the Virgil language [2]. Then this ".o" could be linked in with the existing C/C++ ecosystem, rather than me trying to replicate linking in my compiler. The Virgil compiler really likes being able to nail down static (virtual) addresses; I suppose it can keep doing most of that if it doesn't have to link with too many other object files that constraint its placement choices. [1] The Virgil compiler can generate either ELF32, ELF64, or MachO binaries, so it's incremental work to set a few flags on sections and work out how to emit relocation entries. [2] I have a prototype of this for the Wasm target, so that Virgil programs can import unknown functions from modules, and the compiler generates a Wasm module with appropriate imports.

by msla 5y ago

titzer 5y ago

> Is it possible to hand the Virgil compiler a mix of source and binaries Not currently, no. I've thought about trying a completely transparent behind-the-scenes incremental compilation scheme, basically hiding customly-formatted object-file equivalents somewhere on disk (and/or in memory), checking source file modification times, and stitching them together like a linker would, but I did the cost/benefit analysis and since I am generally working on programs < 50KLOC in size, it's not a problem I have yet. (Relatedly, in my bug/debug/rebug cycle, aka development, I often run my programs in the compiler's built-in interpreter, which only parses and verifies the code, and then interprets IR directly, saving 90% of compilation time). As for linking with other languages, I think it's considerably easier to have the Virgil compiler generate a ".o" [1] that is relocatable (i.e. linkable) and describe the imports/exports of a Virgil module in the Virgil language [2]. Then this ".o" could be linked in with the existing C/C++ ecosystem, rather than me trying to replicate linking in my compiler. The Virgil compiler really likes being able to nail down static (virtual) addresses; I suppose it can keep doing most of that if it doesn't have to link with too many other object files that constraint its placement choices. [1] The Virgil compiler can generate either ELF32, ELF64, or MachO binaries, so it's incremental work to set a few flags on sections and work out how to emit relocation entries. [2] I have a prototype of this for the Wasm target, so that Virgil programs can import unknown functions from modules, and the compiler generates a Wasm module with appropriate imports.