4 ms·
Avo: Better X86 Assembly Generation from Go
- pcwalton 8y agoI really wish Go (really, Plan 9) hadn't made their own assembly syntax. The fact that we have AT&T syntax to deal with instead of just using Intel's syntax is bad enough. I understand the rationale—that you can use the assembly syntax as a sort of IR by unifying the syntax of certain basic operations across different ISAs—but that's properly considered a low-level IR, not assembly. (It's also a pretty bad form of IR for doing any interesting optimizations, which is why Go uses a separate SSA form for that.)
- mmcloughlin 8y agoAuthor of avo here. While I did really enjoy working on this project, I still fully agree with you. Go (Plan 9) syntax is not sufficiently different to make it a powerful intermediate representation, hence SSA as you point out. In fact many of the differences are frustrating subtle changes in instruction mnemonics or operand order. This might be okay if there was a canonical instruction database that made all these differences from Intel and AT&T easy to discover. However these exceptions are really recorded in various parts of the golang.org/x/arch/x86 repo. Producing the instruction database for avo was quite a time-consuming process. See: https://github.com/mmcloughlin/avo/blob/master/internal/load/load.go https://github.com/mmcloughlin/avo/blob/master/internal/load... https://github.com/mmcloughlin/avo/issues/23 https://github.com/mmcloughlin/avo/issues/23 I've gained a lot more familiarity with the assembler and supporting infrastructure, so I'm hoping I may be able to contribute to this area of the core Go project.
- pcwalton 8y agoYeah, AT&T syntax has much of the same issues of poor documentation. In particular, operand order is easy to remember for 2-operand instructions: it's reversed in AT&T. But now with SSE and AVX we have 3- or 4-operand instructions, and the AT&T versions sometimes put the operands in (2, 1, 3), sometimes in (3, 2, 1), etc. There's no documentation for this aside from GCC as far as I can tell. :(
- mmcloughlin 8y agoYes! This tripped me up at one point. My comment says "Otherwise we could be in the bizarre special-case of 3-argument CMP instructions." https://github.com/mmcloughlin/avo/blob/0e253b3753beb409b3dd8028fc81f5fa4b96aaee/internal/opcodescsv/analysis.go#L84 https://github.com/mmcloughlin/avo/blob/0e253b3753beb409b3dd... It gets to the point where the only source of truth is the Go assembler itself. My test suite takes the avo instruction database and generates a massive assembly file with one line per instruction form, then passes this to `go tool asm`.
- haberman 8y agoEven just mapping between the mnemonics is non-trivial. You want to think that it's just a matter of removing size suffixes from AT&T, but it's sadly not that simple. "movslq" in AT&T maps not to "movsl" or even "movs" in Intel, but "movsxd". How is anybody supposed to figure that out?
- nickcw 8y agoI've done a reasonable amount of ARM, x86 and amd64 programming with the go assembler. It's quirks are annoying to start with, but if you use more than one assembly language you begin to see the advantages of a more consistent syntax between them. I suspect the advantages for the go maintainers of a uniformish assembler syntax are the real reason for it.
- pcwalton 8y agoI use ARM and x86-64 extensively and any minor inconvenience caused by different syntax for trivial operations is completely outweighed by the downside of being incompatible with the CPU vendor's manual. LLVM MachineInstr and Cranelift do this much better than Go does, by maintaining a clean separation between the low-level IR and the assembly language.
- ris 8y agoA similar concept for python: https://github.com/Maratyszcza/PeachPy https://github.com/Maratyszcza/PeachPy
- mmcloughlin 8y agoavo author here. Yes PeachPy was the inspiration for this, as well as asmjit. Both mentioned in the credits: https://github.com/mmcloughlin/avo#credits https://github.com/mmcloughlin/avo#credits PeachPy actually has Go output already, and Damian Gryski has used this to great effect: https://github.com/mmcloughlin/avo/issues/40 https://github.com/mmcloughlin/avo/issues/40 Unfortunately because PeachPy wasn't initially written with Go in mind there are some rough edges and Go features it simply doesn't support. avo hopes to fill this gap for Go programmers.
- ris 8y agoAh - peach -> avocado, I see that now.
- mmcloughlin 8y agoHaha yes! PeachPy was written at Georgia Tech and avo was written in California.
- e_d_e_v 8y agoWe actually had Marat come to the Go meetup in Atlanta when he was finishing up his PHD at GT. He had done some preliminary work on SIMD. It was a small meetup ( I think only ~6-7 in attendance) and went over many folks heads, since a lot of people were focused on breaking up Ruby monoliths into Go. SIMD would have been nice, but really we were still reveling in the benefits of getting the hell out of ruby for high-throughput networked services. https://docs.google.com/presentation/d/1MYg8PyhEf0oIvZ9YU2panNkVXsKt5UQBl_vGEaCeB1k/edit?usp=sharing https://docs.google.com/presentation/d/1MYg8PyhEf0oIvZ9YU2pa...
- 8y ago
- jsgoller1 8y agoReally dope - excited to see this project come out!
- nickcw 8y agoThis is a nice project :-) Any chance of ARM/ARM64 support? It would be super useful if you could generate the code to memory then execute it directly.
- bepvte 8y agoI believe this assembly is golangs assembly, which is ISA neutral
- mmcloughlin 8y agoOnly a small number of instructions in Go assembly are architecture neutral. avo currently implements the x86 subset of Go assembler. It's definitely possible to add support for additional architectures, but it may be a fairly significant project.
- mmcloughlin 8y agoThanks :) > Any chance of ARM/ARM64 support? Yes multi-architecture support is definitely something I've had in mind. However I wanted to limit the scope for the initial version, so I could produce a result in a reasonable time and allow me to gauge interest. > It would be super useful if you could generate the code to memory then execute it directly. avo was inspired by PeachPy and asmjit, both of which can produce executable code directly. So implementing an assembler in avo is something I have had in the back of my mind. However it wasn't one of my primary goals. The use case I initially had in mind is massively unrolled crypto routines, where there's a lot of assembly code required but not much going on conceptually. My AES-CTR mode CL (https://go-review.googlesource.com/c/go/+/136896 https://go-review.googlesource.com/c/go/+/136896) and meow hash port (https://github.com/mmcloughlin/meow https://github.com/mmcloughlin/meow) got me thinking in this direction.