4 ms·

I wish there were a standard cross platform assembler library in C for x86, ARM, and WASM. Because frankly writing an asm text generator and shelling out to GCC

by duped 3y ago

I wish there were a standard cross platform assembler library in C for x86, ARM, and WASM. Because frankly writing an asm text generator and shelling out to GCC or NASM is really annoying, and it feels like something so foundational should just exist. I get there's yasm, and libnasm, and asmjit, and libgccjit, and llvm, etc but they're all varying degrees of "bad" for the purposes of a simple code generator for someone that doesn't want to dig through ARM and Intel's developer docs.

lelanthran 3y ago
> I wish there were a standard cross platform assembler library in C for x86, ARM, and WASM. Because frankly writing an asm text generator and shelling out to GCC or NASM is really annoying, and it feels like something so foundational should just exist. It does, it's called C :-) > I get there's yasm, and libnasm, and asmjit, and libgccjit, and llvm, etc but they're all varying degrees of "bad" for the purposes of a simple code generator for someone that doesn't want to dig through ARM and Intel's developer docs. This is only for language developers who aren't writing in C, right? Because if you're writing in C, what would you use this assembler for? Anything a cross-platform assembler emits is going to be portable, so it's only going to be as performant as C is, so why not just emit C? Having a C library that emits C might be a nice thing to have - then all the other languages can emit "code" that is cross-platform.
- duped 3y ago
  C is not assembler nor is it close to assembler nor is it desirable to have a C compiler in your toolchain to compile something that needs assembler For example, C has a calling convention. It has a notion of a stack and heap. Certain things like arbitrary control flow, threading, and process spawning cannot be expressed in C. It has a memory model that you must adhere to. If you actually want to implement a programming language, C is not sufficient as a transpilation target except for mostly trivial designs.
  - kragen 3y ago
    works pretty well most of the time; chicken scheme even compiles call/cc to c
  - lelanthran 3y ago
    > For example, C has a calling convention. It has a notion of a stack and heap. Certain things like arbitrary control flow, threading, and process spawning cannot be expressed in C. It has a memory model that you must adhere to. Yes, all trivially bypassed or worked-around if you're emitting C. Considering the initial constraint was "cross-platform": 1. Calling convention - what would be the point of a calling convention different to the rest of the system? You wouldn't be able to do system calls, for one. And if you need a different calling convention within the language itself C will let you push elements onto a stack and jump to a different piece of code[1], optionally saving and restoring registers. 2. The notion of a stack and heap - what's the alternative you have in mind when using assembler? Assemblers also have a notion of a stack and non-stack addresses, after all. 3. Arbitrary control flow - you can emit goto's to your hearts content (see [1] below). 4. Threading - this is platform-specific by nature, but if you're emitting C you can emit calls to the platform's thread primitives (CreateThreadEx, pthread_create, etc) the same way you'd have to do if you were using an assembler. Nothing stops you from proving a wrapper for threads that your emitted C code calls (so that it works on multiple platforms). 5. Process-spawning - once again, this is platform-specific, so even in assembler you're going to have to make a call to the platform's CreateProcess/posix_spawn/exec functions. If you're going to call a platform function from assembly, you may as well call it from C. 6. Memory model - I'll give you this point, even though the memory model is not that much different from assembler. If you're emitting C, you're still free to allocate/define a large array and use that memory in whatever way you wish. > If you actually want to implement a programming language, C is not sufficient as a transpilation target except for mostly trivial designs. Actually, many languages with non-trivial and advanced features, such as Common Lisp, already have implementations that transpile to C. They implement the entire Common Lisp standard, and don't seem to be missing any features even though they are emitting C and not assembler. Anonymous functions? Done with emitted C. Closures, continuations? Done with emitted C. Garbage collection? Same. What feature do you have in mind that cannot be implemented by emitting C[2], but can be implemented by emitting assembler[3]? [1] Some constraints on the jumping exist, but emitting GCC-specific C let's you do calculated expressions for the jump target - you don't need to `goto` a constant address. [2] I initially thought that something like Goroutines or async functions might be a candidate, but off the top of my head, I can think of at least one way to implement async functions and/or go routines by emitting C. There's probably more. [3] I'm not being contentious, I'd really rather like to know - my own toy language (everyone has one) currently in the phase of "I'm thinking really hard about what it should look like" is going to be emitting C. I couldn't find any feature for the language that can't be done by emitting C, so I really want to know what you have in mind.
userbinator 3y ago
You can generate the opcodes directly if you want to avoid having a separate assembler. AFAIK MSVC does that unless you tell it to generate Asm output, whereas GCC always goes through the separate assembler route.
- deleted 3y ago
  [deleted]
wg0 3y ago
One way would be to target and generate LLVM IR. You get multiple platforms for free if I'm not wrong.
Asm2D 3y ago
Can you elaborate what's "bad" on asmjit? I wrote it for the purpose of making it easy to write JIT compilers in C++ and it has been adopted even by projects that are written in C (like Erlang). It's also one of the smallest libraries out there and has a great performance. I understand why people don't want to use projects such as libgccjit or llvm, but asmjit is just super easy to use compared to others.
- duped 3y ago
  It's in C++. I don't really mind interfacing with C, but C++ is kind of a non-starter. And back when I last used it, the docs were pretty rough. I understand that's changed a lot and they look much better now.
  - a-priori 3y ago
    The fact that it's written in C++ doesn't make it 'bad'. It just means it's not suited to your use, which is your problem not the library's problem.
  - Asm2D 3y ago
    The reason why it's written in C++ is to make it practical. I understand why some people would want it in C, but honestly I have never seen a nice JIT assembler library written in C - I saw few incomplete assemblers in C before as part of other projects, but it was usually ugly code full of macros, so no proper C API to interface with anyway. I think that asmjit's biggest strength is in its completeness and performance, so it can be used to generate code in a sub-millisecond time, which higher level code generators such as LLVM don't offer.
    - duped 3y ago
      I understand, but I don't think you understand me. asmjit is great if you're writing a fast JIT and want to write C++, or can tolerate a C++ requirement. If you're writing a compiler in any other language, requiring C++ is a burden that may be insurmountable. What I'm lamenting is the lack of a widely used, decent C library for generating x86 and ARM code, which doesn't seem to exist, nor does asmjit fill that gap. > I think that asmjit's biggest strength is in its completeness and performance, so it can be used to generate code in a sub-millisecond time, which higher level code generators such as LLVM don't offer. Not everyone cares about sub-millsecond performance for code generation. It's easier to generate GCC syntax ASM and shell out to `asm` or `cc` than to write bindings to asmjit and interface from a different language, and the performance is fine because lowering your AST into an IR that can be used to generate asm either as text or the input to asmjit is a much bigger bottleneck.

by duped 3y ago

lelanthran 3y ago

> I wish there were a standard cross platform assembler library in C for x86, ARM, and WASM. Because frankly writing an asm text generator and shelling out to GCC or NASM is really annoying, and it feels like something so foundational should just exist. It does, it's called C :-) > I get there's yasm, and libnasm, and asmjit, and libgccjit, and llvm, etc but they're all varying degrees of "bad" for the purposes of a simple code generator for someone that doesn't want to dig through ARM and Intel's developer docs. This is only for language developers who aren't writing in C, right? Because if you're writing in C, what would you use this assembler for? Anything a cross-platform assembler emits is going to be portable, so it's only going to be as performant as C is, so why not just emit C? Having a C library that emits C might be a nice thing to have - then all the other languages can emit "code" that is cross-platform.

duped 3y ago
C is not assembler nor is it close to assembler nor is it desirable to have a C compiler in your toolchain to compile something that needs assembler For example, C has a calling convention. It has a notion of a stack and heap. Certain things like arbitrary control flow, threading, and process spawning cannot be expressed in C. It has a memory model that you must adhere to. If you actually want to implement a programming language, C is not sufficient as a transpilation target except for mostly trivial designs.
- kragen 3y ago
  works pretty well most of the time; chicken scheme even compiles call/cc to c
- lelanthran 3y ago
  > For example, C has a calling convention. It has a notion of a stack and heap. Certain things like arbitrary control flow, threading, and process spawning cannot be expressed in C. It has a memory model that you must adhere to. Yes, all trivially bypassed or worked-around if you're emitting C. Considering the initial constraint was "cross-platform": 1. Calling convention - what would be the point of a calling convention different to the rest of the system? You wouldn't be able to do system calls, for one. And if you need a different calling convention within the language itself C will let you push elements onto a stack and jump to a different piece of code[1], optionally saving and restoring registers. 2. The notion of a stack and heap - what's the alternative you have in mind when using assembler? Assemblers also have a notion of a stack and non-stack addresses, after all. 3. Arbitrary control flow - you can emit goto's to your hearts content (see [1] below). 4. Threading - this is platform-specific by nature, but if you're emitting C you can emit calls to the platform's thread primitives (CreateThreadEx, pthread_create, etc) the same way you'd have to do if you were using an assembler. Nothing stops you from proving a wrapper for threads that your emitted C code calls (so that it works on multiple platforms). 5. Process-spawning - once again, this is platform-specific, so even in assembler you're going to have to make a call to the platform's CreateProcess/posix_spawn/exec functions. If you're going to call a platform function from assembly, you may as well call it from C. 6. Memory model - I'll give you this point, even though the memory model is not that much different from assembler. If you're emitting C, you're still free to allocate/define a large array and use that memory in whatever way you wish. > If you actually want to implement a programming language, C is not sufficient as a transpilation target except for mostly trivial designs. Actually, many languages with non-trivial and advanced features, such as Common Lisp, already have implementations that transpile to C. They implement the entire Common Lisp standard, and don't seem to be missing any features even though they are emitting C and not assembler. Anonymous functions? Done with emitted C. Closures, continuations? Done with emitted C. Garbage collection? Same. What feature do you have in mind that cannot be implemented by emitting C[2], but can be implemented by emitting assembler[3]? [1] Some constraints on the jumping exist, but emitting GCC-specific C let's you do calculated expressions for the jump target - you don't need to `goto` a constant address. [2] I initially thought that something like Goroutines or async functions might be a candidate, but off the top of my head, I can think of at least one way to implement async functions and/or go routines by emitting C. There's probably more. [3] I'm not being contentious, I'd really rather like to know - my own toy language (everyone has one) currently in the phase of "I'm thinking really hard about what it should look like" is going to be emitting C. I couldn't find any feature for the language that can't be done by emitting C, so I really want to know what you have in mind.

userbinator 3y ago

You can generate the opcodes directly if you want to avoid having a separate assembler. AFAIK MSVC does that unless you tell it to generate Asm output, whereas GCC always goes through the separate assembler route.

deleted 3y ago
[deleted]

wg0 3y ago

One way would be to target and generate LLVM IR. You get multiple platforms for free if I'm not wrong.

Asm2D 3y ago

Can you elaborate what's "bad" on asmjit? I wrote it for the purpose of making it easy to write JIT compilers in C++ and it has been adopted even by projects that are written in C (like Erlang). It's also one of the smallest libraries out there and has a great performance. I understand why people don't want to use projects such as libgccjit or llvm, but asmjit is just super easy to use compared to others.

duped 3y ago
It's in C++. I don't really mind interfacing with C, but C++ is kind of a non-starter. And back when I last used it, the docs were pretty rough. I understand that's changed a lot and they look much better now.
- a-priori 3y ago
  The fact that it's written in C++ doesn't make it 'bad'. It just means it's not suited to your use, which is your problem not the library's problem.
- Asm2D 3y ago
  The reason why it's written in C++ is to make it practical. I understand why some people would want it in C, but honestly I have never seen a nice JIT assembler library written in C - I saw few incomplete assemblers in C before as part of other projects, but it was usually ugly code full of macros, so no proper C API to interface with anyway. I think that asmjit's biggest strength is in its completeness and performance, so it can be used to generate code in a sub-millisecond time, which higher level code generators such as LLVM don't offer.
  - duped 3y ago
    I understand, but I don't think you understand me. asmjit is great if you're writing a fast JIT and want to write C++, or can tolerate a C++ requirement. If you're writing a compiler in any other language, requiring C++ is a burden that may be insurmountable. What I'm lamenting is the lack of a widely used, decent C library for generating x86 and ARM code, which doesn't seem to exist, nor does asmjit fill that gap. > I think that asmjit's biggest strength is in its completeness and performance, so it can be used to generate code in a sub-millisecond time, which higher level code generators such as LLVM don't offer. Not everyone cares about sub-millsecond performance for code generation. It's easier to generate GCC syntax ASM and shell out to `asm` or `cc` than to write bindings to asmjit and interface from a different language, and the performance is fine because lowering your AST into an IR that can be used to generate asm either as text or the input to asmjit is a much bigger bottleneck.