6 ms·

Adding the ability to just read header files and all the definitions in them without having to translate. I saw this addition in zig and thought that it was am

by txdv 5y ago

Adding the ability to just read header files and all the definitions in them without having to translate. I saw this addition in zig and thought that it was amazing, a lot of languages have this need to create definitions in their own language, C# with [DllImport](https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.dllimportattribute?view=net-5.0 https://docs.microsoft.com/en-us/dotnet/api/system.runtime.i...), [crystal with lib](https://crystal-lang.org/reference/syntax_and_semantics/c_bindings/lib.html https://crystal-lang.org/reference/syntax_and_semantics/c_bi...), I think since C is so ubiquitous and being able to just consume header files just removes so much boilerplate creation that is easy to get wrong. zig also decided to just call an existing compiler (clang) and if you do zig cc --version then zig thinks it is clang, because the zig compiler just calls into clang main. D has gone with the route of calling gcc.

nikki93 5y ago
LuaJIT was "good enough" at this that it worked pretty well for me -- mostly for automatically binding to headers from my own code where I was careful to avoid some of the potential gotchas. With a macro the header was stringified and sent to LuaJIT; could've also had a build step. eg: https://github.com/nikki93/cgame/blob/master/src/sprite.h#L9 https://github.com/nikki93/cgame/blob/master/src/sprite.h#L9 An added benefit is you get ffi-reflect -- https://github.com/corsix/ffi-reflect https://github.com/corsix/ffi-reflect -- over the resulting types / functions...
dleslie 5y ago
Scheme is a language where I wish this was part of the spec. So many Schemes are intended to be embedded or extended with C, and every one of them has a bespoke way of declaring C bindings and martialling data between C and Scheme.
- shakna 5y ago
  There are ongoing efforts to stabilise a SRFI (like SRFI198 [0]), which would be the first step towards this. (First get the SRFI, then once that's stable and well implemented, it can be considered for the next spec). Unfortunately there's a lot of disagreement between the bigger players as to how this should be handled: > > 1) The authors appear to be deadlocked > No need to soften it with an "appear", we are intractably deadlocked. [0] https://srfi.schemers.org/srfi-198/srfi-198.html https://srfi.schemers.org/srfi-198/srfi-198.html
- froh 5y ago
  I agree. to bystanders, odd autocorrect: it's marshalling. marshalling data. although fight and war are indeed also appropriate associations for binary data exchange (struct conversions) between languages.
  - watersb 5y ago
    I often court martial my data. Not mocking the spelling of the previous post; it's a strange term anyway and in this instance either spelling works.
  - dmurray 5y ago
    TIL martial and marshal are unrelated etymologically, though marshalling data is by analogy with marshalling troops, and related to the military office. Marshal = mariscalcus = "horse servant", compare mare and seneschal. Martial = from Mars, god of war. https://en.wiktionary.org/wiki/marshal https://en.wiktionary.org/wiki/marshal
    - froh 5y ago
      TIL: marshalling == ceremonial arrangement and management of a gathering these CS gals of olde really had a hand for naming :-)
TazeTSchnitzel 5y ago
`zig cc` makes Zig into a more convenient C cross-compiler than upstream Clang!
- viraptor 5y ago
  Why is it more convenient?
  - nickcw 5y ago
    Zig can cross compile and link libc on lots of different architectures out of the box, which is amazing given how much effort this takes to set up cross compilers normally.
    - saurik 5y ago
      FWIW, this isn't actually difficult: you just need a sysroot, and past that you can just use clang (maybe ironically, for any platform other than Apple ones, as there still isn't a good common linker for mach-O; like, this still works fine, but it is in fact very annoying--but not impossible--to compile a macOS binary from any platform other than macOS). To cross-compile to Linux you can just download a handful of rpm files (the kernel and libc header packages, as well as the gcc package for the CRT) from CentOS or deb files from Ubuntu and extract them to a folder, and for Windows you just need a handful of Msys2 packages (the same general set). That's really all you need: my current major project--Orchid--compiles for Linux, macOS, iOS, Android, and Windows, and does so using the Android NDK copy of clang (just because it is easy to install and easy to ensure us always "the same" on every machine: I do cross-compiled reproducible builds). As long as you are using C/C++ it really should be easy to cross-compile (and yes: this is saurik saying something not only good about clang but specifically saying they finally nailed this use case, which they really hadn't a while back when people kept touting this as a feature... they really did eventually nail it). (That said, I do appreciate that not having to download and extract those packages is certainly "more convenient" than doing anything at all, but I just want to point out that it really isn't that "much effort".)
      - 5y ago
_old_dude_ 5y ago
yes, Java is adding jextract [1] as a replacement for JNI (Java Native Interface) which directly consumes header files using clang and produces a jar wrapping the native calls. It's not bolted in the compiler so less nice than Zig but the jar produced by jextract is not ABI dependent so more portable. The ABI dependent code is generated by the JIT using the the Foreign Linker API (the equivalent of C# DllImport) at runtime. [1] https://inside.java/2020/10/06/jextract/ https://inside.java/2020/10/06/jextract/
- pjmlp 5y ago
  Actually the equivalent of @dll.import from J++, funny how things turn out. https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-6.0/aa242534(v=vs.60) https://docs.microsoft.com/en-us/previous-versions/visualstu...
- MrBuddyCasino 5y ago
  That code also contains a nice C parser that is handy, backed by LLVM. I recently switched from JNA to Panama for a project, and so far no regrets, other than having to mess with the Java module system.
  - pjmlp 5y ago
    It is already stable enough?
    - MrBuddyCasino 5y ago
      Too early to tell. It clearly has received more care & attention so far than all the alternatives, so I’m not too worried about it being in „incubator“ status. The project lead knows his shit.
Rounin 5y ago
Not that this is terribly relevant to the discussion, but back in the day, I did try compiling bzflag with zig cc and then with clang, and the first executable crashed, while the second worked. Despite being told numerous compile-time options to use by different manuals and documents and people on IRC, nothing appeared to change the outcome. It might be due to some undefined behaviour in bzflag, of course, but it made zig cc a non-starter for C.
- ifreund 5y ago
  This is due to zig enabling clangs UBSAN by default in unoptimized builds. This can be disabled with `-fno-sanitize=undefined` or by enabling optimizations with `-O2` or `-O3`. https://github.com/ziglang/zig/issues/4830#issuecomment-605491606 https://github.com/ziglang/zig/issues/4830#issuecomment-6054...
  - Rounin 5y ago
    Well, as mentioned, I got a whole list of different compile-time options to try - no-this and no-that, but the executable still crashed in the end.
- matu3ba 5y ago
  Please be specific with version numbers. Did you file an upstream bug already? Do you mean https://github.com/BZFlag-Dev/bzflag https://github.com/BZFlag-Dev/bzflag ? Thats c++ and must be compiled with zig c++.
  - Rounin 5y ago
    Ah, well, it must have been zig c++, then, necessarily. Was I planning to file an upstream bug? I was unaware. I did contact them about it, actually, but it wasn't solved at the time.
- DrBazza 5y ago
  Apropos of nothing, but found out that the zig compiler can run C++: https://twitter.com/andy_kelley/status/1391468499676008451?s=20 https://twitter.com/andy_kelley/status/1391468499676008451?s...
WalterBright 5y ago
> D has gone with the route of calling gcc. Not exactly. D is going the route of actually implementing a C compiler in the D compiler. Here's the C parser: https://github.com/dlang/dmd/pull/12507/files#diff-2fc171ca9f600bfba68ca8a62af0d4ec031d09cf65b9649900cca773379def6f https://github.com/dlang/dmd/pull/12507/files#diff-2fc171ca9... You could argue that writing a C compiler instead of using libclang is an indicator of brain-damage, but I lay out my reasons here: https://digitalmars.com/d/archives/digitalmars/D/Add_ImportC_compiler_to_dmd_349611.html#N349631 https://digitalmars.com/d/archives/digitalmars/D/Add_ImportC... And besides, C is such a simple language it's kinda fun.
- beeforpork 5y ago
  So how extremely could I exploit this? Could I write a program entirely in C, compile it with the D compiler, and use D's unit test framework? That would be insane (very good insane, obviously).
  - WalterBright 5y ago
    That is a great question. The point of ImportC is to import the declarations from a C file. But it turns out you can simply give the C file on the command line, and it'll compile it and make an executable: echo hello.c int printf(const char*,...); int main() { printf("hello world\n"); } dmd hello.c ./hello hello world But then there's the question "should D features be allowed in the C code?" This is a very subversive question. Is it really C if that is allowed? Already, the C code allows for Compile Time Function Execution, which comes for free by passing the C code through the D front end. You can see it in action in the test cases in this file: https://github.com/dlang/dmd/pull/12507/files#diff-b94776a026b50e985b5e0bf8c2be0c17fdca9730c5934f47d9ebe6ab9ede35bd https://github.com/dlang/dmd/pull/12507/files#diff-b94776a02... Scroll down a bit and you'll see it executes C functions at compile time. I decided to flirt with heresy and leave this in, because it sure makes adding tests convenient! I'm less sure about adding unittests to the C compiler. Certainly, you can always add the unittests to the D code that is importing the C code. This will probably hold the charges of abomination to a minimum.

by txdv 5y ago

nikki93 5y ago

LuaJIT was "good enough" at this that it worked pretty well for me -- mostly for automatically binding to headers from my own code where I was careful to avoid some of the potential gotchas. With a macro the header was stringified and sent to LuaJIT; could've also had a build step. eg: https://github.com/nikki93/cgame/blob/master/src/sprite.h#L9 https://github.com/nikki93/cgame/blob/master/src/sprite.h#L9 An added benefit is you get ffi-reflect -- https://github.com/corsix/ffi-reflect https://github.com/corsix/ffi-reflect -- over the resulting types / functions...

dleslie 5y ago

Scheme is a language where I wish this was part of the spec. So many Schemes are intended to be embedded or extended with C, and every one of them has a bespoke way of declaring C bindings and martialling data between C and Scheme.

shakna 5y ago
There are ongoing efforts to stabilise a SRFI (like SRFI198 [0]), which would be the first step towards this. (First get the SRFI, then once that's stable and well implemented, it can be considered for the next spec). Unfortunately there's a lot of disagreement between the bigger players as to how this should be handled: > > 1) The authors appear to be deadlocked > No need to soften it with an "appear", we are intractably deadlocked. [0] https://srfi.schemers.org/srfi-198/srfi-198.html https://srfi.schemers.org/srfi-198/srfi-198.html
froh 5y ago
I agree. to bystanders, odd autocorrect: it's marshalling. marshalling data. although fight and war are indeed also appropriate associations for binary data exchange (struct conversions) between languages.
- watersb 5y ago
  I often court martial my data. Not mocking the spelling of the previous post; it's a strange term anyway and in this instance either spelling works.
- dmurray 5y ago
  TIL martial and marshal are unrelated etymologically, though marshalling data is by analogy with marshalling troops, and related to the military office. Marshal = mariscalcus = "horse servant", compare mare and seneschal. Martial = from Mars, god of war. https://en.wiktionary.org/wiki/marshal https://en.wiktionary.org/wiki/marshal
  - froh 5y ago
    TIL: marshalling == ceremonial arrangement and management of a gathering these CS gals of olde really had a hand for naming :-)

TazeTSchnitzel 5y ago

`zig cc` makes Zig into a more convenient C cross-compiler than upstream Clang!

viraptor 5y ago
Why is it more convenient?
- nickcw 5y ago
  Zig can cross compile and link libc on lots of different architectures out of the box, which is amazing given how much effort this takes to set up cross compilers normally.
  - saurik 5y ago
    FWIW, this isn't actually difficult: you just need a sysroot, and past that you can just use clang (maybe ironically, for any platform other than Apple ones, as there still isn't a good common linker for mach-O; like, this still works fine, but it is in fact very annoying--but not impossible--to compile a macOS binary from any platform other than macOS). To cross-compile to Linux you can just download a handful of rpm files (the kernel and libc header packages, as well as the gcc package for the CRT) from CentOS or deb files from Ubuntu and extract them to a folder, and for Windows you just need a handful of Msys2 packages (the same general set). That's really all you need: my current major project--Orchid--compiles for Linux, macOS, iOS, Android, and Windows, and does so using the Android NDK copy of clang (just because it is easy to install and easy to ensure us always "the same" on every machine: I do cross-compiled reproducible builds). As long as you are using C/C++ it really should be easy to cross-compile (and yes: this is saurik saying something not only good about clang but specifically saying they finally nailed this use case, which they really hadn't a while back when people kept touting this as a feature... they really did eventually nail it). (That said, I do appreciate that not having to download and extract those packages is certainly "more convenient" than doing anything at all, but I just want to point out that it really isn't that "much effort".)
    - 5y ago

_old_dude_ 5y ago

yes, Java is adding jextract [1] as a replacement for JNI (Java Native Interface) which directly consumes header files using clang and produces a jar wrapping the native calls. It's not bolted in the compiler so less nice than Zig but the jar produced by jextract is not ABI dependent so more portable. The ABI dependent code is generated by the JIT using the the Foreign Linker API (the equivalent of C# DllImport) at runtime. [1] https://inside.java/2020/10/06/jextract/ https://inside.java/2020/10/06/jextract/

pjmlp 5y ago
Actually the equivalent of @dll.import from J++, funny how things turn out. https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-6.0/aa242534(v=vs.60) https://docs.microsoft.com/en-us/previous-versions/visualstu...
MrBuddyCasino 5y ago
That code also contains a nice C parser that is handy, backed by LLVM. I recently switched from JNA to Panama for a project, and so far no regrets, other than having to mess with the Java module system.
- pjmlp 5y ago
  It is already stable enough?
  - MrBuddyCasino 5y ago
    Too early to tell. It clearly has received more care & attention so far than all the alternatives, so I’m not too worried about it being in „incubator“ status. The project lead knows his shit.

Rounin 5y ago

Not that this is terribly relevant to the discussion, but back in the day, I did try compiling bzflag with zig cc and then with clang, and the first executable crashed, while the second worked. Despite being told numerous compile-time options to use by different manuals and documents and people on IRC, nothing appeared to change the outcome. It might be due to some undefined behaviour in bzflag, of course, but it made zig cc a non-starter for C.

ifreund 5y ago
This is due to zig enabling clangs UBSAN by default in unoptimized builds. This can be disabled with `-fno-sanitize=undefined` or by enabling optimizations with `-O2` or `-O3`. https://github.com/ziglang/zig/issues/4830#issuecomment-605491606 https://github.com/ziglang/zig/issues/4830#issuecomment-6054...
- Rounin 5y ago
  Well, as mentioned, I got a whole list of different compile-time options to try - no-this and no-that, but the executable still crashed in the end.
matu3ba 5y ago
Please be specific with version numbers. Did you file an upstream bug already? Do you mean https://github.com/BZFlag-Dev/bzflag https://github.com/BZFlag-Dev/bzflag ? Thats c++ and must be compiled with zig c++.
- Rounin 5y ago
  Ah, well, it must have been zig c++, then, necessarily. Was I planning to file an upstream bug? I was unaware. I did contact them about it, actually, but it wasn't solved at the time.
DrBazza 5y ago
Apropos of nothing, but found out that the zig compiler can run C++: https://twitter.com/andy_kelley/status/1391468499676008451?s=20 https://twitter.com/andy_kelley/status/1391468499676008451?s...

WalterBright 5y ago

> D has gone with the route of calling gcc. Not exactly. D is going the route of actually implementing a C compiler in the D compiler. Here's the C parser: https://github.com/dlang/dmd/pull/12507/files#diff-2fc171ca9f600bfba68ca8a62af0d4ec031d09cf65b9649900cca773379def6f https://github.com/dlang/dmd/pull/12507/files#diff-2fc171ca9... You could argue that writing a C compiler instead of using libclang is an indicator of brain-damage, but I lay out my reasons here: https://digitalmars.com/d/archives/digitalmars/D/Add_ImportC_compiler_to_dmd_349611.html#N349631 https://digitalmars.com/d/archives/digitalmars/D/Add_ImportC... And besides, C is such a simple language it's kinda fun.

beeforpork 5y ago
So how extremely could I exploit this? Could I write a program entirely in C, compile it with the D compiler, and use D's unit test framework? That would be insane (very good insane, obviously).
- WalterBright 5y ago
  That is a great question. The point of ImportC is to import the declarations from a C file. But it turns out you can simply give the C file on the command line, and it'll compile it and make an executable: echo hello.c int printf(const char*,...); int main() { printf("hello world\n"); } dmd hello.c ./hello hello world But then there's the question "should D features be allowed in the C code?" This is a very subversive question. Is it really C if that is allowed? Already, the C code allows for Compile Time Function Execution, which comes for free by passing the C code through the D front end. You can see it in action in the test cases in this file: https://github.com/dlang/dmd/pull/12507/files#diff-b94776a026b50e985b5e0bf8c2be0c17fdca9730c5934f47d9ebe6ab9ede35bd https://github.com/dlang/dmd/pull/12507/files#diff-b94776a02... Scroll down a bit and you'll see it executes C functions at compile time. I decided to flirt with heresy and leave this in, because it sure makes adding tests convenient! I'm less sure about adding unittests to the C compiler. Certainly, you can always add the unittests to the D code that is importing the C code. This will probably hold the charges of abomination to a minimum.