10 ms·
Adding ANSI C11 C compiler to D so it can import and compile C files directly
- mappu 6y agoA bit more discussion on the hybrid-forum-and-NNTP: https://forum.dlang.org/thread/s79ib1$1122$1@digitalmars.com https://forum.dlang.org/thread/s79ib1$1122$1@digitalmars.com
- txdv 6y agoAdding the ability to just read header files and all the definitions in them without having to translate. I saw this addition in zig and thought that it was amazing, a lot of languages have this need to create definitions in their own language, C# with [DllImport](https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.dllimportattribute?view=net-5.0 https://docs.microsoft.com/en-us/dotnet/api/system.runtime.i...), [crystal with lib](https://crystal-lang.org/reference/syntax_and_semantics/c_bindings/lib.html https://crystal-lang.org/reference/syntax_and_semantics/c_bi...), I think since C is so ubiquitous and being able to just consume header files just removes so much boilerplate creation that is easy to get wrong. zig also decided to just call an existing compiler (clang) and if you do zig cc --version then zig thinks it is clang, because the zig compiler just calls into clang main. D has gone with the route of calling gcc.
- nikki93 6y agoLuaJIT was "good enough" at this that it worked pretty well for me -- mostly for automatically binding to headers from my own code where I was careful to avoid some of the potential gotchas. With a macro the header was stringified and sent to LuaJIT; could've also had a build step. eg: https://github.com/nikki93/cgame/blob/master/src/sprite.h#L9 https://github.com/nikki93/cgame/blob/master/src/sprite.h#L9 An added benefit is you get ffi-reflect -- https://github.com/corsix/ffi-reflect https://github.com/corsix/ffi-reflect -- over the resulting types / functions...
- dleslie 6y agoScheme is a language where I wish this was part of the spec. So many Schemes are intended to be embedded or extended with C, and every one of them has a bespoke way of declaring C bindings and martialling data between C and Scheme.
- shakna 6y agoThere are ongoing efforts to stabilise a SRFI (like SRFI198 [0]), which would be the first step towards this. (First get the SRFI, then once that's stable and well implemented, it can be considered for the next spec). Unfortunately there's a lot of disagreement between the bigger players as to how this should be handled: > > 1) The authors appear to be deadlocked > No need to soften it with an "appear", we are intractably deadlocked. [0] https://srfi.schemers.org/srfi-198/srfi-198.html https://srfi.schemers.org/srfi-198/srfi-198.html
- froh 6y agoI agree. to bystanders, odd autocorrect: it's marshalling. marshalling data. although fight and war are indeed also appropriate associations for binary data exchange (struct conversions) between languages.
- watersb 6y agoI often court martial my data. Not mocking the spelling of the previous post; it's a strange term anyway and in this instance either spelling works.
- dmurray 6y agoTIL martial and marshal are unrelated etymologically, though marshalling data is by analogy with marshalling troops, and related to the military office. Marshal = mariscalcus = "horse servant", compare mare and seneschal. Martial = from Mars, god of war. https://en.wiktionary.org/wiki/marshal https://en.wiktionary.org/wiki/marshal
- froh 6y agoTIL: marshalling == ceremonial arrangement and management of a gathering these CS gals of olde really had a hand for naming :-)
- TazeTSchnitzel 6y ago`zig cc` makes Zig into a more convenient C cross-compiler than upstream Clang!
- viraptor 6y agoWhy is it more convenient?
- nickcw 6y agoZig can cross compile and link libc on lots of different architectures out of the box, which is amazing given how much effort this takes to set up cross compilers normally.
- saurik 6y agoFWIW, this isn't actually difficult: you just need a sysroot, and past that you can just use clang (maybe ironically, for any platform other than Apple ones, as there still isn't a good common linker for mach-O; like, this still works fine, but it is in fact very annoying--but not impossible--to compile a macOS binary from any platform other than macOS). To cross-compile to Linux you can just download a handful of rpm files (the kernel and libc header packages, as well as the gcc package for the CRT) from CentOS or deb files from Ubuntu and extract them to a folder, and for Windows you just need a handful of Msys2 packages (the same general set). That's really all you need: my current major project--Orchid--compiles for Linux, macOS, iOS, Android, and Windows, and does so using the Android NDK copy of clang (just because it is easy to install and easy to ensure us always "the same" on every machine: I do cross-compiled reproducible builds). As long as you are using C/C++ it really should be easy to cross-compile (and yes: this is saurik saying something not only good about clang but specifically saying they finally nailed this use case, which they really hadn't a while back when people kept touting this as a feature... they really did eventually nail it). (That said, I do appreciate that not having to download and extract those packages is certainly "more convenient" than doing anything at all, but I just want to point out that it really isn't that "much effort".)
- 6y ago
- _old_dude_ 6y agoyes, Java is adding jextract [1] as a replacement for JNI (Java Native Interface) which directly consumes header files using clang and produces a jar wrapping the native calls. It's not bolted in the compiler so less nice than Zig but the jar produced by jextract is not ABI dependent so more portable. The ABI dependent code is generated by the JIT using the the Foreign Linker API (the equivalent of C# DllImport) at runtime. [1] https://inside.java/2020/10/06/jextract/ https://inside.java/2020/10/06/jextract/
- pjmlp 6y agoActually the equivalent of @dll.import from J++, funny how things turn out. https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-6.0/aa242534(v=vs.60) https://docs.microsoft.com/en-us/previous-versions/visualstu...
- MrBuddyCasino 6y agoThat code also contains a nice C parser that is handy, backed by LLVM. I recently switched from JNA to Panama for a project, and so far no regrets, other than having to mess with the Java module system.
- pjmlp 6y agoIt is already stable enough?
- MrBuddyCasino 6y agoToo early to tell. It clearly has received more care & attention so far than all the alternatives, so I’m not too worried about it being in „incubator“ status. The project lead knows his shit.
- Rounin 6y agoNot that this is terribly relevant to the discussion, but back in the day, I did try compiling bzflag with zig cc and then with clang, and the first executable crashed, while the second worked. Despite being told numerous compile-time options to use by different manuals and documents and people on IRC, nothing appeared to change the outcome. It might be due to some undefined behaviour in bzflag, of course, but it made zig cc a non-starter for C.
- ifreund 6y agoThis is due to zig enabling clangs UBSAN by default in unoptimized builds. This can be disabled with `-fno-sanitize=undefined` or by enabling optimizations with `-O2` or `-O3`. https://github.com/ziglang/zig/issues/4830#issuecomment-605491606 https://github.com/ziglang/zig/issues/4830#issuecomment-6054...
- Rounin 6y agoWell, as mentioned, I got a whole list of different compile-time options to try - no-this and no-that, but the executable still crashed in the end.
- matu3ba 6y agoPlease be specific with version numbers. Did you file an upstream bug already? Do you mean https://github.com/BZFlag-Dev/bzflag https://github.com/BZFlag-Dev/bzflag ? Thats c++ and must be compiled with zig c++.
- Rounin 6y agoAh, well, it must have been zig c++, then, necessarily. Was I planning to file an upstream bug? I was unaware. I did contact them about it, actually, but it wasn't solved at the time.
- DrBazza 6y agoApropos of nothing, but found out that the zig compiler can run C++: https://twitter.com/andy_kelley/status/1391468499676008451?s=20 https://twitter.com/andy_kelley/status/1391468499676008451?s...
- WalterBright 6y ago> D has gone with the route of calling gcc. Not exactly. D is going the route of actually implementing a C compiler in the D compiler. Here's the C parser: https://github.com/dlang/dmd/pull/12507/files#diff-2fc171ca9f600bfba68ca8a62af0d4ec031d09cf65b9649900cca773379def6f https://github.com/dlang/dmd/pull/12507/files#diff-2fc171ca9... You could argue that writing a C compiler instead of using libclang is an indicator of brain-damage, but I lay out my reasons here: https://digitalmars.com/d/archives/digitalmars/D/Add_ImportC_compiler_to_dmd_349611.html#N349631 https://digitalmars.com/d/archives/digitalmars/D/Add_ImportC... And besides, C is such a simple language it's kinda fun.
- beeforpork 6y agoSo how extremely could I exploit this? Could I write a program entirely in C, compile it with the D compiler, and use D's unit test framework? That would be insane (very good insane, obviously).
- WalterBright 6y agoThat is a great question. The point of ImportC is to import the declarations from a C file. But it turns out you can simply give the C file on the command line, and it'll compile it and make an executable: echo hello.c int printf(const char*,...); int main() { printf("hello world\n"); } dmd hello.c ./hello hello world But then there's the question "should D features be allowed in the C code?" This is a very subversive question. Is it really C if that is allowed? Already, the C code allows for Compile Time Function Execution, which comes for free by passing the C code through the D front end. You can see it in action in the test cases in this file: https://github.com/dlang/dmd/pull/12507/files#diff-b94776a026b50e985b5e0bf8c2be0c17fdca9730c5934f47d9ebe6ab9ede35bd https://github.com/dlang/dmd/pull/12507/files#diff-b94776a02... Scroll down a bit and you'll see it executes C functions at compile time. I decided to flirt with heresy and leave this in, because it sure makes adding tests convenient! I'm less sure about adding unittests to the C compiler. Certainly, you can always add the unittests to the D code that is importing the C code. This will probably hold the charges of abomination to a minimum.
- jokoon 6y agoI remember adding TinyCC to my C++ projects, so I could have some kind of a scripting language into a large C++ project that is fastidious to build. I a way I'd rather use C than lua. I wish I could easily integrate python as a scripting language, but python is slow and its runtime is heavy. TinyCC, however, is very light.
- bondant 6y agoI agree with you. I like how easy it is to embed Lua into an application, but as a language I really prefer python. Sadly it is not as easy to integrate the python interpreter into a project.
- aeldidi 6y agoWow that’s really convenient. I was just thinking of how nice it would be to have that feature literally hours ago while working on a side project involving a lot of D/C interop. Major props to Walter and the rest of the D developers for being so seemingly in tune with what people are looking for in a language.
- WalterBright 6y agoWalter here. AMA.
- throw_m239339 6y agoNo question, just thanks for your hardwork and D contributor's. Diversity, competition and choice are important when it comes to programming languages.
- LAC-Tech 6y agoDo you think there's tension in Dlang trying be both a systems language like C++ AND a high level garbage collected language in C#? Which is more important?
- aldacron 6y agoObviously I'm not Walter, but IMO, D's approach to GC does not create any tension. It's just a systems language with a GC that's there if you want to use it. You can disable it completely or only where you need to. Yes, you lose some features when you do so (like automanaged memory for dynamic array operations), but the tradeoffs are well-known and worth the loss when you need to make them. Consider WekaIO, which used D for their filesystem. No GC anywhere: https://dlang.org/blog/2018/12/04/interview-liran-zvibel-of-wekaio/ https://dlang.org/blog/2018/12/04/interview-liran-zvibel-of-...
- skocznymroczny 6y agoI think there's more to the tension than GC. GC is the red herring that everyone jumps to when it comes to such comparisons, but I think there are many more issues that could cause tensions. For example OOP features, inheritance, properties, that kind of things. For system programmer oriented folks these features are usually considered bloat, more high-level/application programmers consider them very valuable features. For me currently D provides a good level of balance between those worlds, but the aforementioned GC isn't an issue for me at all so I have it easy.
- Shadonototro 6y agoI always said this, C is so huge that it make no sense to try to "replace" it, you'd better make sure you can easily consume that ecosystem That's a step in the right direction! I love D!
- pjmlp 6y agoTrue, but we can still try to fix it somehow, because certain eco-systems will never move beyond C and it would be nice not to have to worry about its famous issues as we do to this day.
- galangalalgol 6y agoRewrite it all in Rust! I mean D! Or... Zig maybe? Lots of valid IOT languages, but how many are actually in refrigerators or mice or keyboards or car door oepners? That is a lot of inertia.
- pjmlp 6y agoMy point was fixing C, the ecosystems that value it after 50 years aren't going to rewrite their code into something else, as much as many of us would like it to happen.
- galangalalgol 6y agoYes, I was agreeing with you and forgot to tag my snarkyness as such.
- pjmlp 6y agoAh, sorry if I missed then. Here is one example from Apple in that regard, search for "Memory safe iBoot implementation". https://manuals.info.apple.com/MANUALS/1000/MA1902/en_US/apple-platform-security-guide.pdf https://manuals.info.apple.com/MANUALS/1000/MA1902/en_US/app...
- mariusor 6y ago
- LAC-Tech 6y agoFor me this makes Dlang a very obvious choice when it comes to code that calls C libraries, where I might have previously used C++.
- bsder 6y agoNow if we can get people to quit starting new projects in C++ (looking at you Google) we can finally make some progress in programming languages.
- p0nce 6y agoAs someone having painfully translated some of STB headers over the years to D, this is good news. It's very easy for the translation to go slightly wrong and then code you don't understand breaks, usually loosing the time investment. That said, that work is pretty early to judge.
- bregma 6y agoIf you can't beat them, join them.
- galangalalgol 6y agoSince C90 the standard has been ISO not ANSI. C89 was the last ANSI C. Then ISO adopted essentially that same language as C90, and C95 C99 etc have all been ISO standards.
- WalterBright 6y agoThanks. I renamed it to ISO in the documentation.
- crazypython 6y agoContext: Walter Bright created the first native C++ compiler, Zortech C++. This pull request actually implements its own C compiler! For Walter, considering he's implemented C++98 alone, implementing C must be nothing :) He implemented the ANSI C compiler here in just 5 days. Semi-related project, using libclang to import C++ directly (with C++ templates and all the other features, regular D has C++ linking and exception support built-in): https://github.com/Syniurge/Calypso https://github.com/Syniurge/Calypso Some quotes from the forum https://forum.dlang.org/post/s7a37h$2bbv$1@digitalmars.com https://forum.dlang.org/post/s7a37h$2bbv$1@digitalmars.com : - Writing a C preprocessor is a nightmare. Fortunately, I already did that https://github.com/facebookarchive/warp https://github.com/facebookarchive/warp and we can use it if we choose. - The D code will be able to inline C code, and even CTFE it. - There are a lot of wacky C extensions out there. We only need to implement currently used ones (not the 16 bit stuff), and only the stuff that appears in C headers. - Without a C compiler, we're stuck with, wedded to, and beholden to libclang. I wouldn't be surprised that the eventual cost of adapting ourselves to libclang will exceed the cost of doing our own C compiler.
- mhh__ 6y ago>ANSI C compiler here in just 5 days This is good work but let's be realistic - it doesn't work yet on any real C code I've tried it with.
- crazypython 6y agoHey, fellow Dlanger! Yeah, it doesn't support C headers yet. But an excellent prototype nonetheless.
- greatgoat420 6y ago> Hey, fellow Dlanger! Lol, he is a paid hype man for the Dlang community, and you should ask him about Symmetry Investments and if they are hiring.
- mhh__ 6y ago
- Imperatorn 6y agoD rox
- ape4 6y agoIs the C compiler written in D?
- pjmlp 6y agoYes, dmd was orinally written in C++, but has been ported into D for quite some time now, this is an extension of said compiler. ldc and gdc are a mix of D and C++ due to the respective backends.
- gavinray 6y agoNow we just need rock-solid automatic "extern (C++)" bindgen and we're g2g
- skocznymroczny 6y agoHas there been any notable success story with extern(c++)? I know some folks use it to interop with their own C++ code, but I haven't seen any new bindings to any popular C++ libraries.
- mhh__ 6y agoextern(C++) is what allows the LLVM D compiler to work for example. Binding C++ libraries is quite risky in the sense that the libraries usually aren't meant to be consumed via that API. For that reason it's mainly used (publicly at least, there are things I know that I cannot say) for exposing D to C++ in a sane way (Mangling, vtables, templates, all work, exceptions even!).
- greatgoat420 6y agoI think it is a great sign of the health of the dlang community and project that the creator of the language can submit a PR and get actual honest feedback just like any other participant.