10 ms·

Adding Modules to C in 10 Lines of Code [pdf]

138 points by pmarin 4y ago

unsafecast 4y ago
I've done something different before: // main.c #include <module.c> int main() { moduleFunction(); } // module.c #include <stdio.h> void moduleFunction() { printf("calling modules"); } This has its issues (you can't do parallel compilation anymore; you don't have unexported functions, etc), but it works fine for small things. Super helpful when I want to make a small thing and don't want to bother with headers. You just include all the C files in one file and be done with it.
- actionfromafar 4y ago
  It also automatically enables "global optimisation" in most cases. This style is called "amalgamation" and SQLite is distributed (but not developed) in this way.
  - unsafecast 4y ago
    TIL it has a name. Doing what sqlite does gives you the best of both traditional and single header projects I imagine.
  - RicoElectrico 4y ago
    Does this impose limitations on the code? I think all functions, even private, should be "namespaced" (i.e. in C - prefixed) then? Kind of awkward, but otherwise amalgamation is a great idea.
    - OskarS 4y ago
      C doesn't have "private" functions in that way, what it has is "static" functions, and this is indeed the biggest limitation of doing this kind of amalgamated build. If you mark a function as static, it means that has internal linkage and is only visible inside the translation unit was defined and no symbol is exported for linking. This is not the default: if you just define a function with some name, that symbol is in general visible to all other translation units that gets linked together by the linker (though with no type information attached, just the name of the symbol. Making sure the type information is correct is the purpose of header files). That means you can have any number of functions marked static in different translation units with the same name, and it's fine, they don't see each other. If you're doing an amalgamated build like this, that no longer works: all symbols have to have globally unique names. In addition, of course, you don't get the "separation" that "static" provides. You can't call a static function across a translation unit boundary, but since this just one big translation unit, anything goes: all functions can call all other functions, even if they shouldn't.
    - arinlen 4y ago
      > Does this impose limitations on the code? It does. Any technique that relies on features specific to translation units, such as taking advantage of internal linkage to reuse symbols to hold data specific to translation units, cannot be used anymore. Also, even though global builds are faster, you forego the ability to do incremental builds which makes working over a bug a more time-consuming task.
  - arinlen 4y ago
    > This style is called "amalgamation" and SQLite is distributed (but not developed) in this way. These are actually called "unity builds" or "jumbo builds", and some build systems such as CMake offer support for this. https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.htm...
- formerly_proven 4y ago
  For smaller C++ projects this tends to be way faster than the conventional wisdom of having many CUs, mostly because all the library bloat will only be parsed and instantiated once.
  - kringerkat 4y ago
    Also reduces link time. I find that projects up to around 50K lines compiles instantly as a single file/"unity build" and is my preferred way for most of my projects.
    - arinlen 4y ago
      > Also reduces link time. It's important to stress that it does more than reduce link time: unity builds completely eliminate linking when plugging together submodules, and the final project linking simply has far fewer object files to link.
  - Asooka 4y ago
    This really should be solved at the compiler level, rather than forcing people to restructure their projects.
- jariRG 4y ago
  Afaik this is exactly what Unity builds are. https://en.wikipedia.org/wiki/Unity_build https://en.wikipedia.org/wiki/Unity_build
  - unsafecast 4y ago
    Yeah. Though most projects still use header files with unity builds, so I decided not to call them that. It wasn't the concept of a unity build that I was emphasising, it was the idea of not forward-declaring everything.
    - arinlen 4y ago
      > Yeah. Though most projects still use header files with unity builds, so I decided not to call them that. There is nothing inherently special in header files. They are just a convention where you use a specific extension in source files you mostly use to pass declarations. Nothing stops you from using the .c extension on all files, and there is nothing special in passing declarations in one place and definitions in another place.
      - unsafecast 4y ago
        Of course! But, saying 'unity build' puts emphasis on the fact that you compile all the C files 'in unity', in a single CU. What _I_ wanted to put emphasis on is that you can skip the header files, and that coincidentally means you also have to unity build. The fact that most unity built projects still use headers made me hesitant to use the term, as that would put emphasis on building a single CU, as opposed to not forward declaring in headers.
  - TingPing 4y ago
    Build systems like Meson can even enable/disable this with an option ootb.
- agent281 4y ago
  I never really had a good grasp of C. Why is this code novel? It looks like importing a function in any language.
  - phaer 4y ago
    It's including another C file, normally you'd just include a "header file" in C and compile each C file separately.
    - usrn 4y ago
      For those who don't use C: header files are more or less an interface description and (usually) don't include any code. The code is added from external objects during the second "linking" step.
    - agent281 4y ago
      Oh, I see: including a header file injects forward declarations into the calling file. Each c file gets compiled separately into object code unless you directly import them like the code above is doing. In that case it essentially dumps the full text of the file into the calling file, which causes serial compilation of a single object file. Unless I'm mistaken that clears some stuff up for me. Thank you!
      - phaer 4y ago
        That's my understanding as well, so I think you are correct but I am no C wizard myself :)
  - arinlen 4y ago
    > I never really had a good grasp of C. Why is this code novel? It looks like importing a function in any language. Unity builds are a very old and widely established concept. I dare to speculate they are older than the regular header and source file split, as C required a full blown preprocessor to split code in separate files and also someone had to come up with the include guard trick to get around the limitations of including the same definition multiple times. They aren't employed that often because there are some tradeoffs and in large projects it's benefitial to split them in smaller translation units to limit the scope of incremental builds.
Shadonototra 4y ago
ImportC is shaping up to be one of the greatest feature of D Being able to just import your C code, and call it from C, or when you need to go back to C and resume your work by just importing your D modules I tried it, while it still need some finish, it worked very well! (you had to put your files through a preprocessor though, hopefully this step won't be needed in the future) Every modern language should include a C compiler, it is stupid to give up the entire C ecosystem! No wonder Walter's name is "Bright"! To the people who still doubt about D "because of the GC", it doesn't exist for me, i am making an online 3D game targeting WASM without it, with all the language features! For cases when you need one "if you use D as a scripting language", it will be very helpful to have! https://www.kdom.xyz/ https://www.kdom.xyz/ If you love C, but need something modern with more safeties, give D a try! D is a pragmatic language, very powerful, covers everyone's needs! If you tried it before, now is the time to give it another try!
- Tozen 4y ago
  Another C alternative that people might want to check out or keep an eye on is Vlang (https://vlang.io/ https://vlang.io/, https://github.com/vlang/v/blob/master/doc/docs.md https://github.com/vlang/v/blob/master/doc/docs.md).
- brabel 4y ago
  I've been learning Zig and it seems that Zig and D have a lot of things in common. With Zig one can readily import C code as well, and with `c-call-convention` any Zig function can be made callable from C. However, I've run into lots of bugs in Zig so I got a little disillusioned... how do you think D compares with Zig? Is it able to produce as efficient and small binaries, cross-compile to most platforms, metaprogramming etc?
  - speps 4y ago
    D's standard lib is also a lot more mature than Zig's, obviously because of its age. I'd recommend D above Zig unless you want to contribute bug fixes to Zig.
    - 4y ago
bachmeier 4y ago
On the topic of the presentation, I'd be happy to have a way to kill the preprocessor. Not that I expect that to happen. Reading C code is like reading literature that only makes sense if you know eight languages, four of which are known to fewer than 100 people.
- travisgriggs 4y ago
  This differs from web applications how? (not that you said it did, but the implication seems to be that C is an exception in software development in this regard)
  - kingaillas 4y ago
    I'm not a webapp dev, but C/C++ always felt unique via the preprocessor functionality that essentially does text substitutions on source code before handing off to the compiler. Actual flags are useful, e.g. -codegenfoo -linkflagbar, but I hate digging through a maze of -DFLIPOPTION1 and -DUSERSUPPLIEDTEXT="hahaha" to unscramble what code the compiler actually sees, and what went wrong because of the various user supplied strings (i.e. #define overwrites). If other languages also have this, well ugh sorry. I have grown to despise it.
- Asooka 4y ago
  C would be a lot harder to write without the preprocessor. I have only really had one truly awful experience with the preprocessor in the last 15 years and it was when reading OpenSSL's code. Even then, I didn't have too much trouble to grok it with just Vim.
  - WalterBright 4y ago
    I discovered my C code was a lot better when I backed out all the clever metaprogramming preprocesser usage. C could be easily extended with: 1. modules 2. manifest constants 3. conditional compilation and the preprocessor can be deprecated.
    - 4y ago
khoobid_shoma 4y ago
The title is a bit misleading. This is about extending C grammar used in D.

138 points by pmarin 4y ago

I've done something different before: // main.c #include <module.c> int main() { moduleFunction(); } // module.c #include <stdio.h> void moduleFunction() { printf("calling modules"); } This has its issues (you can't do parallel compilation anymore; you don't have unexported functions, etc), but it works fine for small things. Super helpful when I want to make a small thing and don't want to bother with headers. You just include all the C files in one file and be done with it.

actionfromafar 4y ago
It also automatically enables "global optimisation" in most cases. This style is called "amalgamation" and SQLite is distributed (but not developed) in this way.
- unsafecast 4y ago
  TIL it has a name. Doing what sqlite does gives you the best of both traditional and single header projects I imagine.
- RicoElectrico 4y ago
  Does this impose limitations on the code? I think all functions, even private, should be "namespaced" (i.e. in C - prefixed) then? Kind of awkward, but otherwise amalgamation is a great idea.
  - OskarS 4y ago
    C doesn't have "private" functions in that way, what it has is "static" functions, and this is indeed the biggest limitation of doing this kind of amalgamated build. If you mark a function as static, it means that has internal linkage and is only visible inside the translation unit was defined and no symbol is exported for linking. This is not the default: if you just define a function with some name, that symbol is in general visible to all other translation units that gets linked together by the linker (though with no type information attached, just the name of the symbol. Making sure the type information is correct is the purpose of header files). That means you can have any number of functions marked static in different translation units with the same name, and it's fine, they don't see each other. If you're doing an amalgamated build like this, that no longer works: all symbols have to have globally unique names. In addition, of course, you don't get the "separation" that "static" provides. You can't call a static function across a translation unit boundary, but since this just one big translation unit, anything goes: all functions can call all other functions, even if they shouldn't.
  - arinlen 4y ago
    > Does this impose limitations on the code? It does. Any technique that relies on features specific to translation units, such as taking advantage of internal linkage to reuse symbols to hold data specific to translation units, cannot be used anymore. Also, even though global builds are faster, you forego the ability to do incremental builds which makes working over a bug a more time-consuming task.
- arinlen 4y ago
  > This style is called "amalgamation" and SQLite is distributed (but not developed) in this way. These are actually called "unity builds" or "jumbo builds", and some build systems such as CMake offer support for this. https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.htm...
formerly_proven 4y ago
For smaller C++ projects this tends to be way faster than the conventional wisdom of having many CUs, mostly because all the library bloat will only be parsed and instantiated once.
- kringerkat 4y ago
  Also reduces link time. I find that projects up to around 50K lines compiles instantly as a single file/"unity build" and is my preferred way for most of my projects.
  - arinlen 4y ago
    > Also reduces link time. It's important to stress that it does more than reduce link time: unity builds completely eliminate linking when plugging together submodules, and the final project linking simply has far fewer object files to link.
- Asooka 4y ago
  This really should be solved at the compiler level, rather than forcing people to restructure their projects.
jariRG 4y ago
Afaik this is exactly what Unity builds are. https://en.wikipedia.org/wiki/Unity_build https://en.wikipedia.org/wiki/Unity_build
- unsafecast 4y ago
  Yeah. Though most projects still use header files with unity builds, so I decided not to call them that. It wasn't the concept of a unity build that I was emphasising, it was the idea of not forward-declaring everything.
  - arinlen 4y ago
    > Yeah. Though most projects still use header files with unity builds, so I decided not to call them that. There is nothing inherently special in header files. They are just a convention where you use a specific extension in source files you mostly use to pass declarations. Nothing stops you from using the .c extension on all files, and there is nothing special in passing declarations in one place and definitions in another place.
    - unsafecast 4y ago
      Of course! But, saying 'unity build' puts emphasis on the fact that you compile all the C files 'in unity', in a single CU. What _I_ wanted to put emphasis on is that you can skip the header files, and that coincidentally means you also have to unity build. The fact that most unity built projects still use headers made me hesitant to use the term, as that would put emphasis on building a single CU, as opposed to not forward declaring in headers.
- TingPing 4y ago
  Build systems like Meson can even enable/disable this with an option ootb.
agent281 4y ago
I never really had a good grasp of C. Why is this code novel? It looks like importing a function in any language.
- phaer 4y ago
  It's including another C file, normally you'd just include a "header file" in C and compile each C file separately.
  - usrn 4y ago
    For those who don't use C: header files are more or less an interface description and (usually) don't include any code. The code is added from external objects during the second "linking" step.
  - agent281 4y ago
    Oh, I see: including a header file injects forward declarations into the calling file. Each c file gets compiled separately into object code unless you directly import them like the code above is doing. In that case it essentially dumps the full text of the file into the calling file, which causes serial compilation of a single object file. Unless I'm mistaken that clears some stuff up for me. Thank you!
    - phaer 4y ago
      That's my understanding as well, so I think you are correct but I am no C wizard myself :)
- arinlen 4y ago
  > I never really had a good grasp of C. Why is this code novel? It looks like importing a function in any language. Unity builds are a very old and widely established concept. I dare to speculate they are older than the regular header and source file split, as C required a full blown preprocessor to split code in separate files and also someone had to come up with the include guard trick to get around the limitations of including the same definition multiple times. They aren't employed that often because there are some tradeoffs and in large projects it's benefitial to split them in smaller translation units to limit the scope of incremental builds.

Shadonototra 4y ago

ImportC is shaping up to be one of the greatest feature of D Being able to just import your C code, and call it from C, or when you need to go back to C and resume your work by just importing your D modules I tried it, while it still need some finish, it worked very well! (you had to put your files through a preprocessor though, hopefully this step won't be needed in the future) Every modern language should include a C compiler, it is stupid to give up the entire C ecosystem! No wonder Walter's name is "Bright"! To the people who still doubt about D "because of the GC", it doesn't exist for me, i am making an online 3D game targeting WASM without it, with all the language features! For cases when you need one "if you use D as a scripting language", it will be very helpful to have! https://www.kdom.xyz/ https://www.kdom.xyz/ If you love C, but need something modern with more safeties, give D a try! D is a pragmatic language, very powerful, covers everyone's needs! If you tried it before, now is the time to give it another try!

Tozen 4y ago
Another C alternative that people might want to check out or keep an eye on is Vlang (https://vlang.io/ https://vlang.io/, https://github.com/vlang/v/blob/master/doc/docs.md https://github.com/vlang/v/blob/master/doc/docs.md).
brabel 4y ago
I've been learning Zig and it seems that Zig and D have a lot of things in common. With Zig one can readily import C code as well, and with `c-call-convention` any Zig function can be made callable from C. However, I've run into lots of bugs in Zig so I got a little disillusioned... how do you think D compares with Zig? Is it able to produce as efficient and small binaries, cross-compile to most platforms, metaprogramming etc?
- speps 4y ago
  D's standard lib is also a lot more mature than Zig's, obviously because of its age. I'd recommend D above Zig unless you want to contribute bug fixes to Zig.
  - 4y ago

bachmeier 4y ago

On the topic of the presentation, I'd be happy to have a way to kill the preprocessor. Not that I expect that to happen. Reading C code is like reading literature that only makes sense if you know eight languages, four of which are known to fewer than 100 people.

travisgriggs 4y ago
This differs from web applications how? (not that you said it did, but the implication seems to be that C is an exception in software development in this regard)
- kingaillas 4y ago
  I'm not a webapp dev, but C/C++ always felt unique via the preprocessor functionality that essentially does text substitutions on source code before handing off to the compiler. Actual flags are useful, e.g. -codegenfoo -linkflagbar, but I hate digging through a maze of -DFLIPOPTION1 and -DUSERSUPPLIEDTEXT="hahaha" to unscramble what code the compiler actually sees, and what went wrong because of the various user supplied strings (i.e. #define overwrites). If other languages also have this, well ugh sorry. I have grown to despise it.
Asooka 4y ago
C would be a lot harder to write without the preprocessor. I have only really had one truly awful experience with the preprocessor in the last 15 years and it was when reading OpenSSL's code. Even then, I didn't have too much trouble to grok it with just Vim.
- WalterBright 4y ago
  I discovered my C code was a lot better when I backed out all the clever metaprogramming preprocesser usage. C could be easily extended with: 1. modules 2. manifest constants 3. conditional compilation and the preprocessor can be deprecated.
  - 4y ago

khoobid_shoma 4y ago

The title is a bit misleading. This is about extending C grammar used in D.