6 ms·

I write a lot of C. I'm somewhat prone to writing things in C that shouldn't be. There are two somewhat unusual things that make that more viable than it sounds

I write a lot of C. I'm somewhat prone to writing things in C that shouldn't be. There are two somewhat unusual things that make that more viable than it sounds. One, write unit tests in the same source as the implementation. Tests that run through the interface are good too and can be in separate source files, maybe in another directory. Fine grain poke at the internals of static functions really should be written as well and those are best placed where they can call said static functions directly. Make adding tests really low friction and write lots of them. Two, embrace code generators. I've seen it written that C++ solves problems by extending the language and C solves problems by writing more C and that sounds about right. Make adding a code generator as low friction as adding a C source file. It should take seconds to convert an existing source file into a generator that recreates said existing source file. People love ceremony. All the source files are written in lists in cmake files. All their build dependencies are written down in the cmake files as well. All the tests are written in list of tests to run. Every code generator is introduced as an independent project with it's own documentation and careful integration into the build systems. I love writing C because I do none of that overhead. The files to compile are whatever is on disk. The tests to run are all the tests in the source, they aren't listed somewhere else as well. Any file called *.lua.c is a lua program that writes the intended contents of *.c to stdout. *.c.h will be compiled as C and then run to write a C header on stdout. Convention over configuration with a language that requires zero cognitive overhead to write.

memoryperson10 3y ago
Wild, I was just thinking about how I wanted to do reflection on C structs using LuaJIT FFI for serialization.
- JonChesterfield 3y ago
  It's worth considering the other direction - write the struct definition in json or xml, or dicts in python source or whatever, and generate the C structs and the associated functions and data from that. Go slightly further and write out function types like that and you get something equivalent to swig, where instead of parsing C and trying to emit wrappers for other languages from that framework, you have the data in native lua tables that you emit code from.
bluetomcat 3y ago
In procedural C land, unit testing is nowhere near as relevant as it is in memory-safe OO lands. Assertions are far more relevant. You want to detect illegal program states in development as early as possible. In C11, you also have static_assert for things like "is this static lookup table as big as needed".
- monsieurbanana 3y ago
  I get why assertions would be more important than in memory safe languages, but why are unit tests less important in C than in higher-level languages? Or you just mean relative to assertions? (which to me don't really influence each other in that way)
  - bluetomcat 3y ago
    Because the coding style is different. The basic unit of code is the procedure, not abstract entities like DateTimeCalculator or EmployeePayrollManager. Procedures are sensitive to the context where they are called. A procedure calling another procedure usually takes care to provide valid input data. Robust error-checking must be in place, or your program is unsafe. A unit test wouldn't know the calling context of procedures deep in the call graph, and cannot detect unsafe but seemingly working code.
    - vkazanov 3y ago
      This is more of a culture surrounding the language than anything else. Modern C avoids global state, prefers directly specified context and even allocation is external to the function's logic. And testing functions is easy. This is as opposed to procedures, I.e. functions that operate on global state with fragile conext assumptions, which almost impossible to replicate in test environments.
    - lelanthran 3y ago
      > The basic unit of code is the procedure, not abstract entities like DateTimeCalculator or EmployeePayrollManager. I agree, but in almost any non-trivial well-run C project the basic unit is going to be datatypes, opaque pointers, modules, etc. In that case I generally throw in a `bool MyObjectType_test(void)` into the module so, in some sense, I'll have some basic sort of testing. Not as nice as the OO languages provide, but enough to give myself some confidence that changes don't introduce blatant bugs.
iainmerrick 3y ago
People love ceremony. All the source files are written in lists in cmake files. [...] I love writing C because I do none of that overhead. The files to compile are whatever is on disk. I couldn't agree more. I can't stand build tools that don't take globs! I shouldn't need to list the same thing in multiple places just to make it work. Almost all the time, the files on disk should be the only list needed.
- ndiddy 3y ago
  The reason why people write all the source files in a list in a cmake file is that it means that when one developer adds a new file, everyone else's build system automatically regenerates the next time they try to build the project. When you use globs (which cmake does support), it means that the CMakeLists.txt file doesn't get modified when adding a new file, so the cmake generated makefiles/ninjafiles/whatever have no way of knowing that the build was modified. Instead of having the build system regenerate and then the build go through, they get a linker error and have to manually run "cmake -B build" or whatever to fix it.
  - Filligree 3y ago
    Somehow this isn't a problem with Cargo.toml, however. What's cmake missing?
    - staunton 3y ago
      > What's cmake missing? Where to begin... Maybe let's start with "a build system". (I'm serious, CMake isn't a build system and doesn't have one. It supports a few...)
      - madduci 3y ago
        What are the alternatives?
        staunton 3y ago
rwbt 3y ago
I'm curious how your build process works with the codegen approach. Do you have the codegen run a pass everytime you build the project? Or is it a manual process? I've always wanted to use Lua to generate boilerplate C code (like typesafe vectors) instead of relying on macros, so might actually start thinking in this direction going forward.
- turtledragonfly 3y ago
  Not the person you asked, but this sort of thing is pretty easy with templating languages, such as Perl's Template::Toolkit[1] or Python's Jinja2[2]. The advantage of a system like that over a plain programming language (such as Lua you mentioned) is that there's less escaping and string.format(xxx) sorts of things. The output text is the primary thing and the templating language has its own special syntax to "activate" it when you need it, rather than the other way around. That being said, Lua's square-bracket strings are pretty good for bulk text, too (: [1] http://template-toolkit.org/about.html http://template-toolkit.org/about.html [2] https://jinja.palletsprojects.com/en/3.1.x/templates/#synopsis https://jinja.palletsprojects.com/en/3.1.x/templates/#synops...
  - rwbt 3y ago
    Template-Toolkit seems like the right tool for the job. Thanks for sharing.
- JonChesterfield 3y ago
  Instead of code generation as a special case, let it be the default. That kills most of the complexity. src/ contains lots of code in lots of languages, all describing code generators of varying complexity. src/foo.c uses 'cp' as the code generator. src/bar.c.py uses 'python'. gen/ contains the result of the ad hoc code generators. Same tree layout as src, one to one map from those files. All of that is C. Files that get #included and don't make sense standalone I tend to suffix .data, e.g. src/table.data.lua runs to create gen/table.data. obj/ contains all the C compiled to object code or llvm bitcode, depending on the toolchain. I originally did that with the idea that I could distribute gen/ as the source code for people who wanted to build the project without dealing with the complexity of the build system, strongly inspired by sqlite's amalgamation. Sqlite is built from a chaotic collection of TCL and C code generators but you don't see that as a user of sqlite.c. It turns out that debugging the stuff under gen/ when things go wrong is really easy. Valgrind / gdb show you the boring code that the generators stamped out. So in practice I keep that separation because it makes it easier to trace through the system when it behaves unexpectedly. Lua does that really well. Good multiline string literal support. A template string that you call gsub() on a few times then dump to stdout is very quick to put together. You've got string format for more complicated things, maybe using one of the string interpolation implementations found on their wiki.
hgs3 3y ago
> embrace code generators. You also don't need to limit yourself to the C pre-processor. There are other dedicated pre-processing languages, like M4, and templating languages you can use.
- bch 3y ago
  Sort of happy to see m4 mentioned, but for the uninitiated: m4 is the receiver of an awful lot of hate, because things can get unwieldy quickly. Go in with your eyes open.
- JonChesterfield 3y ago
  Definitely don't limit yourself to the C preprocessor. That way lies a tarpit. M4 took several months off me, it can definitely be used to do things but maybe shouldn't be. A really bad play was generating C++ from cmake as a compromise that delighted noone. Python is good at generating C source. I changed to mostly using lua five years ago as python climbed the complexity curve. Sometimes the right thing is C itself, e.g. laying out data based on sizes of various C types. Currently I'm trying to use xslt to emit syntax trees that pretty print to get the generated source in what is probably a dead end.
beamsen 3y ago
> I love writing C because I do none of that overhead. The files to compile are whatever is on disk. The tests to run are all the tests in the source, they aren't listed somewhere else as well. Any file called .lua.c is a lua program that writes the intended contents of .c to stdout. *.c.h will be compiled as C and then run to write a C header on stdout. Do you have any sample projects that showcase both the tests and the code generation? I'd love to take a look.
- JonChesterfield 3y ago
  The tests look much like https://github.com/JonChesterfield/EvilUnit/blob/master/evilunit_string_equal.i https://github.com/JonChesterfield/EvilUnit/blob/master/evil.... That's from the self tests for the test framework. It's syntactically modelled on catch2. The name is because I built the thing out of preprocessor macros to run on freestanding c89. The projects using code generation in that fashion are proprietary. There's a makefile at https://github.com/JonChesterfield/boring-makefile/blob/master/src/makefile https://github.com/JonChesterfield/boring-makefile/blob/mast... set up to do the src/gen/obj codegen by default structure. Just made the ad hoc xml codegen hackery public, some examples of tests written a couple of days ago at https://github.com/JonChesterfield/xml/blob/main/tools/intset.c https://github.com/JonChesterfield/xml/blob/main/tools/intse...

memoryperson10 3y ago

Wild, I was just thinking about how I wanted to do reflection on C structs using LuaJIT FFI for serialization.

JonChesterfield 3y ago
It's worth considering the other direction - write the struct definition in json or xml, or dicts in python source or whatever, and generate the C structs and the associated functions and data from that. Go slightly further and write out function types like that and you get something equivalent to swig, where instead of parsing C and trying to emit wrappers for other languages from that framework, you have the data in native lua tables that you emit code from.

bluetomcat 3y ago

In procedural C land, unit testing is nowhere near as relevant as it is in memory-safe OO lands. Assertions are far more relevant. You want to detect illegal program states in development as early as possible. In C11, you also have static_assert for things like "is this static lookup table as big as needed".

monsieurbanana 3y ago
I get why assertions would be more important than in memory safe languages, but why are unit tests less important in C than in higher-level languages? Or you just mean relative to assertions? (which to me don't really influence each other in that way)
- bluetomcat 3y ago
  Because the coding style is different. The basic unit of code is the procedure, not abstract entities like DateTimeCalculator or EmployeePayrollManager. Procedures are sensitive to the context where they are called. A procedure calling another procedure usually takes care to provide valid input data. Robust error-checking must be in place, or your program is unsafe. A unit test wouldn't know the calling context of procedures deep in the call graph, and cannot detect unsafe but seemingly working code.
  - vkazanov 3y ago
    This is more of a culture surrounding the language than anything else. Modern C avoids global state, prefers directly specified context and even allocation is external to the function's logic. And testing functions is easy. This is as opposed to procedures, I.e. functions that operate on global state with fragile conext assumptions, which almost impossible to replicate in test environments.
  - lelanthran 3y ago
    > The basic unit of code is the procedure, not abstract entities like DateTimeCalculator or EmployeePayrollManager. I agree, but in almost any non-trivial well-run C project the basic unit is going to be datatypes, opaque pointers, modules, etc. In that case I generally throw in a `bool MyObjectType_test(void)` into the module so, in some sense, I'll have some basic sort of testing. Not as nice as the OO languages provide, but enough to give myself some confidence that changes don't introduce blatant bugs.

iainmerrick 3y ago

People love ceremony. All the source files are written in lists in cmake files. [...] I love writing C because I do none of that overhead. The files to compile are whatever is on disk. I couldn't agree more. I can't stand build tools that don't take globs! I shouldn't need to list the same thing in multiple places just to make it work. Almost all the time, the files on disk should be the only list needed.

ndiddy 3y ago
The reason why people write all the source files in a list in a cmake file is that it means that when one developer adds a new file, everyone else's build system automatically regenerates the next time they try to build the project. When you use globs (which cmake does support), it means that the CMakeLists.txt file doesn't get modified when adding a new file, so the cmake generated makefiles/ninjafiles/whatever have no way of knowing that the build was modified. Instead of having the build system regenerate and then the build go through, they get a linker error and have to manually run "cmake -B build" or whatever to fix it.
- Filligree 3y ago
  Somehow this isn't a problem with Cargo.toml, however. What's cmake missing?
  - staunton 3y ago
    > What's cmake missing? Where to begin... Maybe let's start with "a build system". (I'm serious, CMake isn't a build system and doesn't have one. It supports a few...)
    - madduci 3y ago
      What are the alternatives?
      - staunton 3y ago

rwbt 3y ago

I'm curious how your build process works with the codegen approach. Do you have the codegen run a pass everytime you build the project? Or is it a manual process? I've always wanted to use Lua to generate boilerplate C code (like typesafe vectors) instead of relying on macros, so might actually start thinking in this direction going forward.

turtledragonfly 3y ago
Not the person you asked, but this sort of thing is pretty easy with templating languages, such as Perl's Template::Toolkit[1] or Python's Jinja2[2]. The advantage of a system like that over a plain programming language (such as Lua you mentioned) is that there's less escaping and string.format(xxx) sorts of things. The output text is the primary thing and the templating language has its own special syntax to "activate" it when you need it, rather than the other way around. That being said, Lua's square-bracket strings are pretty good for bulk text, too (: [1] http://template-toolkit.org/about.html http://template-toolkit.org/about.html [2] https://jinja.palletsprojects.com/en/3.1.x/templates/#synopsis https://jinja.palletsprojects.com/en/3.1.x/templates/#synops...
- rwbt 3y ago
  Template-Toolkit seems like the right tool for the job. Thanks for sharing.
JonChesterfield 3y ago
Instead of code generation as a special case, let it be the default. That kills most of the complexity. src/ contains lots of code in lots of languages, all describing code generators of varying complexity. src/foo.c uses 'cp' as the code generator. src/bar.c.py uses 'python'. gen/ contains the result of the ad hoc code generators. Same tree layout as src, one to one map from those files. All of that is C. Files that get #included and don't make sense standalone I tend to suffix .data, e.g. src/table.data.lua runs to create gen/table.data. obj/ contains all the C compiled to object code or llvm bitcode, depending on the toolchain. I originally did that with the idea that I could distribute gen/ as the source code for people who wanted to build the project without dealing with the complexity of the build system, strongly inspired by sqlite's amalgamation. Sqlite is built from a chaotic collection of TCL and C code generators but you don't see that as a user of sqlite.c. It turns out that debugging the stuff under gen/ when things go wrong is really easy. Valgrind / gdb show you the boring code that the generators stamped out. So in practice I keep that separation because it makes it easier to trace through the system when it behaves unexpectedly. Lua does that really well. Good multiline string literal support. A template string that you call gsub() on a few times then dump to stdout is very quick to put together. You've got string format for more complicated things, maybe using one of the string interpolation implementations found on their wiki.

hgs3 3y ago

> embrace code generators. You also don't need to limit yourself to the C pre-processor. There are other dedicated pre-processing languages, like M4, and templating languages you can use.

bch 3y ago
Sort of happy to see m4 mentioned, but for the uninitiated: m4 is the receiver of an awful lot of hate, because things can get unwieldy quickly. Go in with your eyes open.
JonChesterfield 3y ago
Definitely don't limit yourself to the C preprocessor. That way lies a tarpit. M4 took several months off me, it can definitely be used to do things but maybe shouldn't be. A really bad play was generating C++ from cmake as a compromise that delighted noone. Python is good at generating C source. I changed to mostly using lua five years ago as python climbed the complexity curve. Sometimes the right thing is C itself, e.g. laying out data based on sizes of various C types. Currently I'm trying to use xslt to emit syntax trees that pretty print to get the generated source in what is probably a dead end.

beamsen 3y ago

> I love writing C because I do none of that overhead. The files to compile are whatever is on disk. The tests to run are all the tests in the source, they aren't listed somewhere else as well. Any file called .lua.c is a lua program that writes the intended contents of .c to stdout. *.c.h will be compiled as C and then run to write a C header on stdout. Do you have any sample projects that showcase both the tests and the code generation? I'd love to take a look.

JonChesterfield 3y ago
The tests look much like https://github.com/JonChesterfield/EvilUnit/blob/master/evilunit_string_equal.i https://github.com/JonChesterfield/EvilUnit/blob/master/evil.... That's from the self tests for the test framework. It's syntactically modelled on catch2. The name is because I built the thing out of preprocessor macros to run on freestanding c89. The projects using code generation in that fashion are proprietary. There's a makefile at https://github.com/JonChesterfield/boring-makefile/blob/master/src/makefile https://github.com/JonChesterfield/boring-makefile/blob/mast... set up to do the src/gen/obj codegen by default structure. Just made the ad hoc xml codegen hackery public, some examples of tests written a couple of days ago at https://github.com/JonChesterfield/xml/blob/main/tools/intset.c https://github.com/JonChesterfield/xml/blob/main/tools/intse...