4 ms·

The concept of a preprocessor is helpful here. Most modern languages no longer have preprocessors, which is a pity. In cases like these, you want complex comp

The concept of a preprocessor is helpful here. Most modern languages no longer have preprocessors, which is a pity. In cases like these, you want complex computed constants. You want those computed at build time, and not at runtime. I can: * Run the math on a whiteboard, compute +5 or +15, hardcode that, and have the world explode when the code changes * Run the math in code, at runtime, and have the system be slow. * Code up the math, have it run at build time, and have a clear documentation of what the constant is, why, how it's computed, and have it change automatically if the code changes. There were awesome things done in C++ templates back in the day, with some pretty magical performance. The preprocessor would unroll loops to the appropriate level for a given processor, align things to cache boundaries, and similar. I think the reason this might have gone away is that with GHz CPUs, few people care as much about performance anymore. Most code is no longer CPU-bound, but programmer-bound, and lower cognitive complexity is more important than better performance.

scott_s 6y ago
This sort of requirement is met by compile time computations. I know C++11, D and Rust have them, and I imagine many more languages do as well. The need for this sort of thing clearly has not gone away, we're just moving away from a free-form, textual pre-processing stage.
- kenniskrag 6y ago
  constexpr in c++ is pretty magic. Loops, try catch etc. is possible (https://en.cppreference.com/w/cpp/language/constexpr https://en.cppreference.com/w/cpp/language/constexpr)
  - filleokus 6y ago
    One of the few things I was truly amazed by when doing some C++ courses during university. Contrived OOP or extremely cumbersome string manipulation wasn't really my thing. But being able to have compile time evaluation of pretty complicated functions without using macros or anything was so cool. Actually thought about wanting it yesterday in Go
  - midjji 6y ago
    Its truly very nice :) So much horrible preprocessor stuff goes away. Not that preprocessing cant be done right, its just that the standard C/C++ preprocessor language breaks almost every rule of a maintainable code. I love compiler sanity checks of math operations with static asserts, and stuff like compile checking for ub and cdb. constexpr int signed_int_fixed_yet(){ // compiler error return std::limits<int>::max +int(1); } tuplify<PodType>() is wonderful too. Just a shame it adds so much boilerplate though, like why couldnt every function have the same constexpr if possible property as lambdas, and is so damned slow and costly to compile. I also keep writing half of a nice interface, then running into template recursion depth, or gcc failing to compile correctly/(identically to clang). Its also still a bit too limited. Like, consider the case where everything is known compile time: int a=5; // lets say sizeof(a) =4 a<<64; // should be compile error, not ub for(int i=0;i<64;++i) a<<i; // should be compile error, // note that either line above can be removed by the compiler. Leaving a=5; template<class T, int R,int C> Matrix{ std::array<T,RC> data; // constexper // should be automatic T operator(int row, int col){ if is_constexpr_in_context(row) // now wouldnt this be nice... static_assert(row<R,""); // only if row is constexpr argument return data[rowC+col]; } }; Matrix<double,3,4> m; auto a=m(1,2) // row is known compiletime, compiler will warn, should be compile error m(a+3,0) // also known compiletime(array default init to 0), should warn, should be compile error for(int r=0;r<3;++r) for(int c=0;c<6;++c) a+=m(r,c); // oob known compiletime, should warn... Its not like this is that hard either, just transform the statements to use m.at<r,c>() and it works, so its just the compiler identifying constexpr context automatically as for lambda and replacing functions with their argument templated equivalents, and autogenerating these functions. It cant fix everything, but the number of errors people made when using such libs would massively decrease, especially since templated matrix classes are almost universally have visible definitions, and it often applies.
    - kenniskrag 6y ago
      > Its also still a bit too limited. Like, consider the case where everything is known compile time: int a=5; // lets say sizeof(a) =4 a<<64; // should be compile error, not ub for(int i=0;i<64;++i) a<<i; // should be compile error, // note that either line above can be removed by the compiler. Leaving a=5; It is UB for signed integers. There is a lot of stuff from c inside c++ :-).
- jdc 6y ago
  Nim too
Ar-Curunir 6y ago
As the sibling comment noted; you don't preprocessors for these; what you want is compile-time computation. It's much more principled than text-based preprocessing
- woofie11 6y ago
  You're arguing semantics. I never said "text-based preprocessing," I said "preprocessing." I specifically pointed to C++ templates which is a preprocessing layer with is most definitely NOT text-based.
asdfasgasdgasdg 6y ago
I don't think a preprocessor can help in this situation, since what's ultimately being read is the data, and the offsets to be read can't be computed without looking at it.
ben509 6y ago
Option 4: * Code up a unit test that replicates end of buffer scenarios and verifies that X bytes is enough and X-1 bytes fails.
- zabzonk 6y ago
  Reading or writing past the end of a buffer in C is undefined behaviour - you cannot therefore write a test in C to check this, as you have no idea how such a test would behave.
  - lmm 6y ago
    Find a compiler vendor who offers stronger guarantees than the bare minimum that's in the standard, and run the test suite with their compiler. Undefined behaviour was supposed to offer compiler vendors space to improve, not to be the limit of what was implemented.
    - scatters 6y ago
      Actually it's more likely than not that your vendor already offers those guarantees as a compiler mode. In clang and gcc it's the sanitizers (ubsan, asan, tsan and msan) while in MSVC it's RTC and SDL. These build modes catch the vast majority of undefined behavior, and with relatively low performance overhead. If you're not running your tests with sanitizers enabled, you're missing out on a nearly free way to locate and eliminate an entire class of bugs.
  - woofie11 6y ago
    Nope. There are plenty of C/C++ tools which do this sort of test. This was a popular one for a while, back when C++ was still used for major new applications: https://en.wikipedia.org/wiki/BoundsChecker https://en.wikipedia.org/wiki/BoundsChecker There are lots of techniques for this. For example, you can instrument malloc() to allocate 10 extra bytes, put a specific known value there, and confirm it didn't change. There are much fancier ones too, which make clever use of hardware features like MXP. It's a whole research field.
cassepipe 6y ago
Zig got rid of the preprocessor but has comptine
kevin_thibedeau 6y ago
> Most modern languages no longer have preprocessors, which is a pity. m4 is a preprocessor for all languages.
dTal 6y ago
Surely this is what optimizing compilers are for? Just put it in the source code, and if all the information required to calculate the constant is present at compile time - why, then the compiler should calculate it, and optimize away all that hairy math! No need for a separate "preprocessing" step, or a hairy template language.
- woofie11 6y ago
  For the most part, no. Optimizing compilers can't do this. There are whole classes of problems one runs into from: - The ability to change variables by reference or through introspection - Exceptions in intermediate computations - Etc. The set of transformations one can do while maintaining guaranteed equivalence is limited.

scott_s 6y ago

This sort of requirement is met by compile time computations. I know C++11, D and Rust have them, and I imagine many more languages do as well. The need for this sort of thing clearly has not gone away, we're just moving away from a free-form, textual pre-processing stage.

kenniskrag 6y ago
constexpr in c++ is pretty magic. Loops, try catch etc. is possible (https://en.cppreference.com/w/cpp/language/constexpr https://en.cppreference.com/w/cpp/language/constexpr)
- filleokus 6y ago
  One of the few things I was truly amazed by when doing some C++ courses during university. Contrived OOP or extremely cumbersome string manipulation wasn't really my thing. But being able to have compile time evaluation of pretty complicated functions without using macros or anything was so cool. Actually thought about wanting it yesterday in Go
- midjji 6y ago
  Its truly very nice :) So much horrible preprocessor stuff goes away. Not that preprocessing cant be done right, its just that the standard C/C++ preprocessor language breaks almost every rule of a maintainable code. I love compiler sanity checks of math operations with static asserts, and stuff like compile checking for ub and cdb. constexpr int signed_int_fixed_yet(){ // compiler error return std::limits<int>::max +int(1); } tuplify<PodType>() is wonderful too. Just a shame it adds so much boilerplate though, like why couldnt every function have the same constexpr if possible property as lambdas, and is so damned slow and costly to compile. I also keep writing half of a nice interface, then running into template recursion depth, or gcc failing to compile correctly/(identically to clang). Its also still a bit too limited. Like, consider the case where everything is known compile time: int a=5; // lets say sizeof(a) =4 a<<64; // should be compile error, not ub for(int i=0;i<64;++i) a<<i; // should be compile error, // note that either line above can be removed by the compiler. Leaving a=5; template<class T, int R,int C> Matrix{ std::array<T,RC> data; // constexper // should be automatic T operator(int row, int col){ if is_constexpr_in_context(row) // now wouldnt this be nice... static_assert(row<R,""); // only if row is constexpr argument return data[rowC+col]; } }; Matrix<double,3,4> m; auto a=m(1,2) // row is known compiletime, compiler will warn, should be compile error m(a+3,0) // also known compiletime(array default init to 0), should warn, should be compile error for(int r=0;r<3;++r) for(int c=0;c<6;++c) a+=m(r,c); // oob known compiletime, should warn... Its not like this is that hard either, just transform the statements to use m.at<r,c>() and it works, so its just the compiler identifying constexpr context automatically as for lambda and replacing functions with their argument templated equivalents, and autogenerating these functions. It cant fix everything, but the number of errors people made when using such libs would massively decrease, especially since templated matrix classes are almost universally have visible definitions, and it often applies.
  - kenniskrag 6y ago
    > Its also still a bit too limited. Like, consider the case where everything is known compile time: int a=5; // lets say sizeof(a) =4 a<<64; // should be compile error, not ub for(int i=0;i<64;++i) a<<i; // should be compile error, // note that either line above can be removed by the compiler. Leaving a=5; It is UB for signed integers. There is a lot of stuff from c inside c++ :-).
jdc 6y ago
Nim too

Ar-Curunir 6y ago

As the sibling comment noted; you don't preprocessors for these; what you want is compile-time computation. It's much more principled than text-based preprocessing

woofie11 6y ago
You're arguing semantics. I never said "text-based preprocessing," I said "preprocessing." I specifically pointed to C++ templates which is a preprocessing layer with is most definitely NOT text-based.

asdfasgasdgasdg 6y ago

I don't think a preprocessor can help in this situation, since what's ultimately being read is the data, and the offsets to be read can't be computed without looking at it.

ben509 6y ago

Option 4: * Code up a unit test that replicates end of buffer scenarios and verifies that X bytes is enough and X-1 bytes fails.

zabzonk 6y ago
Reading or writing past the end of a buffer in C is undefined behaviour - you cannot therefore write a test in C to check this, as you have no idea how such a test would behave.
- lmm 6y ago
  Find a compiler vendor who offers stronger guarantees than the bare minimum that's in the standard, and run the test suite with their compiler. Undefined behaviour was supposed to offer compiler vendors space to improve, not to be the limit of what was implemented.
  - scatters 6y ago
    Actually it's more likely than not that your vendor already offers those guarantees as a compiler mode. In clang and gcc it's the sanitizers (ubsan, asan, tsan and msan) while in MSVC it's RTC and SDL. These build modes catch the vast majority of undefined behavior, and with relatively low performance overhead. If you're not running your tests with sanitizers enabled, you're missing out on a nearly free way to locate and eliminate an entire class of bugs.
- woofie11 6y ago
  Nope. There are plenty of C/C++ tools which do this sort of test. This was a popular one for a while, back when C++ was still used for major new applications: https://en.wikipedia.org/wiki/BoundsChecker https://en.wikipedia.org/wiki/BoundsChecker There are lots of techniques for this. For example, you can instrument malloc() to allocate 10 extra bytes, put a specific known value there, and confirm it didn't change. There are much fancier ones too, which make clever use of hardware features like MXP. It's a whole research field.

cassepipe 6y ago

Zig got rid of the preprocessor but has comptine

kevin_thibedeau 6y ago

> Most modern languages no longer have preprocessors, which is a pity. m4 is a preprocessor for all languages.

dTal 6y ago

Surely this is what optimizing compilers are for? Just put it in the source code, and if all the information required to calculate the constant is present at compile time - why, then the compiler should calculate it, and optimize away all that hairy math! No need for a separate "preprocessing" step, or a hairy template language.

woofie11 6y ago
For the most part, no. Optimizing compilers can't do this. There are whole classes of problems one runs into from: - The ability to change variables by reference or through introspection - Exceptions in intermediate computations - Etc. The set of transformations one can do while maintaining guaranteed equivalence is limited.