7 ms·

It's such a shame people think languages shouldn't have macros because they're complex. They reduce complexity by shifting custom bespoke syntax out of the lang

by yen223 8y ago

It's such a shame people think languages shouldn't have macros because they're complex. They reduce complexity by shifting custom bespoke syntax out of the language core, and into packages that can be used (or not used) as needed.

joshuamorton 8y ago
Procedural macros aren't macros in the c sense though. They generally are done by acting on an ast (more similar to lisp macros). Macros aren't complex, c preprocessor macros are complex. Although generally, magic that introduces names that aren't defined anywhere is bad, whether it's c macros or a python metaclass, but the metaclass at least has some direct connection to the magic name.
- jstimpfle 8y ago
  > c preprocessor macros are complex At its core the C preprocessor language is a simple token substitution scheme (meaning it strictly operates on the word level). I couldn't write one without looking at the spec because it has some ugly edges. But fundamentally it's a simple technology. It's flexible but that also means it must be used carefully.
  - eridius 8y ago
    The core idea of the C preprocessor is relatively simple. But it has very complex effects on the surrounding language.
    - anticensor 8y ago
      C preprocessor is one of the worst design decisions in C. They could have made the directive system into brace-and-semicolon grammar and not introduce a second syntax.
      - v_lisivka 8y ago
        You can use an other implementation of preprocessor for C/C++, with more features, e.g. DMS[0], which works with AST, or even PHP or Perl, but you will need to parse code twice, so compilation will be about 2x slower. [0]: http://www.semanticdesigns.com/Products/DMS/DMSToolkit.html http://www.semanticdesigns.com/Products/DMS/DMSToolkit.html
        anticensor 8y ago
        In C(++), the preprocessing is only way to portably declare directives. All others are vendor-specific. They are working on a common directive and module system now, but it will take at least two more years to be standardised fully.
        v_lisivka 8y ago
        So just ship you preprocessor with your application/library. Look at cvstrac, for example.
  - GolDDranks 8y ago
    It's not complex by itself, but it has some emergent complexity. (Not unlike some other "simple systems": https://en.wikipedia.org/wiki/Rule_110 https://en.wikipedia.org/wiki/Rule_110) I'd say the main cause of complexity with C preprocessor is that it indiscriminately feeds it's output to a complex system. This means that relative small perturbations such as precence or lack of parentheses can have large and dramatic effects. One could say that a system that works around these by adding safeguards such as hygiene-by-default is more complex by it self, but it definitely begets more controlled behaviour.
    - joshuamorton 8y ago
      Indeed, complexity of the implementation leads to a simpler abstraction/conceptual model/end user experience. This is, almost undoubtedly, a good thing.
      - jstimpfle 8y ago
        I want to disagree. Complexity of implementation is never a good thing. If you cannot truly hide it, bad thing (for the user of the API/language/etc). If you can, it's unwarranted complexity in the first place.
        joshuamorton 8y ago
        This doesn't make sense at all. Abstraction is the act of hiding necessary, but irrelevant, complexity. Timsort is much more complex than quicksort or merge sort. A basic doesn't need to know that they're using timsort instead of mergesort. An advanced user may prefer knowing that timsort will always be faster, but not knowing this doesn't harm them. The same goes for many (most?) performance optimizations. A more complex implementation is often wholly transparent to the API (although I guess you could argue that the abstraction is leaky since the system runs faster), but simultaneously often warranted when applied across all users of a system. The same thing is true with macros that act on an AST instead of raw text. You use a wholly different API (acting on a tree structure instead of a string), but this makes many changes easier, and makes them all safer, at the cost of the API having to handle parsing the language for you. There's no downside to the end user. Only a higher cost of implementation. You're free to disagree, but you should be able to demonstrate specific examples of costs to the end user in those two cases, as well as any others.
  - rtpg 8y ago
    They're complex in the sense that they do not follow the rules of the surrounding system. They can produce mismatching parentheses, breaking the pretty basic notion that expressions can be "inlined" into other expressions. Not knowing whether `f()` is a syntax error or not removes a huge part of the foundation that people use to read code they do not know 100%
    - jstimpfle 8y ago
      > They're complex in the sense that they do not follow the rules of the surrounding system. And this is exactly why they are useful. You can do things with it that you cannot do in any other way. Practical things, I want to mention. Some examples: conditional compilation based on feature set. Automatic insertion of additional arguments (like __FILE__, __LINE__, or the sizeof of arguments). Conversion of identifiers to C strings. Computations such as getting the offset of a member in a struct. Ad-hoc code and datastructure generation. Many of these could be individually replaced with complex features built into the core of the language, by making arbitrary ad-hoc decisions that would be hard to standardize and would probably kill the language. > Not knowing whether `f()` is a syntax error or not removes a huge part of the foundation that people use to read code they do not know 100% It's your responsibility to match parentheses when required (which is almost always). That's easy for a macro that is usually 1, or at most 5 lines. And if something goes wrong, it's usually not that hard to find the error and fix it. You need to be aware of some gotchas though (wrap expressions in parens, don't do naked if statements or blocks but wrap them in do..while(0)). That means you will get to learn the C syntax more intimately.
- nothrabannosir 8y ago
  I personally agree, but parent comment has a point: “ast macros are complex” or “cause complexity” is a common criticism leveled against Lisp. This is completely disregarding C and its token macros.
kodablah 8y ago
It's not just complexity, it's compilation time/effort. I personally see no problem with it and am also happy my build.rs script can automatically do logic like code generation at build time. However, I completely understand a Go approach that would never consider such an automatic "go generate".
- staticassertion 8y ago
  > However, I completely understand a Go approach that would never consider such an automatic "go generate". These end up existing anyways though. At least a few places are using Go with their own preprocessor - isn't Kubernetes doing this?
- yen223 8y ago
  My point is exactly that you can opt out of using macros if you so wish, if compilation time were an issue.
  - kodablah 8y ago
    You have less choice as your transitive dependency tree becomes larger while leveraging the popular libraries in the ecosystem.
pjmlp 8y ago
Because most of the time they represent a huge debugging effort. For example, how do you step a macro expansion in Rust at any given point in a source level debugger? Yep, you don't. Even on Lisp, if we want something more developer friendly than (macroexpand) the only option are the commercial Common Lisp environments. Having said this, I also do like having macro support around, provided it gets used judiciously.
- iopq 8y ago
  > For example, how do you step a macro expansion in Rust at any given point in a source level debugger? > Yep, you don't. I don't see any technical reason why that can't be done
  - pjmlp 8y ago
    I provided the example that you have such features in commercial Common Lisp environments. The point it that such kind of tooling is generally not available in FOSS offerings. Yes it can be done, but it isn't currently available.
- bluejekyll 8y ago
  This isn’t what you want, but I do want to make sure that people know that there are tools available for at least viewing the expansion: https://github.com/dtolnay/cargo-expand/blob/master/README.md https://github.com/dtolnay/cargo-expand/blob/master/README.m...
  - pjmlp 8y ago
    Thanks, it looks interesting. Any plans to make it work on stable?
    - steveklabnik 8y ago
      It doens't have a ton of pressure to, given that it's a tool. You write "cargo +nightly" instead of "cargo" and it's fine. We'd of course like to have it on stable, like everything, there's just more pressing stuff to stabilize.

by yen223 8y ago

joshuamorton 8y ago

Procedural macros aren't macros in the c sense though. They generally are done by acting on an ast (more similar to lisp macros). Macros aren't complex, c preprocessor macros are complex. Although generally, magic that introduces names that aren't defined anywhere is bad, whether it's c macros or a python metaclass, but the metaclass at least has some direct connection to the magic name.

jstimpfle 8y ago
> c preprocessor macros are complex At its core the C preprocessor language is a simple token substitution scheme (meaning it strictly operates on the word level). I couldn't write one without looking at the spec because it has some ugly edges. But fundamentally it's a simple technology. It's flexible but that also means it must be used carefully.
- eridius 8y ago
  The core idea of the C preprocessor is relatively simple. But it has very complex effects on the surrounding language.
  - anticensor 8y ago
    C preprocessor is one of the worst design decisions in C. They could have made the directive system into brace-and-semicolon grammar and not introduce a second syntax.
    - v_lisivka 8y ago
      You can use an other implementation of preprocessor for C/C++, with more features, e.g. DMS[0], which works with AST, or even PHP or Perl, but you will need to parse code twice, so compilation will be about 2x slower. [0]: http://www.semanticdesigns.com/Products/DMS/DMSToolkit.html http://www.semanticdesigns.com/Products/DMS/DMSToolkit.html
      - anticensor 8y ago
        In C(++), the preprocessing is only way to portably declare directives. All others are vendor-specific. They are working on a common directive and module system now, but it will take at least two more years to be standardised fully.
        v_lisivka 8y ago
        So just ship you preprocessor with your application/library. Look at cvstrac, for example.
- GolDDranks 8y ago
  It's not complex by itself, but it has some emergent complexity. (Not unlike some other "simple systems": https://en.wikipedia.org/wiki/Rule_110 https://en.wikipedia.org/wiki/Rule_110) I'd say the main cause of complexity with C preprocessor is that it indiscriminately feeds it's output to a complex system. This means that relative small perturbations such as precence or lack of parentheses can have large and dramatic effects. One could say that a system that works around these by adding safeguards such as hygiene-by-default is more complex by it self, but it definitely begets more controlled behaviour.
  - joshuamorton 8y ago
    Indeed, complexity of the implementation leads to a simpler abstraction/conceptual model/end user experience. This is, almost undoubtedly, a good thing.
    - jstimpfle 8y ago
      I want to disagree. Complexity of implementation is never a good thing. If you cannot truly hide it, bad thing (for the user of the API/language/etc). If you can, it's unwarranted complexity in the first place.
      - joshuamorton 8y ago
        This doesn't make sense at all. Abstraction is the act of hiding necessary, but irrelevant, complexity. Timsort is much more complex than quicksort or merge sort. A basic doesn't need to know that they're using timsort instead of mergesort. An advanced user may prefer knowing that timsort will always be faster, but not knowing this doesn't harm them. The same goes for many (most?) performance optimizations. A more complex implementation is often wholly transparent to the API (although I guess you could argue that the abstraction is leaky since the system runs faster), but simultaneously often warranted when applied across all users of a system. The same thing is true with macros that act on an AST instead of raw text. You use a wholly different API (acting on a tree structure instead of a string), but this makes many changes easier, and makes them all safer, at the cost of the API having to handle parsing the language for you. There's no downside to the end user. Only a higher cost of implementation. You're free to disagree, but you should be able to demonstrate specific examples of costs to the end user in those two cases, as well as any others.
- rtpg 8y ago
  They're complex in the sense that they do not follow the rules of the surrounding system. They can produce mismatching parentheses, breaking the pretty basic notion that expressions can be "inlined" into other expressions. Not knowing whether `f()` is a syntax error or not removes a huge part of the foundation that people use to read code they do not know 100%
  - jstimpfle 8y ago
    > They're complex in the sense that they do not follow the rules of the surrounding system. And this is exactly why they are useful. You can do things with it that you cannot do in any other way. Practical things, I want to mention. Some examples: conditional compilation based on feature set. Automatic insertion of additional arguments (like __FILE__, __LINE__, or the sizeof of arguments). Conversion of identifiers to C strings. Computations such as getting the offset of a member in a struct. Ad-hoc code and datastructure generation. Many of these could be individually replaced with complex features built into the core of the language, by making arbitrary ad-hoc decisions that would be hard to standardize and would probably kill the language. > Not knowing whether `f()` is a syntax error or not removes a huge part of the foundation that people use to read code they do not know 100% It's your responsibility to match parentheses when required (which is almost always). That's easy for a macro that is usually 1, or at most 5 lines. And if something goes wrong, it's usually not that hard to find the error and fix it. You need to be aware of some gotchas though (wrap expressions in parens, don't do naked if statements or blocks but wrap them in do..while(0)). That means you will get to learn the C syntax more intimately.
nothrabannosir 8y ago
I personally agree, but parent comment has a point: “ast macros are complex” or “cause complexity” is a common criticism leveled against Lisp. This is completely disregarding C and its token macros.

kodablah 8y ago

It's not just complexity, it's compilation time/effort. I personally see no problem with it and am also happy my build.rs script can automatically do logic like code generation at build time. However, I completely understand a Go approach that would never consider such an automatic "go generate".

staticassertion 8y ago
> However, I completely understand a Go approach that would never consider such an automatic "go generate". These end up existing anyways though. At least a few places are using Go with their own preprocessor - isn't Kubernetes doing this?
yen223 8y ago
My point is exactly that you can opt out of using macros if you so wish, if compilation time were an issue.
- kodablah 8y ago
  You have less choice as your transitive dependency tree becomes larger while leveraging the popular libraries in the ecosystem.

pjmlp 8y ago

Because most of the time they represent a huge debugging effort. For example, how do you step a macro expansion in Rust at any given point in a source level debugger? Yep, you don't. Even on Lisp, if we want something more developer friendly than (macroexpand) the only option are the commercial Common Lisp environments. Having said this, I also do like having macro support around, provided it gets used judiciously.

iopq 8y ago
> For example, how do you step a macro expansion in Rust at any given point in a source level debugger? > Yep, you don't. I don't see any technical reason why that can't be done
- pjmlp 8y ago
  I provided the example that you have such features in commercial Common Lisp environments. The point it that such kind of tooling is generally not available in FOSS offerings. Yes it can be done, but it isn't currently available.
bluejekyll 8y ago
This isn’t what you want, but I do want to make sure that people know that there are tools available for at least viewing the expansion: https://github.com/dtolnay/cargo-expand/blob/master/README.md https://github.com/dtolnay/cargo-expand/blob/master/README.m...
- pjmlp 8y ago
  Thanks, it looks interesting. Any plans to make it work on stable?
  - steveklabnik 8y ago
    It doens't have a ton of pressure to, given that it's a tool. You write "cargo +nightly" instead of "cargo" and it's fine. We'd of course like to have it on stable, like everything, there's just more pressing stuff to stabilize.