9 ms·

Notes on the M4 Macro Language (2008)

92 points by luu 7y ago

Koshkin 7y ago
The original report: https://wolfram.schneider.org/bsd/7thEdManVol2/m4/m4.pdf https://wolfram.schneider.org/bsd/7thEdManVol2/m4/m4.pdf
brodouevencode 7y ago
I'm curious if anyone is still using this tool today. Back when I used it (~8-ish years ago) I found it to be a godsend.
- tannhaeuser 7y ago
  It's used by autotools, and about 15 years ago I used it to generate BIND (dns server) configs, and sendmail.conf files, too. But I'd rather not use if I don't have to :) m4 is still POSIX, though, and maybe someone can elaborate on m4 and tell us about it's strength in times of JSON and (ya|to)ml configs, which I also don't like, perhaps because they're misnamed IMO as "markup languages".
  - brodouevencode 7y ago
    This too is one of the ways I was using it back then. Working in a small CMS shop we were responsible for hosting Drupal sites with custom DNS, and all that. We used m4 to build out the BIND configs, Drupal config files, logging configs, and Apache configs.
  - cryptonector 7y ago
    For JSON, see jq (https://stedolan.github.io/jq https://stedolan.github.io/jq).
- enriquto 7y ago
  I do, from time to time. It's nice to have a preprocessor for normal text files. Yet, most often I find it more practical to use sed or awk for my needs.
- adrianN 7y ago
  I use it as a simple static site generator and for generating C++. The thing I dislike most is that it uses ` and ' for quoting and there is no easy way of escaping quotes.
  - kevin_thibedeau 7y ago
    You can change the quote chars. This is why autotools use [].
- kevin_thibedeau 7y ago
  I wrote a large package of macros for a niche assembler using M4. It has some sophisticated things like a code generator for non-infix expressions. M4 was a blessing and a curse. It is very powerful and saved me from implementing my own macro language but debugging is not for the faint of heart once you get into anything complicated. https://github.com/kevinpt/opbasm/blob/master/opbasm/picoblaze.m4 https://github.com/kevinpt/opbasm/blob/master/opbasm/picobla... Michael Breen's notes are indispensable documentation on how to make effective use of the language.
  - throwaway_pdp09 7y ago
    I'm a fan of M4, and I don't know what these are doing but it surely looks too complex. You have defines within defines and OMG just an extract .. (just a snip follows) `define(`_'''$`'1```_LENGTH',strlenc(''$`'2``))'dnl `; "''$`'2``"' `ifdef(`PB3', ''$`'1```: calltable($4, $1, estr('''$`'2```))dnl return',dnl `table '''$`'1```#, [dec2pbhex(cstr('''$`'2```))] '''$`'1```: loadaddr($2, $3, _'''$`'1```_STR) jump __`'$5`'_handler _'''$`'1```_STR: load&return $1, '''$`'1```#')' )')' I have to ask if this is right, or whether a parser would have been a whole lot less work. Credit for your skillz, but that looks like so much pain.
    - kevin_thibedeau 7y ago
      These are macros that generate user configurable codegen macros. I couldn't hard code register assignments so everything dynamically generated is prepared with these monstrosities. They also can't have fixed names since some need to be invoked multiple times with different parameters. FWIW I didn't intend for things to go this far. I just started with some simple macros and it snowballed.
- mauvehaus 7y ago
  Some years ago at a former employer I used to for generating some Java 5 code. I needed (as I hazily recall) an interface of accessors, immutable implementation, and a factory for the same couple dozen members. It was a lot of boilerplate and the list was changing a fair amount over the course of development. It worked great, and I got it plugged into the ant build without too much difficulty. I'm told the person who took over that codebase was equal parts perplexed and horrified by it. They ripped out the m4 code generation and just put the generated files directly in the source tree. As the codebase had stabilized, the purpose that m4 served was basically gone. So, not still in use, but if I found myself in the same situation today, I'd explore other options and consider doing the same thing. Interface default methods might cover some of the need at this point, but it's so far back, I'm hazy on the details. The nice thing about doing this sort of code generation with m4 rather than the C preprocessor is that m4 preserves formatting (like line breaks) a lot better. Even if you're not using any of the other functionality of m4 (and there's a lot), that alone makes figuring out where things went wrong easier.
- jiggerypokery 7y ago
  I work for a telecom where we've used it to generate network configs for a long time. We just got rid of it because it's impossible to hire people who know it and nobody (new hire or existing people) want to learn it either
- kd5bjo 7y ago
  I dusted it off this semester to help with some complicated reports I needed to make. I needed to include lots of graphs from simulation runs with various parameters. I encoded the parameter values in the graphs’ filenames, and used m4 to write the corresponding gnuplot control file. After tying everything together with make, I could include plots for any parameter as I was writing the report and have the relevant data generated as necessary. All without re-running simulations that had already produced results.
- spsrich2 7y ago
  not today but I used m4 in the late 80s to generate appropriate makefiles from a single source to allow me to compile my programs on HP-ux, Apollo Aegis, SunOS, AIX, etc. Happy days.
- vbezhenar 7y ago
  Used it for some C code, because CPP was not able to accomplish what I wanted. m4 was more powerful and I was able to finish my task. Thankfully I never used it in context of sendmail or autotools, so no reason to hate it for me, I guess.
sigmonsays 7y ago
I always remember m4 encounters from autoconf struggles of building things from source a long time ago (slackware 6). Back when I never cared about m4, Today, I find it interesting to understand what problem its trying to solve. I dont feel like its practical anymore but I think it is important to not discard use of old tools and understand them.
- jolmg 7y ago
  https://en.wikipedia.org/wiki/M4_(computer_language)#History https://en.wikipedia.org/wiki/M4_(computer_language)#History I see it as a more powerful version of CPP. It might be somewhat based on it. CPP seems to have been developed in 1972-3[1], while M4 was developed in 1977. Today, I think we're using template systems like Mustache, Handlebars, ERB, Jinja, etc. where otherwise macro systems would be used. The difference between macro systems / preprocessors and templates is that the latter comes with syntax (like <%= %> for ERB) that always makes it clear what's handled by the template system. With M4 and CPP, it's implicit and instead you have syntax (`' or []) to exclude stuff from being handled by the macro system. I think that change is because, before, macro systems were meant to be used mainly with programming languages to avoid repetition. Then, programming languages became concise and more flexible, so they didn't need to be handled by another language like C has CPP. Our need changed to having to avoid repetition with data files like HTML. Perhaps having implicit expansion when used with an underlying programming language is not so much an issue, or at least it could be handled with a convention like how CPP variables and functions are normally capitalized. With data files, I suppose being able to differentiate between what's HTML and what's ERB was probably more important. [1] https://softwareengineering.stackexchange.com/questions/309389/what-is-the-origin-of-the-c-preprocessor https://softwareengineering.stackexchange.com/questions/3093...
  - kd5bjo 7y ago
    The pattern I’ve fallen into with m4 is to have a preamble that sets up whatever definitions I’m going to need, redefine the quotes “backwards”, and then place a copy of the new close quote on its own line to start the main body of the file. It then acts very similarly to how you describe templating languages, where you mark the parts that should be parsed instead of the parts that shouldn’t.
otterley 7y ago
(2008)
seemslegit 7y ago
Deprecate it with fire.
teddyh 7y ago
M4 has some well-known uses like Sendmail and GNU Autoconf, but I don’t think this can be counted in M4’s favor: For operating in a standard Unix-like environment, there is really no good alternative to m4 for generic text macro expansion. You can try to force cpp – the C preprocessor – into this, but it is not suited to not being part of a C compiler pipeline. For better or for worse, m4 is it in the world of Unix and shell tools, even though its syntax is quite annoying to use. Of course, if you operate within a language ecosystem, you can probably find a nice templating system there. Python, for instance, has Jinja2. Or, if your needs are simple, you can use something more basic, like sed(1).
- downerending 7y ago
  I've looked at m4 before, and want to love it, but it just seems a bit too clumsy to use. And unfortunately, IIRC, unlike sed it's not ubiquitously available. Not really crazy about Jinja2 either, but at this point it seems like it has a lot more mindshare and maintenance.
  - froh 7y ago
    M4 as part of posix should be as widely available as sed, no?
    - downerending 7y ago
      In principle, yes. In practice, it seems like it's often not installed on the random hosts I encounter. This isn't a fatal flaw, but it is a small advantage in favor of sed and awk and probably even perl, which are essentially ubiquitous these days.
- PaulHoule 7y ago
  Last time I used M4 it was for a multi language system where I needed to substitute in PHP, shell scripts, apache conf files, etc.
- Blackthorn 7y ago
  Mustache / Handlebars is pretty widely available throughout languages these days too and is pretty basic. Doesn't help you at the default posix level, but there are even command line versions IIRC
deleted 7y ago
[deleted]
scythe 7y ago
I wonder if the "quoting" problem could be fixed by making m4 recognize Lua-style longstrings, which never need escape characters. Some examples: http://lua-users.org/wiki/StringsTutorial http://lua-users.org/wiki/StringsTutorial I considered using longstrings to make a "better CSV" once but then I realized that quoting is only one of the many problems with CSV.
- iamthad 7y ago
  It's not clear to me what you mean by "longstrings" from that page. Are you referring to the double-bracket quoted style?
  - scythe 7y ago
    Yes. A Lua-style longstring is delimited by [===[ and ]===] where the number of = signs can be arbitrary (including zero); the end must have the same number of = signs as the beginning. Any string can be encoded inside of a longstring without escaping, simply by using long enough delimiters. There are a number of advantages to this notation: it's more readable (usually), it generally results in shorter code files, and longquoted substrings can be directly addressed without translation (though there may be other reasons not to pass that pointer around).
- luizfelberti 7y ago
  I once developed a tiny single-file bash script to do dumb string substitution for arbitrary files and arbitrary strings, and I used this exact Lua mechanic to avoid escaping issues if the delimiters I chose collide with the syntax. It's a pretty great mechanic. My script is here if you wanna check it out https://github.com/luizberti/templ https://github.com/luizberti/templ
angleofrepose 7y ago
Are there in-place macro tools that could be similar to this? Examples: - italics in a live markdown editor usually hides the asterisks after you've closed the pair, but can get them back if you backspace with your cursor at the 's'. - [[]] style links in similar live editors hide/show content as the syntax has been designed to do. These examples do it for me, the author, not just in publication. Other examples include CAD software array-style duplication as a single example. Draw one thing, give a command like "copy 10 times" and then move the piece and see 10 copies laid out. I can imagine using something like that `define` example in m4 and wanting the instruction to propagate throughout my text file, yet retain the command as a sort of undo/toggle option. Again, all for me the author as I think through my work in this editor, publication isn't really the point in these thoughts. The workflow for applying formulas to org mode tables is fantastic. I'm still an absolute novice, but I can keep an array of TBLFM formulas[1] below my table which I write and execute at will (C-c C-c or ,,) and can retain as a sort of history of the table state. There seem to be an endless variety of hooks I can tap into for these kinds of table editing sessions. I'm thinking about this in the context of the process of doing research and writing. When my thoughts are not concrete, I do not have a formal document I'm writing but I'm learning and thinking and connecting information. I am in the process of auditing a whole load of these tools and I want a tool that is the superset of all functionality I see, not sacrificing the awesome transclusion of TiddlyWiki or internal links in org mode for the drawing abilities of OneNote. I think what I want is an exceptionally powerful "viewport" for lack of a better term. I take great inspiration from ZUIs and wonder how to apply the fantastic text based tools I know to a semantically zooming canvas. I don't like directories, I do like juxtaposition of disparate thoughts and projects. Can I have an AutoCAD/OneNote/orgmode? I want one. m4 seems inspires me towards this goal, even though I realize that's not exactly it's intention. [1]: https://orgmode.org/manual/Durations-and-time-values.html#Durations-and-time-values https://orgmode.org/manual/Durations-and-time-values.html#Du...
- j88439h84 7y ago
  Tbh I cant figure out what you're asking for
  - 7y ago
Karrot_Kream 7y ago
I've become a huge fan of M4. At work, we needed to create a container that would create templated configuration files based on environment variables. We were considering using Mustache or other existing template systems, but most of them are fairly closely coupled to a programming language. We didn't want to tie container creation to a language runtime, and M4 seemed to be the most mature, well-documented (albeit crusty) language agnostic macro system. M4 was easy to use, and unless you do something crazy, fairly easy to read. I ended up moving most of the actual envar reading/processing logic to a single base m4 file, and so the actual templates would import all necessary variables from the base m4 file and largely keep the original structure. I wish there was a more "modern" (read: ergonomic) descendant of M4, but M4 is easy enough to understand, flexible, and extensively (if not well) documented enough that it's easy to use, stable, and fast enough for most of my use cases.
Brian_K_White 7y ago
Ah my lovingly hand-crafted fvwm config & menu files on that 386.
luizfelberti 7y ago
M4 is so impotent and crippling difficult to use, that it led me to create my own dumb templating engine[0] because there is no standard POSIX tool that will let you, simply and in the dumbest way possible, substitute strings within a file. If you try using awk or sed, you will need to figure out a magical way to escape whatever arbitrary value you are trying to replace, so you don't break sed's regex or awk's syntax. M4 was one of the last things I tried, and it was an utter disaster. If any of M4's magical characters collides with the syntax of the file you're trying to template: tough luck. And there is no way to escape your file for M4 either, it's syntax doesn't support that. I eventually discovered that Bash 4.4+ has some built-in functions for "dumb" string substitution[1], which is exactly what's missing from the default POSIX tools. Since then I wrote this little wrapper around that functionality, as a simple, single-file, dependency-free Bash script. If you find something like that useful, it's on the link below: [0] https://github.com/luizberti/templ https://github.com/luizberti/templ [1] https://www.tldp.org/LDP/abs/html/string-manipulation.html https://www.tldp.org/LDP/abs/html/string-manipulation.html
- diroussel 7y ago
  Would perl or ruby not suit your requirements?
  - luizfelberti 7y ago
    Ruby most definitely wouldn't, it doesn't come pre-installed on almost any AMI/server/Docker Image/etc. As for perl, it does come "with the system" more often than Ruby, but also not as much as bash. Even if it did, I reckon bash is something that nowadays people are more accustomed to than perl, and would consider it more of a no brainer.
- MarkSweep 7y ago
  There is likely already a simple templating program installed on your linux OS: envsubst. It is included as part of the gettext-base package on Ubuntu and installed by default I believe. https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html https://www.gnu.org/software/gettext/manual/html_node/envsub... The templating language looks similar to capability to that provided by templ.
0xff00ffee 7y ago
In this thread the top two comments couldn't be more opposite: PRO: "I've become a huge fan of M4 ... M4 was easy to use, and unless you do something crazy, fairly easy to read. " CON: "M4 is so impotent and crippling difficult to use, that it led me to create my own dumb templating engine[0]"
dang 7y ago
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=%22M4%22&sort=byDate&type=story&storyText=none https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... has many M4s but these two threads sound related: 2018: https://news.ycombinator.com/item?id=17781062 https://news.ycombinator.com/item?id=17781062 2015: https://news.ycombinator.com/item?id=10054484 https://news.ycombinator.com/item?id=10054484
jdblair 7y ago
Back in 2002 I wrote a documentation templating system in m4. It provided a lot of the same tags as markdown, and generated HTML. In retrospect I have no idea why I wrote it in m4 - I think it was a "because I can" kind of thing. I also had a system for ripping CDs written in GNU make, so let's say I had an odd taste in scripting tools.
orbifold 7y ago
I've seen M4 used to implement instruction decoding logic (for the POWER instruction set) in a design and to describe a bus interconnect in system verilog. Still not sure if it was a good or bad idea...
hzhou321 7y ago
Promoting MyDef: https://hz2.org/blog/mydef_general.html https://hz2.org/blog/mydef_general.html.
jeffrallen 7y ago
dnl This is a comment: M4 is insane.

92 points by luu 7y ago

The original report: https://wolfram.schneider.org/bsd/7thEdManVol2/m4/m4.pdf https://wolfram.schneider.org/bsd/7thEdManVol2/m4/m4.pdf

brodouevencode 7y ago

I'm curious if anyone is still using this tool today. Back when I used it (~8-ish years ago) I found it to be a godsend.

tannhaeuser 7y ago
It's used by autotools, and about 15 years ago I used it to generate BIND (dns server) configs, and sendmail.conf files, too. But I'd rather not use if I don't have to :) m4 is still POSIX, though, and maybe someone can elaborate on m4 and tell us about it's strength in times of JSON and (ya|to)ml configs, which I also don't like, perhaps because they're misnamed IMO as "markup languages".
- brodouevencode 7y ago
  This too is one of the ways I was using it back then. Working in a small CMS shop we were responsible for hosting Drupal sites with custom DNS, and all that. We used m4 to build out the BIND configs, Drupal config files, logging configs, and Apache configs.
- cryptonector 7y ago
  For JSON, see jq (https://stedolan.github.io/jq https://stedolan.github.io/jq).
enriquto 7y ago
I do, from time to time. It's nice to have a preprocessor for normal text files. Yet, most often I find it more practical to use sed or awk for my needs.
adrianN 7y ago
I use it as a simple static site generator and for generating C++. The thing I dislike most is that it uses ` and ' for quoting and there is no easy way of escaping quotes.
- kevin_thibedeau 7y ago
  You can change the quote chars. This is why autotools use [].
kevin_thibedeau 7y ago
I wrote a large package of macros for a niche assembler using M4. It has some sophisticated things like a code generator for non-infix expressions. M4 was a blessing and a curse. It is very powerful and saved me from implementing my own macro language but debugging is not for the faint of heart once you get into anything complicated. https://github.com/kevinpt/opbasm/blob/master/opbasm/picoblaze.m4 https://github.com/kevinpt/opbasm/blob/master/opbasm/picobla... Michael Breen's notes are indispensable documentation on how to make effective use of the language.
- throwaway_pdp09 7y ago
  I'm a fan of M4, and I don't know what these are doing but it surely looks too complex. You have defines within defines and OMG just an extract .. (just a snip follows) `define(`_'''$`'1```_LENGTH',strlenc(''$`'2``))'dnl `; "''$`'2``"' `ifdef(`PB3', ''$`'1```: calltable($4, $1, estr('''$`'2```))dnl return',dnl `table '''$`'1```#, [dec2pbhex(cstr('''$`'2```))] '''$`'1```: loadaddr($2, $3, _'''$`'1```_STR) jump __`'$5`'_handler _'''$`'1```_STR: load&return $1, '''$`'1```#')' )')' I have to ask if this is right, or whether a parser would have been a whole lot less work. Credit for your skillz, but that looks like so much pain.
  - kevin_thibedeau 7y ago
    These are macros that generate user configurable codegen macros. I couldn't hard code register assignments so everything dynamically generated is prepared with these monstrosities. They also can't have fixed names since some need to be invoked multiple times with different parameters. FWIW I didn't intend for things to go this far. I just started with some simple macros and it snowballed.
mauvehaus 7y ago
Some years ago at a former employer I used to for generating some Java 5 code. I needed (as I hazily recall) an interface of accessors, immutable implementation, and a factory for the same couple dozen members. It was a lot of boilerplate and the list was changing a fair amount over the course of development. It worked great, and I got it plugged into the ant build without too much difficulty. I'm told the person who took over that codebase was equal parts perplexed and horrified by it. They ripped out the m4 code generation and just put the generated files directly in the source tree. As the codebase had stabilized, the purpose that m4 served was basically gone. So, not still in use, but if I found myself in the same situation today, I'd explore other options and consider doing the same thing. Interface default methods might cover some of the need at this point, but it's so far back, I'm hazy on the details. The nice thing about doing this sort of code generation with m4 rather than the C preprocessor is that m4 preserves formatting (like line breaks) a lot better. Even if you're not using any of the other functionality of m4 (and there's a lot), that alone makes figuring out where things went wrong easier.
jiggerypokery 7y ago
I work for a telecom where we've used it to generate network configs for a long time. We just got rid of it because it's impossible to hire people who know it and nobody (new hire or existing people) want to learn it either
kd5bjo 7y ago
I dusted it off this semester to help with some complicated reports I needed to make. I needed to include lots of graphs from simulation runs with various parameters. I encoded the parameter values in the graphs’ filenames, and used m4 to write the corresponding gnuplot control file. After tying everything together with make, I could include plots for any parameter as I was writing the report and have the relevant data generated as necessary. All without re-running simulations that had already produced results.
spsrich2 7y ago
not today but I used m4 in the late 80s to generate appropriate makefiles from a single source to allow me to compile my programs on HP-ux, Apollo Aegis, SunOS, AIX, etc. Happy days.
vbezhenar 7y ago
Used it for some C code, because CPP was not able to accomplish what I wanted. m4 was more powerful and I was able to finish my task. Thankfully I never used it in context of sendmail or autotools, so no reason to hate it for me, I guess.

sigmonsays 7y ago

I always remember m4 encounters from autoconf struggles of building things from source a long time ago (slackware 6). Back when I never cared about m4, Today, I find it interesting to understand what problem its trying to solve. I dont feel like its practical anymore but I think it is important to not discard use of old tools and understand them.

jolmg 7y ago
https://en.wikipedia.org/wiki/M4_(computer_language)#History https://en.wikipedia.org/wiki/M4_(computer_language)#History I see it as a more powerful version of CPP. It might be somewhat based on it. CPP seems to have been developed in 1972-3[1], while M4 was developed in 1977. Today, I think we're using template systems like Mustache, Handlebars, ERB, Jinja, etc. where otherwise macro systems would be used. The difference between macro systems / preprocessors and templates is that the latter comes with syntax (like <%= %> for ERB) that always makes it clear what's handled by the template system. With M4 and CPP, it's implicit and instead you have syntax (`' or []) to exclude stuff from being handled by the macro system. I think that change is because, before, macro systems were meant to be used mainly with programming languages to avoid repetition. Then, programming languages became concise and more flexible, so they didn't need to be handled by another language like C has CPP. Our need changed to having to avoid repetition with data files like HTML. Perhaps having implicit expansion when used with an underlying programming language is not so much an issue, or at least it could be handled with a convention like how CPP variables and functions are normally capitalized. With data files, I suppose being able to differentiate between what's HTML and what's ERB was probably more important. [1] https://softwareengineering.stackexchange.com/questions/309389/what-is-the-origin-of-the-c-preprocessor https://softwareengineering.stackexchange.com/questions/3093...
- kd5bjo 7y ago
  The pattern I’ve fallen into with m4 is to have a preamble that sets up whatever definitions I’m going to need, redefine the quotes “backwards”, and then place a copy of the new close quote on its own line to start the main body of the file. It then acts very similarly to how you describe templating languages, where you mark the parts that should be parsed instead of the parts that shouldn’t.

otterley 7y ago

(2008)

seemslegit 7y ago

Deprecate it with fire.

teddyh 7y ago

M4 has some well-known uses like Sendmail and GNU Autoconf, but I don’t think this can be counted in M4’s favor: For operating in a standard Unix-like environment, there is really no good alternative to m4 for generic text macro expansion. You can try to force cpp – the C preprocessor – into this, but it is not suited to not being part of a C compiler pipeline. For better or for worse, m4 is it in the world of Unix and shell tools, even though its syntax is quite annoying to use. Of course, if you operate within a language ecosystem, you can probably find a nice templating system there. Python, for instance, has Jinja2. Or, if your needs are simple, you can use something more basic, like sed(1).

downerending 7y ago
I've looked at m4 before, and want to love it, but it just seems a bit too clumsy to use. And unfortunately, IIRC, unlike sed it's not ubiquitously available. Not really crazy about Jinja2 either, but at this point it seems like it has a lot more mindshare and maintenance.
- froh 7y ago
  M4 as part of posix should be as widely available as sed, no?
  - downerending 7y ago
    In principle, yes. In practice, it seems like it's often not installed on the random hosts I encounter. This isn't a fatal flaw, but it is a small advantage in favor of sed and awk and probably even perl, which are essentially ubiquitous these days.
PaulHoule 7y ago
Last time I used M4 it was for a multi language system where I needed to substitute in PHP, shell scripts, apache conf files, etc.
Blackthorn 7y ago
Mustache / Handlebars is pretty widely available throughout languages these days too and is pretty basic. Doesn't help you at the default posix level, but there are even command line versions IIRC

deleted 7y ago

[deleted]

scythe 7y ago

I wonder if the "quoting" problem could be fixed by making m4 recognize Lua-style longstrings, which never need escape characters. Some examples: http://lua-users.org/wiki/StringsTutorial http://lua-users.org/wiki/StringsTutorial I considered using longstrings to make a "better CSV" once but then I realized that quoting is only one of the many problems with CSV.

iamthad 7y ago
It's not clear to me what you mean by "longstrings" from that page. Are you referring to the double-bracket quoted style?
- scythe 7y ago
  Yes. A Lua-style longstring is delimited by [===[ and ]===] where the number of = signs can be arbitrary (including zero); the end must have the same number of = signs as the beginning. Any string can be encoded inside of a longstring without escaping, simply by using long enough delimiters. There are a number of advantages to this notation: it's more readable (usually), it generally results in shorter code files, and longquoted substrings can be directly addressed without translation (though there may be other reasons not to pass that pointer around).
luizfelberti 7y ago
I once developed a tiny single-file bash script to do dumb string substitution for arbitrary files and arbitrary strings, and I used this exact Lua mechanic to avoid escaping issues if the delimiters I chose collide with the syntax. It's a pretty great mechanic. My script is here if you wanna check it out https://github.com/luizberti/templ https://github.com/luizberti/templ

angleofrepose 7y ago

Are there in-place macro tools that could be similar to this? Examples: - italics in a live markdown editor usually hides the asterisks after you've closed the pair, but can get them back if you backspace with your cursor at the 's'. - [[]] style links in similar live editors hide/show content as the syntax has been designed to do. These examples do it for me, the author, not just in publication. Other examples include CAD software array-style duplication as a single example. Draw one thing, give a command like "copy 10 times" and then move the piece and see 10 copies laid out. I can imagine using something like that `define` example in m4 and wanting the instruction to propagate throughout my text file, yet retain the command as a sort of undo/toggle option. Again, all for me the author as I think through my work in this editor, publication isn't really the point in these thoughts. The workflow for applying formulas to org mode tables is fantastic. I'm still an absolute novice, but I can keep an array of TBLFM formulas[1] below my table which I write and execute at will (C-c C-c or ,,) and can retain as a sort of history of the table state. There seem to be an endless variety of hooks I can tap into for these kinds of table editing sessions. I'm thinking about this in the context of the process of doing research and writing. When my thoughts are not concrete, I do not have a formal document I'm writing but I'm learning and thinking and connecting information. I am in the process of auditing a whole load of these tools and I want a tool that is the superset of all functionality I see, not sacrificing the awesome transclusion of TiddlyWiki or internal links in org mode for the drawing abilities of OneNote. I think what I want is an exceptionally powerful "viewport" for lack of a better term. I take great inspiration from ZUIs and wonder how to apply the fantastic text based tools I know to a semantically zooming canvas. I don't like directories, I do like juxtaposition of disparate thoughts and projects. Can I have an AutoCAD/OneNote/orgmode? I want one. m4 seems inspires me towards this goal, even though I realize that's not exactly it's intention. [1]: https://orgmode.org/manual/Durations-and-time-values.html#Durations-and-time-values https://orgmode.org/manual/Durations-and-time-values.html#Du...

j88439h84 7y ago
Tbh I cant figure out what you're asking for
- 7y ago

Karrot_Kream 7y ago

I've become a huge fan of M4. At work, we needed to create a container that would create templated configuration files based on environment variables. We were considering using Mustache or other existing template systems, but most of them are fairly closely coupled to a programming language. We didn't want to tie container creation to a language runtime, and M4 seemed to be the most mature, well-documented (albeit crusty) language agnostic macro system. M4 was easy to use, and unless you do something crazy, fairly easy to read. I ended up moving most of the actual envar reading/processing logic to a single base m4 file, and so the actual templates would import all necessary variables from the base m4 file and largely keep the original structure. I wish there was a more "modern" (read: ergonomic) descendant of M4, but M4 is easy enough to understand, flexible, and extensively (if not well) documented enough that it's easy to use, stable, and fast enough for most of my use cases.

Brian_K_White 7y ago

Ah my lovingly hand-crafted fvwm config & menu files on that 386.

luizfelberti 7y ago

M4 is so impotent and crippling difficult to use, that it led me to create my own dumb templating engine[0] because there is no standard POSIX tool that will let you, simply and in the dumbest way possible, substitute strings within a file. If you try using awk or sed, you will need to figure out a magical way to escape whatever arbitrary value you are trying to replace, so you don't break sed's regex or awk's syntax. M4 was one of the last things I tried, and it was an utter disaster. If any of M4's magical characters collides with the syntax of the file you're trying to template: tough luck. And there is no way to escape your file for M4 either, it's syntax doesn't support that. I eventually discovered that Bash 4.4+ has some built-in functions for "dumb" string substitution[1], which is exactly what's missing from the default POSIX tools. Since then I wrote this little wrapper around that functionality, as a simple, single-file, dependency-free Bash script. If you find something like that useful, it's on the link below: [0] https://github.com/luizberti/templ https://github.com/luizberti/templ [1] https://www.tldp.org/LDP/abs/html/string-manipulation.html https://www.tldp.org/LDP/abs/html/string-manipulation.html

diroussel 7y ago
Would perl or ruby not suit your requirements?
- luizfelberti 7y ago
  Ruby most definitely wouldn't, it doesn't come pre-installed on almost any AMI/server/Docker Image/etc. As for perl, it does come "with the system" more often than Ruby, but also not as much as bash. Even if it did, I reckon bash is something that nowadays people are more accustomed to than perl, and would consider it more of a no brainer.
MarkSweep 7y ago
There is likely already a simple templating program installed on your linux OS: envsubst. It is included as part of the gettext-base package on Ubuntu and installed by default I believe. https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html https://www.gnu.org/software/gettext/manual/html_node/envsub... The templating language looks similar to capability to that provided by templ.

0xff00ffee 7y ago

In this thread the top two comments couldn't be more opposite: PRO: "I've become a huge fan of M4 ... M4 was easy to use, and unless you do something crazy, fairly easy to read. " CON: "M4 is so impotent and crippling difficult to use, that it led me to create my own dumb templating engine[0]"

dang 7y ago

https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=%22M4%22&sort=byDate&type=story&storyText=none https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... has many M4s but these two threads sound related: 2018: https://news.ycombinator.com/item?id=17781062 https://news.ycombinator.com/item?id=17781062 2015: https://news.ycombinator.com/item?id=10054484 https://news.ycombinator.com/item?id=10054484

jdblair 7y ago

Back in 2002 I wrote a documentation templating system in m4. It provided a lot of the same tags as markdown, and generated HTML. In retrospect I have no idea why I wrote it in m4 - I think it was a "because I can" kind of thing. I also had a system for ripping CDs written in GNU make, so let's say I had an odd taste in scripting tools.

orbifold 7y ago

I've seen M4 used to implement instruction decoding logic (for the POWER instruction set) in a design and to describe a bus interconnect in system verilog. Still not sure if it was a good or bad idea...

hzhou321 7y ago

Promoting MyDef: https://hz2.org/blog/mydef_general.html https://hz2.org/blog/mydef_general.html.

jeffrallen 7y ago

dnl This is a comment: M4 is insane.