4 ms·

Instead of interpreters, if one has less of a "must be a full featured prog.lang" mentality and a fast compiler like Go or Nim [1] (or is willing to wait, for s

• benhoyt 5y ago
  rp looks like a very interesting and clever approach. I'll have to take a closer look. Is https://github.com/c-blake/cligen/blob/master/examples/rp.nim#L23 https://github.com/c-blake/cligen/blob/master/examples/rp.ni... the extent of rp's documentation?
  • cb321 5y ago
    > the extent of rp's "documentation" Unfortunately, yes. My hope was that it would be compact/small enough to be "quasi-self-documenting" to the likes of "compiler writer types". Probably not to ordinary users. (The extra asterisks are to get nice ANSI SGR escape highlighting and/or rST markup out of the auto-documentation system.) {EDIT2: Also, I would be happy to add some rp.README if you want to contribute one. You seem a great explainer. } FWIW, I think of this kind of generation as part of the Go mentality more generally, but I am not a Go user/in that community. So, maybe that is speaking out of turn. I also have a C version of this that I call `crp` I could provide if anyone wants (and yes, short for "crap"). C is a kind of higher ceremony language for such things { EDIT1: but even more established than awk... :-) }
    • benhoyt 5y ago
      Code generation is used quite a bit in Go, though I haven't seen it used quite like this before. Now you've got me thinking what an rp-like tool would look like in Go. I think the error handling and statement vs expression distinction might get in the way a bit, but perhaps you could overcome that with helper functions.
      • cb321 5y ago
        I think the design is just about 100X easier for mere mortals { in bogus "effortmeters" ;-) }, but it is absolutely true that some backend languages are more suited than others. It is also true that having real optimizing compilers (in whatever lang) as backends will always result in faster running programs (if data is big enough that amortizing optimizer costs is worth it). {EDIT: i.e. "so, trade-offs, costs, YMMV, etc., etc. }
      • cb321 5y ago
        I'll go ahead and post crp.nim here. pjmlp rightly (and often here) observes that people become very trapped in their thinking about PLs. It perhaps helps to emphasize that the whole program generator need not be in the same PL as the generated program. But for the record, I do not think C is a compiled language very well suited to these one-liner tasks. There are probably better ones than Nim, too. My point is mostly about system design. One can probably say that the best backend is probably syntactically non-noisy and easy for whoever your users are. If you are your only user, go nuts some afternoon. :-) import std/[strutils,os,hashes,sets],cligen/[osUt,mslice] #% exec* mdOpen split from cligen/parseopt3 import optionNormalize proc toDef(fields, delim, genF: string): string = result.add "char const * const rpNmFields = \"" & fields & "\";\n" let sep = initSep(delim) let row = fields.toMSlice var s: seq[MSlice] var nms: HashSet[string] sep.split(row, s) # No maxSplit - define every field; Could infer it from the for j, f in s: #..highest referenced field with a `where` & `stmts` parse. let nm = optionNormalize(genF % [ $f ]) # Prevent duplicate def errors.. if nm notin nms: #..and warn users about collision. result.add "int const " & nm & " = " & $j & ";\n" nms.incl nm else: stderr.write "crp: WARNING: ", nm, " collides with earlier field\n" proc crp(prelude="", begin="", where="1", stmts:seq[string], epilog="", fields="", genF="$1", comp="", run=true, args="", outp="/tmp/crpXXX", input="/dev/stdin", delim=" \t", uncheck=false, maxSplit=0): int = ## Gen+Run *prelude*,*fields*,*begin*,*where*,*stmts*,*epilog* row processor ## against *input*. Defined within *where* & every *stmt* are: ## *s[idx]* & *row* => C strings, *i(idx)* => int64, *f(idx)* => double. ## *nf* & *nr* (like *AWK*); NOTE: *idx* is **0-origin**. ## A generated program is left at *outp*.c, easily copied for "utilitizing". ## If you know *AWK* & C, you can learn *crp* PRONTO. Examples (need data): ## **crp 'printf("%s %s\\n", s[1], s[0]);'** # Swap field order ## **crp -w'nr % 100==0' 'printf("%s\\n", row);'** # Prn each 100th row ## **crp -b'int t=0' t+=nf -e'printf("%d\\n", t)'** # Prn total field count ## **crp -b'int t=0' -w'i(0)>0' 't+=i(0)' -e'printf("%d\\n", t)'** # Total>0 ## **crp 'float x=f(0)' 'printf("%g\\n", (1+x)/x)'** # cache field 0 parse ## **crp -d, -fa,b,c 'printf("%s %g\\n",s[a],f(b)+i(c))'** # named fields ## Add niceties (eg. `#include "mystuff.h"`) to *prelude* in ~/.config/crp. let fields = if fields.len == 0: fields else: toDef(fields, delim, genF) let check = if fields.len == 0: " " elif not uncheck: """ if (nr == 0) { if (strcmp(row, rpNmFields) == 0) { nr++; continue; // {fields} {!uncheck} } else { exit(2); } while ((rpNmRead = getline(&row, &rpNmAlloc, rpNmFile)) != -1) { row[rpNmRead - 1] = '\0'; // chop newline ${6}s = rpNmSplit(s, &rpNmAlloc, row, "$3", $7, &nf); // {delim,maxSplit} if ($8) { // {where} auto ()s? """ % [prelude, fields, delim, indent(begin, 2), input, check, $maxSplit, where] for i, stmt in stmts: program.add " " & stmt & "; // {stmt" & $i & "}\n" if stmts.len == 0: program.add " /**/;\n" program.add " }\n nr++;\n }\n" program.add indent(epilog, 2) program.add "; // {epilogue}\n}\n" let mode = if run: "-run" else: "" let args = if args.len > 0: args else: "-I$HOME/s -O" let digs = count(outp, 'X') let hsh = toHex(program.hash and ((1 shl 16*digs) - 1), digs) let outp = if digs > 0: outp[0 ..< ^digs] & hsh else: outp let comp = if comp.len > 0: comp else: "tcc $1 $2 -o$3 $4" % [ mode, args, outp, outp & ".c"] let f = mkdirOpen(outp & ".c", fmWrite) f.write program f.close execShellCmd(comp & (if run: " < " & input else: "")) when isMainModule: import cligen; include cligen/mergeCfgEnv dispatch crp,help={"prelude" : "Nim code for prelude/imports section", "begin" : "Nim code for begin/pre-loop section", "where" : "Nim code for row inclusion", "stmts" : "Nim stmts to run under `where`", "epilog" : "Nim code for epilog/end loop section", "fields" : "`delim`-sep field names (match row0)", "genF" : "make field names from this fmt; eg c$1", "comp" : "\"\" => tcc {if run: \"-run\"} {args}", "run" : "Run at once using tcc -run .. < input", "args" : "\"\" => -I$HOME/s -O", "outp" : "output executable; .c NOT REMOVED", "input" : "path to read as input", "delim" : "inp delim chars for strtok", "uncheck" : "do not check&skip header row vs fields", "maxSplit": "max split; 0 => unbounded"}, cmdName="crp"
  • cb321 5y ago
    I should have said, those help strings at the very bottom of the file are also part of the "documentation" to the extent the variable names in the proc signature are not self explanatory.
• pjmlp 5y ago
  The problem with that approach is that it tends to create the false belief on developers inexperienced with compiler development that C is a must have for doing compilers. Then one needs to go around explaining, that no, some one programming language could have been chosen and it was just a matter of convinience why C was picked up.
  • cb321 5y ago
    This is fair, but argument in favor of the approach is that it is so simple it really could be done with any backend language {EDIT: by a 1st year CS student in an afternoon, um, maybe.. ;-)}..Go, Pascal, Ada, even slow running ones like Python/Perl/etc. or slow compiling ones like Rust. So, were the design to be more common people might not be so prone to make that assumption. OTOH, getting people to not make wrong assumptions is an eternal challenge. :-) My own view is that awk was done more or less for these one-liner/simple purposes but by people like Aho for whom full featured languages are barely more thinking than my 60 lines (in either doing the parser or using). :-)