9 ms·

Sed challenge: join cal -y months into a single column

110 points by jolmg 5y ago

1vuio0pswjnm7 5y ago
s/sed/GNU sed/ This will not work with BSD or Plan9 versions of sed They do not support using newline characters in a substitution command. Also, the other seds do not have the GNU "steps" extension, e.g., 2~3. The blog author's solution strategy is 100% reliant on this "steps" feature.
- jolmg 5y ago
  Indeed, there are improvements that could be made. I took the easy way not only by using GNU extensions, but also by calling sed multiple times instead of just once, and I even called awk at the end, though that was just because that bit didn't seem that significant. The spirit of the challenge wasn't exclusive to GNU, though. :) I was even hoping there'd be comments that show a better solution to learn from them, GNU or not. I got an APL one at least, so that was cool.
  - 1vuio0pswjnm7 5y ago
    Heres a solution that uses only sh and portable sed. No awk. No wc. This should work with GNU sed, BSD sed and Plan9 sed. #!/bin/sh col1(){ cal -y|sed -n "$1{s/$.\{23\}$$.*$/\1/p;}";} col2(){ cal -y|sed -n "$1{ s/$.\{23\}$$.\{20\}$$.*$/\2/p;}";} col3(){ cal -y|sed -n "$1{ s/.\{46\}//p;}";} cal_y(){ for x in 3,10 11,18 19,26 27,34;do col1 "$x";col2 "$x";col3 "$x";done;} test $# -eq 4||exec echo usage: $0 7 jul 20 aug; x=$(echo $2 $4|sed -n '/[A-Z]/p'); if test "$x";then echo use lowercase;else cal_y|sed '/^Su/d;/^ *$/d;/^ *[A-S]/s/^/@/ /Jan/,/Feb/s/$/ jan /; /Feb/,/Mar/s/$/ feb /; /Mar/,/Apr/s/$/ mar /; /Apr/,/May/s/$/ apr /; /May/,/Jun/s/$/ may /; /Jun/,/Jul/s/$/ jun /; /Jul/,/Aug/s/$/ jul /; /Aug/,/Sep/s/$/ aug /; /Sep/,/Oct/s/$/ sep /; /Oct/,/Nov/s/$/ oct /; /Nov/,/Dec/s/$/ nov /; /Dec/,$s/$/ dec /;/@/d' \ |sed -n "/ $1 .*$2/,/ $3 .*$4/{ / 1 /d;p;}"|sed -n '$=';fi
    - jolmg 5y ago
      > Heres a solution that uses only sh and portable sed. No awk. No wc. No, it isn't! :) The challenge was specifically about joining the months of `cal -y` into a single column with sed. It's about the challenge of working with a grid of grids, with the restrictive nature of sed, having only one hold-space to hold 2 months separately. You totally missed doing that with your solution. Of course, it's loads easier to just skip the challenge altogether! Maybe the point of the sed challenge wasn't clear. It's about stretching the abilities of sed to its limits. Of being able to use all its features to its fullest. I think this is the first time that I had a real use for m~n, and I still haven't found a good use for `D` or `P`. EDIT: You're using `cal -y` now, but you've still gone outside the challenge space: > what kind of sed script I’d need in the middle of a `cal -y | ... | wc -l` pipeline to get the same answer Meaning the working space is just substituting the ellipses for `sed` commands. It's a single `cal -y` call, a single `wc -l` call, and no other shell code except pipes. I did an awk command in the last bit, but here is the sed equivalent: sed -E ' /[a-z]$/{h;s/.* //;x} /[a-z]$/!G /\b7\b.*July/,/20.*August/!d s/\n.*// ' Ideally, it was supposed to be just a single sed command, but I wanted to try out the org-mode blocks for showing the process in multiple stages and also use the m~n addresses, which it seemed like I wouldn't be able to if I did the single call.
      - 1vuio0pswjnm7 5y ago
        This solution uses temp files: 111, and 111-a through 111-l. 111 is the full calendar, the others are individual months. No dependence on any GNU sed features. Will work with GNU sed, BSD sed and Plan9 sed. Uses only sed commands and pipes, single cal -y, sed instead of wc. test $# -eq 4||exec echo usage: $0 7 jul 20 aug; x=$(echo $2 $4|sed -n '/[A-Z]/p'); if test "$x";then echo use lowercase;exit;fi cal -y \ |sed -n -e "w 111" \ -e "3,10{s/$.\{23\}$$.*$/\1/w 111-a }" -e "r 111"|sed -n \ -e "3,10{s/$.\{23\}$$.\{20\}$$.*$/\2/w 111-b }" -e "3,10r 111"|sed -n \ -e "3,10{s/.\{46\}//w 111-c }" -e "3,10r 111"|sed -n \ -e "11,18{s/$.\{23\}$$.*$/\1/w 111-d }" -e "11,18r 111"|sed -n \ -e "11,18{s/$.\{23\}$$.\{20\}$$.*$/\2/w 111-e }" -e "11,18r 111"|sed -n \ -e "11,18{s/.\{46\}//w 111-f }" -e "11,18r 111"|sed -n \ -e "19,26{s/$.\{23\}$$.*$/\1/w 111-g }" -e "19,26r 111"|sed -n \ -e "19,26{s/$.\{23\}$$.\{20\}$$.*$/\2/w 111-h }" -e "19,26r 111"|sed -n \ -e "19,26{s/.\{46\}//w 111-i }" -e "19,26r 111"|sed -n \ -e "27,34{s/$.\{23\}$$.*$/\1/w 111-j }" -e "27,34r 111"|sed -n \ -e "27,34{s/$.\{23\}$$.\{20\}$$.*$/\2/w 111-k }" -e "27,34r 111"|sed -n \ -e "27,34{s/.\{46\}//w 111-l }" -e "27,34r 111"|sed '2,$d' \ |sed -e "r 111-a" -e "r 111-b" -e "r 111-c" \ -e "r 111-d" -e "r 111-e" -e "r 111-f" \ -e "r 111-g" -e "r 111-h" -e "r 111-i" \ -e "r 111-j" -e "r 111-k" -e "r 111-l" \ |sed '/^Su/d;/^ *$/d;/^ *[A-S]/s/^/@/ /Jan/,/Feb/s/$/ jan /; /Feb/,/Mar/s/$/ feb /; /Mar/,/Apr/s/$/ mar /; /Apr/,/May/s/$/ apr /; /May/,/Jun/s/$/ may /; /Jun/,/Jul/s/$/ jun /; /Jul/,/Aug/s/$/ jul /; /Aug/,/Sep/s/$/ aug /; /Sep/,/Oct/s/$/ sep /; /Oct/,/Nov/s/$/ oct /; /Nov/,/Dec/s/$/ nov /; /Dec/,$s/$/ dec /;/@/d' \ |sed -n "/ $1 .*$2/,/ $3 .*$4/{ / 1 /d;p;}"|sed -n '$='; rm 111 111-[a-l]
  - saltcured 5y ago
    Here's the opposite of your approach, using only basic POSIX shell techniques... #!/bin/sh # trim left space triple ' '* lt() { s="$1"; while [ ! "${s## }" = "$s" ]; do s="${s## }"; done; printf "%s" "$s"; } # trim right space ' '* rt() { s="$1"; while [ ! "${s%% }" = "$s" ]; do s="${s%% }"; done; printf "%s" "$s"; } # count months for first/last special treatment n=1 # usage: reform pos line1 line2 ... line7 reform() { o=$1; m="$(printf "%s" $(expr substr "$2" $o 20))"; l1="$(expr substr "$3" $o 20)" [ $n -ne 1 ] && l1="$(lt "$l1")" # suffix of spliced week shift 3 printf "%s %s\n" "$l1" "$m" while [ "$#" -gt 0 ]; do l="$(expr substr "$1" $o 20 )"; lx="$(rt "$l")"; shift if [ -z "$lx" ]; then : elif [ "$l" = "$lx" -o $n -eq 12 ]; then printf "%s\n" "$lx" # whole week elif [ -n "$lx" ]; then printf "%s " "$lx" # prefix of spliced week fi done n=$(( $n + 1 )) } # buffer 8 lines into arg vector via recursion hack # then replace lines 3x to project out 3 months in this row buffer() { if [ "$#" -eq 7 ]; then reform 1 "$@"; reform 24 "$@"; reform 47 "$@" else IFS= read line || return 1 [ "$#" -eq 0 ] && read discard buffer "$@" "$line" fi } # strip preamble and process 4 rows of months process() { read discard; read discard; for i in 1 2 3 4; do buffer; done; } cal -y ${1:-2021} | process
    - 1vuio0pswjnm7 5y ago
      expr is not part of the shell, it is a separate program
raspyberr 5y ago
I really like the start of your blog. Please do consider adding RSS/Atom support. I'll add you to my feed if you do.
- jolmg 5y ago
  Thanks! Alright. I haven't looked much into RSS/Atom yet, but I'll try to add it by next week. That should ease my feeling for a need of a schedule. I was just wondering if there were RSS/Atom users that would use it.
  - jen729w 5y ago
    Hell yeah, still plenty of us who RSS.
glandium 5y ago
You still reached for awk ;)
antholeole 5y ago
Is there any sources for practice problems like this with sed and other Unix utilities? Like a leetcode, but for solving problems in a Unix environment. I really enjoy these kinds of issues and see some wizards (like this post!) be able to do some really cool stuff, and I’d like to have it in my tool belt, but I’m not sure where I can get practice problems. Thanks!
- azeirah 5y ago
  Not entirely sure if you'll find many sed posts there, but answering open/new stack overflow and related stack exchange questions is a great way to find some real-world practice and help others out in the meantime.
otherflavors 5y ago
I'd have suggested to make this more challenging and only use POSIX sed, but -y is not part of POSIX cal anyway
- jolmg 5y ago
  Another way to make it more challenging would be to do everything in a single `sed` call. Maybe it's for that kind of script that features like `D`'s and `P`'s "up to the first newline" detail become useful.
clon 5y ago
At what point should you give up writing a sed "program" and just write a few lines of Python to arrive at an answer? Seems he went way past that point.
- jolmg 5y ago
  It was even easier to just count with my finger. :) The point was a sed challenge. Challenges are fun sometimes. It was also a good exercise for doing things with sed. There are times when sed is the most terse way to shape text to how you want it. It's also very portable.
- fragmede 5y ago
  > Of course, I just looked at the calendar and counted
- azalemeth 5y ago
  Is answering the challenge he answered with python actually easier? I know counting the weeks is easier, but is parsing text like a ninja far less verbose in posix-like tools? Learning awk made me much more productive, at the very least...
  - helsinki 5y ago
    cal -y | sed -E '1,2d;s/(.{20}) /\1\n/g;' | python3 -c “import sys, calendar;[print(ln.strip()) for ln in sys.stdin.readlines() if ln.strip() in calendar.month_name]” easier to do: python3 -c “import calendar;[print(m) for m in calendar.month_name]”
    - t0mas88 5y ago
      I think you've only read the title? :-) The article is about a script that prints the full calendar format, with all the days/dates in between to be able to count some weeks. Not just a list of month names.
  - pcwalton 5y ago
    Ruby and Perl have awk modes, activated with command-line switches, that are strictly more powerful than awk. I tried using awk recently for a text-processing task and it was a fun learning experience, but I ended up rewriting in Ruby. There's basically no advantage that awk has over Ruby or Perl, aside from portability to embedded systems, and plenty of disadvantages.
jez 5y ago
This is super cool--I've wanted something notebook-like for blog posts in the past, and I had no clue it was possible in org-mode! I can also empathize with the desire for a single-column calendar output, but my first instinct wasn't to implement it with sed: https://github.com/jez/calz https://github.com/jez/calz Substantially more lines than the sed implementation, but also more feature rich, because I can do things like calz from last month to next month calz from now to feb etc. Also: pretty colors.
- jolmg 5y ago
  > This is super cool--I've wanted something notebook-like for blog posts in the past, and I had no clue it was possible in org-mode! Thanks! You can access the org file by changing the URL extension from .html to .org. Maybe I should make that a link.
raziel2p 5y ago
Wow. I've only ever used sed for simple regex operations, reading this has opened up a whole new world for me. ... I will force myself not to make use of it because you're a bad influence and I already have to stop myself from using too much grep/sed/awk magic, but this is seriously impressive, and an entertaining read. Well done!
- jolmg 5y ago
  Thank you for the kind words! :)
- mananaysiempre 5y ago
  I can’t bring myself to write blog posts, but here’s a single-precedence-level infix-to-postfix notation transformer in sed written as part of a solution for last year’s Advent of Code 18a (https://adventofcode.com/2020/day/18 https://adventofcode.com/2020/day/18): #!/bin/sh -eu set -o pipefail sed -E \ -e 's/([+*])\s*([0-9]+)/ \2 \1 /g' \ -e ':parens' \ -e 's/([+*])\s*($[^()]+$)/ \2 \1 /g' \ -e 's/$([^()]+)$/ \1 /g' \ -e '/\(/bparens' \ -e 's/\s+/ /g' -e 's/^ //' -e 's/ $//' \ -e '$a\' -e '[+ z1<r]sr z1<r p' | \ dc (I’m very, very sorry for fuelling your addiction.) UPD: The original version claimed this was a precedence parser. It isn’t, it just does parentheses :(
  - jolmg 5y ago
    Maybe I'm doing something wrong, but passing it "3 * 4 + 5 * 6" gives me 102 instead of 42.
    - mananaysiempre 5y ago
      You are correct; I seemed to remember there was a catch but was too lazy to check, sorry. This isn’t a precedence parser, that was 18b, while this one solves 18a, which was just to resolve parentheses correctly and disregard precedence. (Original comment fixed.)
- nvartolomei 5y ago
  How about a chess engine written in sed? https://github.com/moldabekov/chess-sed/blob/master/chess.sed https://github.com/moldabekov/chess-sed/blob/master/chess.se.... And accompanying blog post in Russian https://habr.com/ru/post/191006/ https://habr.com/ru/post/191006/.
tlackemann 5y ago
org-mode blog! How do you like it? Been considering this for myself recently but haven't looked into RSS much. I like the theme you built.
- jolmg 5y ago
  > org-mode blog! How do you like it? I've just gotten started! I think it's comfortable in the sense that I won't need to copy and paste stuff to and from the terminal and can just code from the same file and see how the result changes. That's really neat! I imagine I'll be feeling restricted at some point, since I'm not doing custom HTML for each post, and I want to keep the org file readable on its own, so no excessive markup. > I like the theme you built. Thanks, though it's really almost no styling. The syntax highlighting is done by the ruby library `rouge`. The colors were just black on white, but the Dark Reader Firefox extension I have installed would change that to these off-black and off-white colors, and I liked it, so I set them in the CSS.
rak1507 5y ago
Just for fun, I tried it in APL. cal ← ⎕SH 'cal -y' ⍝ get output of cal command bymonth ← {⍵⊂⍨0=23|⍳≢⍵}⍤1↑2↓cal ⍝ partition into month groups lines ← {⍵⊂⍨0=8|⍳≢⍵}⍤1⍉bymonth ⍝ partition into chunks in each line months ← ~∘' '¨,⍉⊃¨lines ⍝ extract month names and remove spaces days ← (23⍴1 0 0)⊂⍤1↑⊃,/,⌿2↓¨lines ⍝ partition into days trimmed ← ∊⍤1⍉{⍵/⍨(' '∨.≠¨⍵)∨∧\' '∧.=¨⍵}⍤1⍉¯1↓days ⍝ remove excess spaces ⎕←53↑(¯2↓⍤1⊢trimmed) , ↑months\⍨∨/' 1 '⍷trimmed ⍝ append months to calendar where the 1s are, and print Surprisingly tricky (but still easier than sed IMO :P)
- eloisius 5y ago
  And people say Ruby is beautiful
- deleted 5y ago
  [deleted]
- wahern 5y ago
  It's too bad APL never enjoyed ubiquity on Unix-like platforms. The world would have been a better place. Maybe if Stallman or some early GNU participant had championed APL we'd all be either using APL or constantly excusing ourselves. Like Emacs did for Lisp. I'm decent with sed. I wouldn't call myself an expert, but know how to employ addressing and hold spaces, which are sadly aracana to most people who employ sed. I ended up learning sed because it was there, proved useful, and it fit into the command-line toolkit really well. (In order of importance.) APL would never slot into that toolkit as well as other tools, but it's value-add would more than compensate. Unfortunately, the value-add isn't enough to compensate for the fact that it's not there and in your face every time you're exploring a problem and its solution space.
  - jolmg 5y ago
    > Maybe if Stallman or some early GNU participant had championed APL we'd all be either using APL or constantly excusing ourselves. I think the major issue is that APL seems to require more than ASCII to work it.
helsinki 5y ago
cal -y | sed -E '1,2d;s/(.{20}) /\1\n/g;' | egrep “^. .* $“ or cal -y | sed -E '1,2d;s/(.{20}) /\1\n/g;' | python3 -c “from calendar import month_name;import sys;[print(ln.strip()) for ln in sys.stdin.readlines() if ln.strip() in month_name]”
- rak1507 5y ago
  Neither of them seem to work for me - what am I missing? https://tio.run/##lcxBDoIwFIThvaeYNCYUI0VwydpTiJpKCzSB16bthqhnR0hMWLv98808ZejnuZEDsglvBK2QXZAUx1JVIefiVZ4@KfK6qCnvqmQhuvPagd0FxAF7tvtn66bYWzoja8Bab0csY01KepjRWR8xWor9g@Soq18JU6iuzhuKfCARojeOpyla6zEQDK1gycqQ8FqqwZAOPIVpsfGVbc83Ns9f https://tio.run/##lcxBDoIwFIThvaeYNCYUI0VwydpTiJpKCzSB16bthq...
  - helsinki 5y ago
    Sorry about that. I was typing from my phone, blindly. This is a cool website, thanks. Here is the correct solution: cal -y | sed -E '1,2d;s/(.{20}) /\1\n/g;' | python3 -c "from calendar import month_name;import sys;[print(ln.strip()) for ln in sys.stdin.readlines() if ln.strip() in month_name and len(ln.strip())]" https://tio.run/##TY1BDsIgFAWv8tJNIbGtrUvWnsIag4VaEvohwKZRz440MdHtZN68u4xLzpO0aDa8ELVCc0bdHwYlYsfa53B8c3RjP1L3EHVR/JYWRyc0E6o5uBVlrEnJALN6FxJWR2m5kVy1@JK4RXHxwVBiltqYgvGMc8wuwBIM7ULBylAbtFTWkI6Mw8z46bv2K0OSQrn9712rnD8 https://tio.run/##TY1BDsIgFAWv8tJNIbGtrUvWnsIag4VaEvohwKZRz4...
- t0mas88 5y ago
  That's only a list of names of months, the article creates a single column view with all the dates be between to then count the number of weeks between two dates.
saltcured 5y ago
For those who want to never learn sed addressing modes, here's a slight cheat using only simpler shell logic. The cheat is to realize you can concatenate multiple calls to `cal -1` instead of reshaping the 3x4 grid into 12x1, so just focus on the per-month splicing task... #!/bin/sh g() { read m; read n [ $1 -eq 1 ] \ && printf '%20.20s %s\n' "$n" "$m" \ || printf ' %s %s\n' "$n" "$m" sed -e 's/ *$/ /' } f() { cal -1 $@ | grep '[0-9]' | g $@ | head -c -1 } y=${1:-`date +%Y`} for m in {1..12}; do f $m $y; done printf "\n"
- deleted 5y ago
  [deleted]
- Tepix 5y ago
  On macOS you have to change it slightly to use the "-m" parameter to cal because it doesn't have the "-1" option.
deleted 5y ago
[deleted]
pixelbeat__ 5y ago
I know this is not in the spirit of the article, but getting the number of weeks is quite useful generally. One can do this with the standard date util like: $ echo $(((($(date +%s -d 'Aug 31') - $(date +%s -d 'Jul 1'))/(60*60*24*7)))) 8 # 8.7 weeks truncated $ echo $(((($(date +%W -d 'Aug 31') - $(date +%W -d 'Jul 1'))))) 9 It's useful to get the number of weeks from git logs for release notes etc., which can be done like: oldvs='v1.0' oldts=$(git log -n1 --format="%at" "$oldvs") weeks=$((($(date +%s) - $oldts)/(60*60*24*7)))
heresie-dabord 5y ago
The APL code (posted by rak1507) wins my admiration too. Now, given that this confounded problem has taken too much of my time @^_^@ I'll just say that the simplest option seems to be ncal -w -3 -y -b -h because it already computes the week numbers. Fiddle away with sed afterward. By the way, the "fold" utility [1] can help here too: ncal -w -3 -y -b -h | sed 's/ /*/g' | fold -w 27 Cheers! [1] https://en.wikipedia.org/wiki/Fold_(Unix) https://en.wikipedia.org/wiki/Fold_(Unix)
lsferreira42 5y ago
Using some shell utilities turns out to be very easy: LANG=c cal -y|grep '[a-z]'|egrep -v 'Su ' | xargs
- deleted 5y ago
  [deleted]
elcaro 5y ago
Just for fun, I tried it in Raku. say qx< cal -hy > # Command output as string .lines.skip.join("\n").split(/\n\n+/) # Skip year & split groups .map({ |([Z] .lines.join.comb(22).batch(3)) }) # Single column structure .map({ |(.[2] ~ .[0].trim, |.[3 .. *]) }) # Month name on first row .grep(*.trim.chars).join("\n") # Join non-blank lines .subst(/\s+ ^^ \s\s+ 1»/, ' 1', :g) # Collapse weeks together I was never going to beat APL... Though I imagine some clever Rakoons could probably find a few places where this can be golfed down.
- elcaro 5y ago
  The first operation where I split to lines, then join, then split again... was annoying me. So now I've got this say qx< cal -hy >.lines.skip.batch(9) .map({ |([Z] .join.comb(22).batch(3)) }) .map({ |(.[2] ~ .[0].trim, |.[3 .. *]) }) .grep(*.trim.chars).join("\n") .subst(/\s+ ^^ \s\s+ 1»/, ' 1', :g)
deleted 5y ago
[deleted]

110 points by jolmg 5y ago

s/sed/GNU sed/ This will not work with BSD or Plan9 versions of sed They do not support using newline characters in a substitution command. Also, the other seds do not have the GNU "steps" extension, e.g., 2~3. The blog author's solution strategy is 100% reliant on this "steps" feature.

jolmg 5y ago
Indeed, there are improvements that could be made. I took the easy way not only by using GNU extensions, but also by calling sed multiple times instead of just once, and I even called awk at the end, though that was just because that bit didn't seem that significant. The spirit of the challenge wasn't exclusive to GNU, though. :) I was even hoping there'd be comments that show a better solution to learn from them, GNU or not. I got an APL one at least, so that was cool.
- 1vuio0pswjnm7 5y ago
  Heres a solution that uses only sh and portable sed. No awk. No wc. This should work with GNU sed, BSD sed and Plan9 sed. #!/bin/sh col1(){ cal -y|sed -n "$1{s/$.\{23\}$$.*$/\1/p;}";} col2(){ cal -y|sed -n "$1{ s/$.\{23\}$$.\{20\}$$.*$/\2/p;}";} col3(){ cal -y|sed -n "$1{ s/.\{46\}//p;}";} cal_y(){ for x in 3,10 11,18 19,26 27,34;do col1 "$x";col2 "$x";col3 "$x";done;} test $# -eq 4||exec echo usage: $0 7 jul 20 aug; x=$(echo $2 $4|sed -n '/[A-Z]/p'); if test "$x";then echo use lowercase;else cal_y|sed '/^Su/d;/^ *$/d;/^ *[A-S]/s/^/@/ /Jan/,/Feb/s/$/ jan /; /Feb/,/Mar/s/$/ feb /; /Mar/,/Apr/s/$/ mar /; /Apr/,/May/s/$/ apr /; /May/,/Jun/s/$/ may /; /Jun/,/Jul/s/$/ jun /; /Jul/,/Aug/s/$/ jul /; /Aug/,/Sep/s/$/ aug /; /Sep/,/Oct/s/$/ sep /; /Oct/,/Nov/s/$/ oct /; /Nov/,/Dec/s/$/ nov /; /Dec/,$s/$/ dec /;/@/d' \ |sed -n "/ $1 .*$2/,/ $3 .*$4/{ / 1 /d;p;}"|sed -n '$=';fi
  - jolmg 5y ago
    > Heres a solution that uses only sh and portable sed. No awk. No wc. No, it isn't! :) The challenge was specifically about joining the months of `cal -y` into a single column with sed. It's about the challenge of working with a grid of grids, with the restrictive nature of sed, having only one hold-space to hold 2 months separately. You totally missed doing that with your solution. Of course, it's loads easier to just skip the challenge altogether! Maybe the point of the sed challenge wasn't clear. It's about stretching the abilities of sed to its limits. Of being able to use all its features to its fullest. I think this is the first time that I had a real use for m~n, and I still haven't found a good use for `D` or `P`. EDIT: You're using `cal -y` now, but you've still gone outside the challenge space: > what kind of sed script I’d need in the middle of a `cal -y | ... | wc -l` pipeline to get the same answer Meaning the working space is just substituting the ellipses for `sed` commands. It's a single `cal -y` call, a single `wc -l` call, and no other shell code except pipes. I did an awk command in the last bit, but here is the sed equivalent: sed -E ' /[a-z]$/{h;s/.* //;x} /[a-z]$/!G /\b7\b.*July/,/20.*August/!d s/\n.*// ' Ideally, it was supposed to be just a single sed command, but I wanted to try out the org-mode blocks for showing the process in multiple stages and also use the m~n addresses, which it seemed like I wouldn't be able to if I did the single call.
    - 1vuio0pswjnm7 5y ago
      This solution uses temp files: 111, and 111-a through 111-l. 111 is the full calendar, the others are individual months. No dependence on any GNU sed features. Will work with GNU sed, BSD sed and Plan9 sed. Uses only sed commands and pipes, single cal -y, sed instead of wc. test $# -eq 4||exec echo usage: $0 7 jul 20 aug; x=$(echo $2 $4|sed -n '/[A-Z]/p'); if test "$x";then echo use lowercase;exit;fi cal -y \ |sed -n -e "w 111" \ -e "3,10{s/$.\{23\}$$.*$/\1/w 111-a }" -e "r 111"|sed -n \ -e "3,10{s/$.\{23\}$$.\{20\}$$.*$/\2/w 111-b }" -e "3,10r 111"|sed -n \ -e "3,10{s/.\{46\}//w 111-c }" -e "3,10r 111"|sed -n \ -e "11,18{s/$.\{23\}$$.*$/\1/w 111-d }" -e "11,18r 111"|sed -n \ -e "11,18{s/$.\{23\}$$.\{20\}$$.*$/\2/w 111-e }" -e "11,18r 111"|sed -n \ -e "11,18{s/.\{46\}//w 111-f }" -e "11,18r 111"|sed -n \ -e "19,26{s/$.\{23\}$$.*$/\1/w 111-g }" -e "19,26r 111"|sed -n \ -e "19,26{s/$.\{23\}$$.\{20\}$$.*$/\2/w 111-h }" -e "19,26r 111"|sed -n \ -e "19,26{s/.\{46\}//w 111-i }" -e "19,26r 111"|sed -n \ -e "27,34{s/$.\{23\}$$.*$/\1/w 111-j }" -e "27,34r 111"|sed -n \ -e "27,34{s/$.\{23\}$$.\{20\}$$.*$/\2/w 111-k }" -e "27,34r 111"|sed -n \ -e "27,34{s/.\{46\}//w 111-l }" -e "27,34r 111"|sed '2,$d' \ |sed -e "r 111-a" -e "r 111-b" -e "r 111-c" \ -e "r 111-d" -e "r 111-e" -e "r 111-f" \ -e "r 111-g" -e "r 111-h" -e "r 111-i" \ -e "r 111-j" -e "r 111-k" -e "r 111-l" \ |sed '/^Su/d;/^ *$/d;/^ *[A-S]/s/^/@/ /Jan/,/Feb/s/$/ jan /; /Feb/,/Mar/s/$/ feb /; /Mar/,/Apr/s/$/ mar /; /Apr/,/May/s/$/ apr /; /May/,/Jun/s/$/ may /; /Jun/,/Jul/s/$/ jun /; /Jul/,/Aug/s/$/ jul /; /Aug/,/Sep/s/$/ aug /; /Sep/,/Oct/s/$/ sep /; /Oct/,/Nov/s/$/ oct /; /Nov/,/Dec/s/$/ nov /; /Dec/,$s/$/ dec /;/@/d' \ |sed -n "/ $1 .*$2/,/ $3 .*$4/{ / 1 /d;p;}"|sed -n '$='; rm 111 111-[a-l]
- saltcured 5y ago
  Here's the opposite of your approach, using only basic POSIX shell techniques... #!/bin/sh # trim left space triple ' '* lt() { s="$1"; while [ ! "${s## }" = "$s" ]; do s="${s## }"; done; printf "%s" "$s"; } # trim right space ' '* rt() { s="$1"; while [ ! "${s%% }" = "$s" ]; do s="${s%% }"; done; printf "%s" "$s"; } # count months for first/last special treatment n=1 # usage: reform pos line1 line2 ... line7 reform() { o=$1; m="$(printf "%s" $(expr substr "$2" $o 20))"; l1="$(expr substr "$3" $o 20)" [ $n -ne 1 ] && l1="$(lt "$l1")" # suffix of spliced week shift 3 printf "%s %s\n" "$l1" "$m" while [ "$#" -gt 0 ]; do l="$(expr substr "$1" $o 20 )"; lx="$(rt "$l")"; shift if [ -z "$lx" ]; then : elif [ "$l" = "$lx" -o $n -eq 12 ]; then printf "%s\n" "$lx" # whole week elif [ -n "$lx" ]; then printf "%s " "$lx" # prefix of spliced week fi done n=$(( $n + 1 )) } # buffer 8 lines into arg vector via recursion hack # then replace lines 3x to project out 3 months in this row buffer() { if [ "$#" -eq 7 ]; then reform 1 "$@"; reform 24 "$@"; reform 47 "$@" else IFS= read line || return 1 [ "$#" -eq 0 ] && read discard buffer "$@" "$line" fi } # strip preamble and process 4 rows of months process() { read discard; read discard; for i in 1 2 3 4; do buffer; done; } cal -y ${1:-2021} | process
  - 1vuio0pswjnm7 5y ago
    expr is not part of the shell, it is a separate program

raspyberr 5y ago

I really like the start of your blog. Please do consider adding RSS/Atom support. I'll add you to my feed if you do.

jolmg 5y ago
Thanks! Alright. I haven't looked much into RSS/Atom yet, but I'll try to add it by next week. That should ease my feeling for a need of a schedule. I was just wondering if there were RSS/Atom users that would use it.
- jen729w 5y ago
  Hell yeah, still plenty of us who RSS.

glandium 5y ago

You still reached for awk ;)

antholeole 5y ago

Is there any sources for practice problems like this with sed and other Unix utilities? Like a leetcode, but for solving problems in a Unix environment. I really enjoy these kinds of issues and see some wizards (like this post!) be able to do some really cool stuff, and I’d like to have it in my tool belt, but I’m not sure where I can get practice problems. Thanks!

azeirah 5y ago
Not entirely sure if you'll find many sed posts there, but answering open/new stack overflow and related stack exchange questions is a great way to find some real-world practice and help others out in the meantime.

otherflavors 5y ago

I'd have suggested to make this more challenging and only use POSIX sed, but -y is not part of POSIX cal anyway

jolmg 5y ago
Another way to make it more challenging would be to do everything in a single `sed` call. Maybe it's for that kind of script that features like `D`'s and `P`'s "up to the first newline" detail become useful.

clon 5y ago

At what point should you give up writing a sed "program" and just write a few lines of Python to arrive at an answer? Seems he went way past that point.

jolmg 5y ago
It was even easier to just count with my finger. :) The point was a sed challenge. Challenges are fun sometimes. It was also a good exercise for doing things with sed. There are times when sed is the most terse way to shape text to how you want it. It's also very portable.
fragmede 5y ago
> Of course, I just looked at the calendar and counted
azalemeth 5y ago
Is answering the challenge he answered with python actually easier? I know counting the weeks is easier, but is parsing text like a ninja far less verbose in posix-like tools? Learning awk made me much more productive, at the very least...
- helsinki 5y ago
  cal -y | sed -E '1,2d;s/(.{20}) /\1\n/g;' | python3 -c “import sys, calendar;[print(ln.strip()) for ln in sys.stdin.readlines() if ln.strip() in calendar.month_name]” easier to do: python3 -c “import calendar;[print(m) for m in calendar.month_name]”
  - t0mas88 5y ago
    I think you've only read the title? :-) The article is about a script that prints the full calendar format, with all the days/dates in between to be able to count some weeks. Not just a list of month names.
- pcwalton 5y ago
  Ruby and Perl have awk modes, activated with command-line switches, that are strictly more powerful than awk. I tried using awk recently for a text-processing task and it was a fun learning experience, but I ended up rewriting in Ruby. There's basically no advantage that awk has over Ruby or Perl, aside from portability to embedded systems, and plenty of disadvantages.

jez 5y ago

This is super cool--I've wanted something notebook-like for blog posts in the past, and I had no clue it was possible in org-mode! I can also empathize with the desire for a single-column calendar output, but my first instinct wasn't to implement it with sed: https://github.com/jez/calz https://github.com/jez/calz Substantially more lines than the sed implementation, but also more feature rich, because I can do things like calz from last month to next month calz from now to feb etc. Also: pretty colors.

jolmg 5y ago
> This is super cool--I've wanted something notebook-like for blog posts in the past, and I had no clue it was possible in org-mode! Thanks! You can access the org file by changing the URL extension from .html to .org. Maybe I should make that a link.

raziel2p 5y ago

Wow. I've only ever used sed for simple regex operations, reading this has opened up a whole new world for me. ... I will force myself not to make use of it because you're a bad influence and I already have to stop myself from using too much grep/sed/awk magic, but this is seriously impressive, and an entertaining read. Well done!

jolmg 5y ago
Thank you for the kind words! :)
mananaysiempre 5y ago
I can’t bring myself to write blog posts, but here’s a single-precedence-level infix-to-postfix notation transformer in sed written as part of a solution for last year’s Advent of Code 18a (https://adventofcode.com/2020/day/18 https://adventofcode.com/2020/day/18): #!/bin/sh -eu set -o pipefail sed -E \ -e 's/([+*])\s*([0-9]+)/ \2 \1 /g' \ -e ':parens' \ -e 's/([+*])\s*($[^()]+$)/ \2 \1 /g' \ -e 's/$([^()]+)$/ \1 /g' \ -e '/\(/bparens' \ -e 's/\s+/ /g' -e 's/^ //' -e 's/ $//' \ -e '$a\' -e '[+ z1<r]sr z1<r p' | \ dc (I’m very, very sorry for fuelling your addiction.) UPD: The original version claimed this was a precedence parser. It isn’t, it just does parentheses :(
- jolmg 5y ago
  Maybe I'm doing something wrong, but passing it "3 * 4 + 5 * 6" gives me 102 instead of 42.
  - mananaysiempre 5y ago
    You are correct; I seemed to remember there was a catch but was too lazy to check, sorry. This isn’t a precedence parser, that was 18b, while this one solves 18a, which was just to resolve parentheses correctly and disregard precedence. (Original comment fixed.)
nvartolomei 5y ago
How about a chess engine written in sed? https://github.com/moldabekov/chess-sed/blob/master/chess.sed https://github.com/moldabekov/chess-sed/blob/master/chess.se.... And accompanying blog post in Russian https://habr.com/ru/post/191006/ https://habr.com/ru/post/191006/.

tlackemann 5y ago

org-mode blog! How do you like it? Been considering this for myself recently but haven't looked into RSS much. I like the theme you built.

jolmg 5y ago
> org-mode blog! How do you like it? I've just gotten started! I think it's comfortable in the sense that I won't need to copy and paste stuff to and from the terminal and can just code from the same file and see how the result changes. That's really neat! I imagine I'll be feeling restricted at some point, since I'm not doing custom HTML for each post, and I want to keep the org file readable on its own, so no excessive markup. > I like the theme you built. Thanks, though it's really almost no styling. The syntax highlighting is done by the ruby library `rouge`. The colors were just black on white, but the Dark Reader Firefox extension I have installed would change that to these off-black and off-white colors, and I liked it, so I set them in the CSS.

rak1507 5y ago

Just for fun, I tried it in APL. cal ← ⎕SH 'cal -y' ⍝ get output of cal command bymonth ← {⍵⊂⍨0=23|⍳≢⍵}⍤1↑2↓cal ⍝ partition into month groups lines ← {⍵⊂⍨0=8|⍳≢⍵}⍤1⍉bymonth ⍝ partition into chunks in each line months ← ~∘' '¨,⍉⊃¨lines ⍝ extract month names and remove spaces days ← (23⍴1 0 0)⊂⍤1↑⊃,/,⌿2↓¨lines ⍝ partition into days trimmed ← ∊⍤1⍉{⍵/⍨(' '∨.≠¨⍵)∨∧\' '∧.=¨⍵}⍤1⍉¯1↓days ⍝ remove excess spaces ⎕←53↑(¯2↓⍤1⊢trimmed) , ↑months\⍨∨/' 1 '⍷trimmed ⍝ append months to calendar where the 1s are, and print Surprisingly tricky (but still easier than sed IMO :P)

eloisius 5y ago
And people say Ruby is beautiful
deleted 5y ago
[deleted]
wahern 5y ago
It's too bad APL never enjoyed ubiquity on Unix-like platforms. The world would have been a better place. Maybe if Stallman or some early GNU participant had championed APL we'd all be either using APL or constantly excusing ourselves. Like Emacs did for Lisp. I'm decent with sed. I wouldn't call myself an expert, but know how to employ addressing and hold spaces, which are sadly aracana to most people who employ sed. I ended up learning sed because it was there, proved useful, and it fit into the command-line toolkit really well. (In order of importance.) APL would never slot into that toolkit as well as other tools, but it's value-add would more than compensate. Unfortunately, the value-add isn't enough to compensate for the fact that it's not there and in your face every time you're exploring a problem and its solution space.
- jolmg 5y ago
  > Maybe if Stallman or some early GNU participant had championed APL we'd all be either using APL or constantly excusing ourselves. I think the major issue is that APL seems to require more than ASCII to work it.

helsinki 5y ago

cal -y | sed -E '1,2d;s/(.{20}) /\1\n/g;' | egrep “^. .* $“ or cal -y | sed -E '1,2d;s/(.{20}) /\1\n/g;' | python3 -c “from calendar import month_name;import sys;[print(ln.strip()) for ln in sys.stdin.readlines() if ln.strip() in month_name]”

rak1507 5y ago
Neither of them seem to work for me - what am I missing? https://tio.run/##lcxBDoIwFIThvaeYNCYUI0VwydpTiJpKCzSB16bthqhnR0hMWLv98808ZejnuZEDsglvBK2QXZAUx1JVIefiVZ4@KfK6qCnvqmQhuvPagd0FxAF7tvtn66bYWzoja8Bab0csY01KepjRWR8xWor9g@Soq18JU6iuzhuKfCARojeOpyla6zEQDK1gycqQ8FqqwZAOPIVpsfGVbc83Ns9f https://tio.run/##lcxBDoIwFIThvaeYNCYUI0VwydpTiJpKCzSB16bthq...
- helsinki 5y ago
  Sorry about that. I was typing from my phone, blindly. This is a cool website, thanks. Here is the correct solution: cal -y | sed -E '1,2d;s/(.{20}) /\1\n/g;' | python3 -c "from calendar import month_name;import sys;[print(ln.strip()) for ln in sys.stdin.readlines() if ln.strip() in month_name and len(ln.strip())]" https://tio.run/##TY1BDsIgFAWv8tJNIbGtrUvWnsIag4VaEvohwKZRz440MdHtZN68u4xLzpO0aDa8ELVCc0bdHwYlYsfa53B8c3RjP1L3EHVR/JYWRyc0E6o5uBVlrEnJALN6FxJWR2m5kVy1@JK4RXHxwVBiltqYgvGMc8wuwBIM7ULBylAbtFTWkI6Mw8z46bv2K0OSQrn9712rnD8 https://tio.run/##TY1BDsIgFAWv8tJNIbGtrUvWnsIag4VaEvohwKZRz4...
t0mas88 5y ago
That's only a list of names of months, the article creates a single column view with all the dates be between to then count the number of weeks between two dates.

saltcured 5y ago

For those who want to never learn sed addressing modes, here's a slight cheat using only simpler shell logic. The cheat is to realize you can concatenate multiple calls to `cal -1` instead of reshaping the 3x4 grid into 12x1, so just focus on the per-month splicing task... #!/bin/sh g() { read m; read n [ $1 -eq 1 ] \ && printf '%20.20s %s\n' "$n" "$m" \ || printf ' %s %s\n' "$n" "$m" sed -e 's/ *$/ /' } f() { cal -1 $@ | grep '[0-9]' | g $@ | head -c -1 } y=${1:-`date +%Y`} for m in {1..12}; do f $m $y; done printf "\n"

deleted 5y ago
[deleted]
Tepix 5y ago
On macOS you have to change it slightly to use the "-m" parameter to cal because it doesn't have the "-1" option.

deleted 5y ago

[deleted]

pixelbeat__ 5y ago

I know this is not in the spirit of the article, but getting the number of weeks is quite useful generally. One can do this with the standard date util like: $ echo $(((($(date +%s -d 'Aug 31') - $(date +%s -d 'Jul 1'))/(60*60*24*7)))) 8 # 8.7 weeks truncated $ echo $(((($(date +%W -d 'Aug 31') - $(date +%W -d 'Jul 1'))))) 9 It's useful to get the number of weeks from git logs for release notes etc., which can be done like: oldvs='v1.0' oldts=$(git log -n1 --format="%at" "$oldvs") weeks=$((($(date +%s) - $oldts)/(60*60*24*7)))

heresie-dabord 5y ago

The APL code (posted by rak1507) wins my admiration too. Now, given that this confounded problem has taken too much of my time @^_^@ I'll just say that the simplest option seems to be ncal -w -3 -y -b -h because it already computes the week numbers. Fiddle away with sed afterward. By the way, the "fold" utility [1] can help here too: ncal -w -3 -y -b -h | sed 's/ /*/g' | fold -w 27 Cheers! [1] https://en.wikipedia.org/wiki/Fold_(Unix) https://en.wikipedia.org/wiki/Fold_(Unix)

lsferreira42 5y ago

Using some shell utilities turns out to be very easy: LANG=c cal -y|grep '[a-z]'|egrep -v 'Su ' | xargs

deleted 5y ago
[deleted]

elcaro 5y ago

Just for fun, I tried it in Raku. say qx< cal -hy > # Command output as string .lines.skip.join("\n").split(/\n\n+/) # Skip year & split groups .map({ |([Z] .lines.join.comb(22).batch(3)) }) # Single column structure .map({ |(.[2] ~ .[0].trim, |.[3 .. *]) }) # Month name on first row .grep(*.trim.chars).join("\n") # Join non-blank lines .subst(/\s+ ^^ \s\s+ 1»/, ' 1', :g) # Collapse weeks together I was never going to beat APL... Though I imagine some clever Rakoons could probably find a few places where this can be golfed down.

elcaro 5y ago
The first operation where I split to lines, then join, then split again... was annoying me. So now I've got this say qx< cal -hy >.lines.skip.batch(9) .map({ |([Z] .join.comb(22).batch(3)) }) .map({ |(.[2] ~ .[0].trim, |.[3 .. *]) }) .grep(*.trim.chars).join("\n") .subst(/\s+ ^^ \s\s+ 1»/, ' 1', :g)

deleted 5y ago

[deleted]