3 ms·

Lst.sh – Portable Shell Array Library

30 points by vermaden 5y ago

CyberShadow 5y ago
Another issue with the "Once Upon a Time" style is that it's impossible to distinguish an empty list from a list containing a single, zero-length element.
_wolfie_ 5y ago
> Portable Array Library Is the $'\n' construct actually portable? I cannot find it in Shell Command Language spec [0]. 0: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... .
- CyberShadow 5y ago
  ShellCheck agrees: https://github.com/koalaman/shellcheck/wiki/SC3003 https://github.com/koalaman/shellcheck/wiki/SC3003
  - chasil 5y ago
    This isn't POSIX either: local IFS
chasil 5y ago
POSIX shells don't allow arrays, but they can be mimiced with eval. This function will construct a set of related variables that simulate an array: parr_add () { # $1:array $2:value eval parr_c=${1}_c parr_c=$((0+$parr_c)) eval ${1}_${parr_c}=\"$2\" parr_c=$((1+$parr_c)) eval ${1}_c=$parr_c unset parr_c } I could call this function from a loop that reads the password file: IFS=: while read user pw uid gid comm home shell <&9 do parr_add acc "$user" done 9< /etc/passwd Then, I could access these "array elements" in a loop: count=0 while [ $acc_c -gt $count ] do eval user=\$acc_$count printf %s\\n "$user" count=$((1+count)) done Example run: $ ./parr.sh | head -5 root bin daemon adm lp It isn't that hard to survive without arrays.
kazinator 5y ago
> Portable You immediately notice that the code uses local which POSIX mentions in a table of reserved commands that all produce unspecified results. You can't have local in a POSIX-confirming script even if you write your own function by that name. While that could still be portable to a few shells, the $'...' Bash extension makes it less so. If those things are acceptable, why not assume that the shell has Korn-shell-style arrays.
- chasil 5y ago
  In the Linux world, Debian/Ubuntu made a fateful decision to deploy the Almquist shell as /bin/sh. They left /bin/bash as the interactive shell, but anything assuming the presence of a "bashism" in /bin/sh suddenly broke. Also inconvenient is that scripts calling /bin/bash behave differently from bash running as /bin/sh (alias expansion is disabled in the former, and likely other POSIX violations that I have not seen). Almquist allows the local keyword, in violation of the POSIX standard. The wiki says that this is the only extension. The original article's motivation is for Almquist running under FreeBSD. Almquist is also used in Busybox with a few bash-compatibilty adjustments. However, there are other pure POSIX shells out there, and it is worth adhering to the standard. Almquist is also much faster than bash, so speed joins compatibility as factors for standard adherence.
  - spicybright 5y ago
    This decision has always confused me. Does debian use a bunch of scripts that rely specifically on sh or something? Is the speed hit that bad using bash?
    - chasil 5y ago
      The Debian/Ubuntu decision to forbid non-POSIX shell features was not likely made for scripts that "rely" on strict POSIX conformance, as most shells implement it (bash has a POSIXLY_CORRECT mode). I think that portability and speed were greater concerns. There were many support tickets asking for bash to be restored as /bin/sh. https://bugs.launchpad.net/ubuntu/+source/dash/+bug/141481 https://bugs.launchpad.net/ubuntu/+source/dash/+bug/141481

30 points by vermaden 5y ago

Another issue with the "Once Upon a Time" style is that it's impossible to distinguish an empty list from a list containing a single, zero-length element.

_wolfie_ 5y ago

> Portable Array Library Is the $'\n' construct actually portable? I cannot find it in Shell Command Language spec [0]. 0: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... .

CyberShadow 5y ago
ShellCheck agrees: https://github.com/koalaman/shellcheck/wiki/SC3003 https://github.com/koalaman/shellcheck/wiki/SC3003
- chasil 5y ago
  This isn't POSIX either: local IFS

chasil 5y ago

POSIX shells don't allow arrays, but they can be mimiced with eval. This function will construct a set of related variables that simulate an array: parr_add () { # $1:array $2:value eval parr_c=${1}_c parr_c=$((0+$parr_c)) eval ${1}_${parr_c}=\"$2\" parr_c=$((1+$parr_c)) eval ${1}_c=$parr_c unset parr_c } I could call this function from a loop that reads the password file: IFS=: while read user pw uid gid comm home shell <&9 do parr_add acc "$user" done 9< /etc/passwd Then, I could access these "array elements" in a loop: count=0 while [ $acc_c -gt $count ] do eval user=\$acc_$count printf %s\\n "$user" count=$((1+count)) done Example run: $ ./parr.sh | head -5 root bin daemon adm lp It isn't that hard to survive without arrays.

kazinator 5y ago

> Portable You immediately notice that the code uses local which POSIX mentions in a table of reserved commands that all produce unspecified results. You can't have local in a POSIX-confirming script even if you write your own function by that name. While that could still be portable to a few shells, the $'...' Bash extension makes it less so. If those things are acceptable, why not assume that the shell has Korn-shell-style arrays.

chasil 5y ago
In the Linux world, Debian/Ubuntu made a fateful decision to deploy the Almquist shell as /bin/sh. They left /bin/bash as the interactive shell, but anything assuming the presence of a "bashism" in /bin/sh suddenly broke. Also inconvenient is that scripts calling /bin/bash behave differently from bash running as /bin/sh (alias expansion is disabled in the former, and likely other POSIX violations that I have not seen). Almquist allows the local keyword, in violation of the POSIX standard. The wiki says that this is the only extension. The original article's motivation is for Almquist running under FreeBSD. Almquist is also used in Busybox with a few bash-compatibilty adjustments. However, there are other pure POSIX shells out there, and it is worth adhering to the standard. Almquist is also much faster than bash, so speed joins compatibility as factors for standard adherence.
- spicybright 5y ago
  This decision has always confused me. Does debian use a bunch of scripts that rely specifically on sh or something? Is the speed hit that bad using bash?
  - chasil 5y ago
    The Debian/Ubuntu decision to forbid non-POSIX shell features was not likely made for scripts that "rely" on strict POSIX conformance, as most shells implement it (bash has a POSIXLY_CORRECT mode). I think that portability and speed were greater concerns. There were many support tickets asking for bash to be restored as /bin/sh. https://bugs.launchpad.net/ubuntu/+source/dash/+bug/141481 https://bugs.launchpad.net/ubuntu/+source/dash/+bug/141481