4 ms·

I always use mktemp and trap to remove tmp files. But in this script i would like to avoid external commands. I will even remove the use of tmp files kn thw fu

I always use mktemp and trap to remove tmp files. But in this script i would like to avoid external commands. I will even remove the use of tmp files kn thw future.

goombacloud 5y ago
To spare you the trap you can open the file as FD in your bash process (e.g., exec {my_fd}>"$TMPFILE") and then directly delete it before doing anything else, and use the /proc handle to access it ("/proc/$$/fd/${my_fd}")
- goombacloud 5y ago
  One more fun experiment: avoid the tempfile on disk/tmpfs by storing data into the pipe connecting two processes, where the first one can already exit for the pipe to give EOF when reading beyond the buffered data. true | sleep infinity & STDINPID="$!" TMPPIPE="/proc/$STDINPID/fd/0" echo hello > "$TMPPIPE" echo write more > "$TMPPIPE" cat "$TMPPIPE" echo write again > "$TMPPIPE" cat "$TMPPIPE" kill -9 "$STDINPID" # clean up pipe Now I would like to avoid the additional process by using the current shell process instead.
  - goombacloud 5y ago
    Still needs to spawn a temporary process but it can be killed early: true | sleep infinity & STDINPID="$!" exec {mypipe}<"/proc/$STDINPID/fd/0" # hijack FD TMPPIPE="/proc/$$/fd/${mypipe}" disown "$STDINPID" # suppress killed message on stderr kill -9 "$STDINPID" # clean up process now already echo hello > "$TMPPIPE" echo write more > "$TMPPIPE" cat "$TMPPIPE" echo write again > "$TMPPIPE" cat "$TMPPIPE"
    - Klasiaster 5y ago
      Seems "sleep 60" is safer and still good enough unless the system is completely overloaded.
    - dzove855 5y ago
      Since now, multiple concurrent connections will work after my patch. (Need to add subshell) i will need to consider using fd as output cache. Instead of temporary files.
      - Klasiaster 5y ago
        This here is a simple echo server that uses tr to uppercase: exec {checkfd}>/dev/null CHECKFDPATH="/proc/$$/fd/${checkfd}" (while [ -e "$CHECKFDPATH" ]; do sleep 1; done) > >(true) & STDINPID="$!" WRITER="/proc/$STDINPID/fd/1" while IFS= read -r LINE; do # only echo lines if we didn't close the connection yet if [ -e "$CHECKFDPATH" ]; then echo "$LINE" | tr '[:lower:]' '[:upper:]' > "$WRITER" if [ "$LINE" = "bye" ]; then exec {checkfd}<&- fi else echo "received while closed: $LINE" fi done < <(nc -q 1 -l 8080 < "$WRITER") if [ -e "$CHECKFDPATH" ]; then # close checkfd to close writer pipe exec {checkfd}<&- fi
        Klasiaster 5y ago
        This variant here works a bit different: exec {checkfd}>/dev/null CHECKFDPATH="/proc/$$/fd/${checkfd}" (while [ -e "$CHECKFDPATH" ]; do sleep 1; done) > >(true) & STDINPID="$!" disown "$STDINPID" READER="/proc/$STDINPID/fd/1" { while IFS= read -r LINE; do echo "$LINE" | tr '[:lower:]' '[:upper:]' if [ "$LINE" = "bye" ]; then echo exiting > /dev/stderr break fi done < "$READER" ; kill -9 "$STDINPID" 2>/dev/null || true ; } | { nc -q 1 -l 8080 > "$READER" ; kill -9 "$STDINPID" 2>/dev/null || true ; } exec {checkfd}<&-
      - dzove855 5y ago
        No need of tr here in your example. The fd will only be used to stor it temporary since, we first need to sent headers, and the headers need to be sent before the bofy. The server base os working fine now with accept and a patch request os already sent to bash. So probaböy on the next release accept will be full usable.
- ghthor 5y ago
  That's super clever, thanks for pointing this out!
- stouset 5y ago
  That's an awesome technique, thanks for it! It is Linux-specific though.
  - Klasiaster 5y ago
    It's also useful for signaling between processes to have them continue doing something as long as this proc path exists (the fd can also just be backed with >/dev/null instead of a real file that needs to be removed)
    - dzove855 5y ago
      For creating some kind IPC, i would create a named fifo or even use coproc (see man bash)
    - labawi 5y ago
      Normally, the pid and fd could be reused, so you should either make it a known unique path and verify it's still the same, lock the pid via some means (not sure if and how it's possible), or use another technique.
mmmooo 5y ago
can use $RANDOM to get more entropy/avoid collisions, though as you said, better not to use files at all.

I always use mktemp and trap to remove tmp files. But in this script i would like to avoid external commands. I will even remove the use of tmp files kn thw future.

goombacloud 5y ago

To spare you the trap you can open the file as FD in your bash process (e.g., exec {my_fd}>"$TMPFILE") and then directly delete it before doing anything else, and use the /proc handle to access it ("/proc/$$/fd/${my_fd}")

goombacloud 5y ago
One more fun experiment: avoid the tempfile on disk/tmpfs by storing data into the pipe connecting two processes, where the first one can already exit for the pipe to give EOF when reading beyond the buffered data. true | sleep infinity & STDINPID="$!" TMPPIPE="/proc/$STDINPID/fd/0" echo hello > "$TMPPIPE" echo write more > "$TMPPIPE" cat "$TMPPIPE" echo write again > "$TMPPIPE" cat "$TMPPIPE" kill -9 "$STDINPID" # clean up pipe Now I would like to avoid the additional process by using the current shell process instead.
- goombacloud 5y ago
  Still needs to spawn a temporary process but it can be killed early: true | sleep infinity & STDINPID="$!" exec {mypipe}<"/proc/$STDINPID/fd/0" # hijack FD TMPPIPE="/proc/$$/fd/${mypipe}" disown "$STDINPID" # suppress killed message on stderr kill -9 "$STDINPID" # clean up process now already echo hello > "$TMPPIPE" echo write more > "$TMPPIPE" cat "$TMPPIPE" echo write again > "$TMPPIPE" cat "$TMPPIPE"
  - Klasiaster 5y ago
    Seems "sleep 60" is safer and still good enough unless the system is completely overloaded.
  - dzove855 5y ago
    Since now, multiple concurrent connections will work after my patch. (Need to add subshell) i will need to consider using fd as output cache. Instead of temporary files.
    - Klasiaster 5y ago
      This here is a simple echo server that uses tr to uppercase: exec {checkfd}>/dev/null CHECKFDPATH="/proc/$$/fd/${checkfd}" (while [ -e "$CHECKFDPATH" ]; do sleep 1; done) > >(true) & STDINPID="$!" WRITER="/proc/$STDINPID/fd/1" while IFS= read -r LINE; do # only echo lines if we didn't close the connection yet if [ -e "$CHECKFDPATH" ]; then echo "$LINE" | tr '[:lower:]' '[:upper:]' > "$WRITER" if [ "$LINE" = "bye" ]; then exec {checkfd}<&- fi else echo "received while closed: $LINE" fi done < <(nc -q 1 -l 8080 < "$WRITER") if [ -e "$CHECKFDPATH" ]; then # close checkfd to close writer pipe exec {checkfd}<&- fi
      - Klasiaster 5y ago
        This variant here works a bit different: exec {checkfd}>/dev/null CHECKFDPATH="/proc/$$/fd/${checkfd}" (while [ -e "$CHECKFDPATH" ]; do sleep 1; done) > >(true) & STDINPID="$!" disown "$STDINPID" READER="/proc/$STDINPID/fd/1" { while IFS= read -r LINE; do echo "$LINE" | tr '[:lower:]' '[:upper:]' if [ "$LINE" = "bye" ]; then echo exiting > /dev/stderr break fi done < "$READER" ; kill -9 "$STDINPID" 2>/dev/null || true ; } | { nc -q 1 -l 8080 > "$READER" ; kill -9 "$STDINPID" 2>/dev/null || true ; } exec {checkfd}<&-
    - dzove855 5y ago
      No need of tr here in your example. The fd will only be used to stor it temporary since, we first need to sent headers, and the headers need to be sent before the bofy. The server base os working fine now with accept and a patch request os already sent to bash. So probaböy on the next release accept will be full usable.
ghthor 5y ago
That's super clever, thanks for pointing this out!
stouset 5y ago
That's an awesome technique, thanks for it! It is Linux-specific though.
- Klasiaster 5y ago
  It's also useful for signaling between processes to have them continue doing something as long as this proc path exists (the fd can also just be backed with >/dev/null instead of a real file that needs to be removed)
  - dzove855 5y ago
    For creating some kind IPC, i would create a named fifo or even use coproc (see man bash)
  - labawi 5y ago
    Normally, the pid and fd could be reused, so you should either make it a known unique path and verify it's still the same, lock the pid via some means (not sure if and how it's possible), or use another technique.

mmmooo 5y ago

can use $RANDOM to get more entropy/avoid collisions, though as you said, better not to use files at all.