4 ms·
"Have you ever wanted to process a file through a pipeline and write the output back to the file?" Personally, no. I prefer to output text processsing results
by 1vuio0pswjnm7 5y ago
"Have you ever wanted to process a file through a pipeline and write the output back to the file?"
Personally, no. I prefer to output text processsing results to a different file, maybe a temporary one, then replace the first file with the second file after looking at the second file.
awk '{do_stuff()}' 1.txt | sort -u | column --table > 2.txt
less 2.txt
mv 2.txt 1.txt
With this sponge program I cannot check the second file before replacing the first one. The only reason I can think of not to use a second file is if I did not have enough space for the second file. In that case, I would use a fifo as the second file. (I would still look at the fifo first before overwriting the first file.)
mkfifo 1.fifo
awk '{do_stuff()}' 1.txt | sort -u | column --table > 1.fifo &
less -f 1.fifo
awk '{do_stuff()}' 1.txt | sort -u | column --table > 1.fifo &
cat 1.fifo > 1.txt
- ikiris 5y agoYeah they should have called that tool yolocat
- pletnes 5y agoIf 2.txt is tracked by git, there’s no need to go through these hoops. Then, sponge starts to make sense. Except from that case, I agree totally.
- 1vuio0pswjnm7 5y agoDid git exist when this sponge program was written?
- varenc 5y agoYou’re implying that all uses of `sponge` are interactive with a user actively involved. I used sponge a lot in scripts where I’ve already checked and validated its behavior. When I’m confident it works I can keep just use sponge for simplicity.
- deleted 5y ago[deleted]
- TowerTall 5y agoNo backups?
- hulitu 5y agoReal programmers don't make backups.
- mrlonglong 5y agoReal programmers throw their source code out onto the internet.
- efreak 5y agoReal programmers don't keep source code.
- salamander014 5y agoReal programmers leave for more pay before somebody asked about backups.
- dspillett 5y agoPerhaps the backups are a separate process, run beforehand or after this process once several changes (perhaps to several files) have been made, so not backing up intermediate states?
- whirlwin 5y agoBackups should be handled by an external process, otherwise, your script also needs to take into account: - Number of backups to retain and cleaning up old versions - Monitoring of backups. What if backing up fails? - The script itself can tamper with the backups unintentionally? Speaking from (bad) experience
- 1vuio0pswjnm7 5y agosponge as a shell script for non-interactive use, i.e., where we do not need to check the second file #!/bin/sh test $# = 1||exec echo usage: $0 file; test -f $1||exec echo $1 does not exist; test ! -f $1.tmp||exec echo $1.tmp exists; cat > $1.tmp; mv $1.tmp $1;
- aix1 5y agoThe script should probably be using `mktemp` instead of potentially overwriting uninvolved files (even if they have `.tmp` in the name).
- progval 5y agoWhen you do, just make sure you use the `--directory` option of `mktemp` to use the same directory as the final file. This makes sure both files are on the same filesystem, so `mv` is atomic (and faster).
- sdoering 5y agoAnd so I learned of mktemp today. Just was intrigued, googled it, read about it and know it solves a problem I had elegantly. Some new tool in my belt from now on. Thank you.
- tzs 5y agoFor a simple case like this I'd probably just go with something like tmp.$$.tmp.
- martinflack 5y agoYou might need a -f on that final mv. While you're at it, may as well create a backup of the $1 file.
- 1vuio0pswjnm7 5y agosponge creates no backups Never have I ever needed -f.