4 ms·
So then, what is the one liner to preserve the filename rather than get a new deduped.txt? Also how do you apply that command to the next file using shell hist
by triangleman 7y ago
So then, what is the one liner to preserve the filename rather than get a new deduped.txt?
Also how do you apply that command to the next file using shell history?
- gcmeplz 7y agoUse sponge! https://linux.die.net/man/1/sponge https://linux.die.net/man/1/sponge awk '!n[$0]++' fileName | sponge fileName
- Vogtinator 7y agoOr using gawk, gawk -i inplace 'foo {bar}' file
- dredmorbius 7y agodedupe.awk <file >file.tmp && mv file.tmp file Multifile versions vary, I'd prefer listing them out, alternatively you could read from a command output (ls, find, etc.) with a 'while read; do ... done' loop: for f in file1 file2 file3 do dedupe.awk <$f > ${f}.tmp && mv ${f}.tmp $f done If you want to apply to specific files on an ad hoc basis, you could wrap the whole thing in a shell function with filename or list as a parameter. Or 'gawk -i' as suggested. Properly using tempfile would also be an improvement.