3 ms·

It's strange that ls has grown to 58 options, but still can't output \0-terminated filenames: https://unix.stackexchange.com/questions/112125/is-there-a-reason

by p1mrx 7y ago

It's strange that ls has grown to 58 options, but still can't output \0-terminated filenames: https://unix.stackexchange.com/questions/112125/is-there-a-reason-why-ls-does-not-have-a-zero-or-0-option https://unix.stackexchange.com/questions/112125/is-there-a-r... As an exercise, try to sort a directory by size or date, and pass the result to xargs, while supporting any valid filename. I eventually just gave up and made my script ignore any filenames containing \n.

whelming_wave 7y ago
Here you go: sort all files in the current directory by modification time, whitespace-in-filenames-safe. The `printf (od -> sed)' construction converts back out of null-separated characters into newline-separated, though feel free to replace that with anything accepting null-separated input. Granted, `sort --zero-terminated' is a GNU extension and kinda cheating, but it's even available on macOS so it's probably fine. printf '%b' $( find . -maxdepth 1 -exec sh -c ' printf '\''%s %s\0'\'' "$(stat -f '\''%m'\'' "$1")" "$1" ' sh {} \; | \ sort --zero-terminated | \ od -v -b | \ sed 's/^[^ ]*// s/ *$// s/ */ \\/g s/\\000/\\012/g') If you're running this under zsh, you'll need to prefix it with `command' to use the system executable: zsh's builtin printf doesn't support printing octal escape codes for normally printable characters, and you may have to assign the output to a variable and explicitly word-split it. This is all POSIX as far as I know, except for the sort.
ailideex 7y ago
Another alternative find /etc/ -mindepth 1 -maxdepth 1 -type f \ -printf '%s:%T@:%p\000' \ | sort -z -t: -k1,1g \ | sed -E -z 's/^[0-9]+:[0-9.]+://g' \ | tr '\000' '\n' Sorts by size, can just change -k1,1 to -k2,2 to sort by date. Tested on fedora 31 with gnu find, sed, sort and tr. And really the answer given at the stackexchange link is quite on point: > However ls is really a tool for direct consumption by a human, and in that case further processing is less useful. For futher processing, find(1) is more suited. Find is the right solution, find gives you an enormous amount of control, to re-implement find in ls seems counter productive.
loeg 7y ago
By size: python -c 'import os; \ os.write(1, b"\0".join( \ sorted(os.listdir(b"."), key=lambda p: os.stat(p).st_size)))' \ | xargs -0 ... By date: swap st_size for st_mtime. (Python3 does try to make this hard on us — Unix path names are just bags of bytes, but Python insists that printing must have some known encoding. I get around that by abusing os.write(1, ...) and by adding the dummy b"." argument to os.listdir(), which forces the results to be in bytes format.)
jackewiehose 7y ago
> I eventually just gave up and made my script ignore any filenames containing \n I think we can all do that. Did anyone ever came across a legitimate reason for filenames with newlines? All I can think of are intentionally bad-named files to exploit bugs. Those filenames should be avoided in the first place.
- strbean 7y ago
  Seems like a great argument for ensuring those are handled correctly, so you don't get bit by a maliciously named file.
  - oblio 7y ago
    He is handling it correctly, he's ignoring those files.
    - why-oh-why 7y ago
      Maybe being ignored (i.e. hidden) is the malicious intent. What if the largest file on your drive has that filename and no tool can show it?

by p1mrx 7y ago

whelming_wave 7y ago

Here you go: sort all files in the current directory by modification time, whitespace-in-filenames-safe. The `printf (od -> sed)' construction converts back out of null-separated characters into newline-separated, though feel free to replace that with anything accepting null-separated input. Granted, `sort --zero-terminated' is a GNU extension and kinda cheating, but it's even available on macOS so it's probably fine. printf '%b' $( find . -maxdepth 1 -exec sh -c ' printf '\''%s %s\0'\'' "$(stat -f '\''%m'\'' "$1")" "$1" ' sh {} \; | \ sort --zero-terminated | \ od -v -b | \ sed 's/^[^ ]*// s/ *$// s/ */ \\/g s/\\000/\\012/g') If you're running this under zsh, you'll need to prefix it with `command' to use the system executable: zsh's builtin printf doesn't support printing octal escape codes for normally printable characters, and you may have to assign the output to a variable and explicitly word-split it. This is all POSIX as far as I know, except for the sort.

ailideex 7y ago

Another alternative find /etc/ -mindepth 1 -maxdepth 1 -type f \ -printf '%s:%T@:%p\000' \ | sort -z -t: -k1,1g \ | sed -E -z 's/^[0-9]+:[0-9.]+://g' \ | tr '\000' '\n' Sorts by size, can just change -k1,1 to -k2,2 to sort by date. Tested on fedora 31 with gnu find, sed, sort and tr. And really the answer given at the stackexchange link is quite on point: > However ls is really a tool for direct consumption by a human, and in that case further processing is less useful. For futher processing, find(1) is more suited. Find is the right solution, find gives you an enormous amount of control, to re-implement find in ls seems counter productive.

loeg 7y ago

By size: python -c 'import os; \ os.write(1, b"\0".join( \ sorted(os.listdir(b"."), key=lambda p: os.stat(p).st_size)))' \ | xargs -0 ... By date: swap st_size for st_mtime. (Python3 does try to make this hard on us — Unix path names are just bags of bytes, but Python insists that printing must have some known encoding. I get around that by abusing os.write(1, ...) and by adding the dummy b"." argument to os.listdir(), which forces the results to be in bytes format.)

jackewiehose 7y ago

> I eventually just gave up and made my script ignore any filenames containing \n I think we can all do that. Did anyone ever came across a legitimate reason for filenames with newlines? All I can think of are intentionally bad-named files to exploit bugs. Those filenames should be avoided in the first place.

strbean 7y ago
Seems like a great argument for ensuring those are handled correctly, so you don't get bit by a maliciously named file.
- oblio 7y ago
  He is handling it correctly, he's ignoring those files.
  - why-oh-why 7y ago
    Maybe being ignored (i.e. hidden) is the malicious intent. What if the largest file on your drive has that filename and no tool can show it?