4 ms·

Can you imagine trying to have meaningful file names today on these new 20+ TB drives that hold a few hundred million files, if file names were still limited to

Can you imagine trying to have meaningful file names today on these new 20+ TB drives that hold a few hundred million files, if file names were still limited to 8 or even 14 characters?

megapoliss 1y ago
For everage user nothing changes - each folder will just have "thumb.db" with all metadata and long names.
layer8 1y ago
You could maintain an index for the 2^64 or 2^112 names this allows.
duped 1y ago
PATH_MAX is still comically small today.
- throw0101b 1y ago
  > PATH_MAX is still comically small today. Per getconf -a, I see PATH_MAX (and _POSIX_PATH_MAX) as 4096. Is that small? What would be not-small?
  - duped 1y ago
    Yes that's small, it's also incorrect. The correct thing is to recognize PATH_MAX can't be defined by the kernel or in limits.h, so what you're seeing is a hint and not the actual limit. "Not small" is "limited only by resource constraints." Software often breaks when it hits large (but correct!) paths even though there's no technical limitation to using them, and valid ways to construct them, even if POSIX APIs are required by spec to fail for some valid paths because of arbitrary limits. Linux is actually pretty good about ignoring unnecessary error conditions even if it violates the spec, other unixes not so much.
    - wahern 1y ago
      The Linux kernel doesn't actually support path names longer than 4095 (4096 with NUL). See fs/namei.c:getname_flags, which is used early in every syscall that takes a file path. That function will attempt to copy the path into kernel space, but fails when the length exceeds 4095: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/namei.c?id=181d8e39#n207 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... (strncpy_from_user is defined in lib/strncpy_from_user.c) IIRC, some real-world systems set PATH_MAX to INT_MAX (Solaris?), but I don't know if any modern Unix systems support arbitrary length paths. Everybody demanded features that required the kernel to cache the path in the kernel, at which point the notion of just walking the userspace path buffer (no separate allocation, no need for a limit) went out the window. EDIT: glibc sets NL_TEXTMAX to INT_MAX. I always get confused when discussing this issue. NL_TEXTMAX made good on the threat that these MAX macros might be (effectively) unlimited and so shouldn't be used to size buffers, but I don't think any system ever did so with PATH_MAX.
    - AStonesThrow 1y ago
      So, a user should be able to exhaust resources by making paths longer and longer until it just errors out? And programmers should allocate buffers of absurd size in order to contain pathnames of unpredictable depth?
      - rurban 1y ago
        Also paths need to stay identifiable. And you would not be able to recognize differences of such overlong identifiers in the middle. Unicode confusables aside
- lupusreal 1y ago
  I regularly hit it when using yt-dlp, particularly from twitter where it puts the text of the tweet into the file name by default.
  - jwilk 1y ago
    That's more likely NAME_MAX, which is 255.
NoMoreNicksLeft 1y ago
Yeh, but everyone still has DOS PTSD. Underscores instead of spaces, sticking to 7bit ascii characters, etc. so not much has changed. On the rare occasion that I need a slash in a filename, I've been using the full-width solidus unicode character... Also discovered that I could fool the Macos Finder file sorting the other day if I put zero-width spaces in between numerals in the filename. Not sure how I feel about that one though.
- hulitu 1y ago
  > Underscores instead of spaces After being biten so many times with spaces or special characters in filenames, one learns.

Can you imagine trying to have meaningful file names today on these new 20+ TB drives that hold a few hundred million files, if file names were still limited to 8 or even 14 characters?

megapoliss 1y ago

For everage user nothing changes - each folder will just have "thumb.db" with all metadata and long names.

layer8 1y ago

You could maintain an index for the 2^64 or 2^112 names this allows.

duped 1y ago

PATH_MAX is still comically small today.

throw0101b 1y ago
> PATH_MAX is still comically small today. Per getconf -a, I see PATH_MAX (and _POSIX_PATH_MAX) as 4096. Is that small? What would be not-small?
- duped 1y ago
  Yes that's small, it's also incorrect. The correct thing is to recognize PATH_MAX can't be defined by the kernel or in limits.h, so what you're seeing is a hint and not the actual limit. "Not small" is "limited only by resource constraints." Software often breaks when it hits large (but correct!) paths even though there's no technical limitation to using them, and valid ways to construct them, even if POSIX APIs are required by spec to fail for some valid paths because of arbitrary limits. Linux is actually pretty good about ignoring unnecessary error conditions even if it violates the spec, other unixes not so much.
  - wahern 1y ago
    The Linux kernel doesn't actually support path names longer than 4095 (4096 with NUL). See fs/namei.c:getname_flags, which is used early in every syscall that takes a file path. That function will attempt to copy the path into kernel space, but fails when the length exceeds 4095: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/namei.c?id=181d8e39#n207 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... (strncpy_from_user is defined in lib/strncpy_from_user.c) IIRC, some real-world systems set PATH_MAX to INT_MAX (Solaris?), but I don't know if any modern Unix systems support arbitrary length paths. Everybody demanded features that required the kernel to cache the path in the kernel, at which point the notion of just walking the userspace path buffer (no separate allocation, no need for a limit) went out the window. EDIT: glibc sets NL_TEXTMAX to INT_MAX. I always get confused when discussing this issue. NL_TEXTMAX made good on the threat that these MAX macros might be (effectively) unlimited and so shouldn't be used to size buffers, but I don't think any system ever did so with PATH_MAX.
  - AStonesThrow 1y ago
    So, a user should be able to exhaust resources by making paths longer and longer until it just errors out? And programmers should allocate buffers of absurd size in order to contain pathnames of unpredictable depth?
    - rurban 1y ago
      Also paths need to stay identifiable. And you would not be able to recognize differences of such overlong identifiers in the middle. Unicode confusables aside
lupusreal 1y ago
I regularly hit it when using yt-dlp, particularly from twitter where it puts the text of the tweet into the file name by default.
- jwilk 1y ago
  That's more likely NAME_MAX, which is 255.

NoMoreNicksLeft 1y ago

Yeh, but everyone still has DOS PTSD. Underscores instead of spaces, sticking to 7bit ascii characters, etc. so not much has changed. On the rare occasion that I need a slash in a filename, I've been using the full-width solidus unicode character... Also discovered that I could fool the Macos Finder file sorting the other day if I put zero-width spaces in between numerals in the filename. Not sure how I feel about that one though.

hulitu 1y ago
> Underscores instead of spaces After being biten so many times with spaces or special characters in filenames, one learns.