4 ms·

This fix to me reduces the performance for nothing. In Linux (or most general on any UNIX system that I saw) a path should not be longer (total) than PATH_MAX,

This fix to me reduces the performance for nothing. In Linux (or most general on any UNIX system that I saw) a path should not be longer (total) than PATH_MAX, that is typically defined to 4096 bytes. What is the point on allocating something statically at this point? And yes, I know that really that is only a limit of the system call path lenght, and in theory you can work with longer paths (by changing the current directory to a path and then opening a file from there), because filesystems does (stupidly in my opinion) support it. But in reality, how many applications will break? Does it make sense to support them? Also the code in question seems to be dealing with a filename more than a path. A file name shouldn't be longer than NAME_MAX, and that is an hard limit of many (possibly all?) filesystems, as far as I know. So why? It would be simpler and more optimized to just truncate the name at PATH_MAX. Avoid the overflow and the crash but give an error. Why hard limits are considered that bad? We waste time supporting edge cases that no one would really use in a real system (no way someone needs a path longer than 4096 bytes...), for what? In Windows the limit is 260 characters, and nobody seems to be bothered by that, only in Windows 10 you can increase that.

wahern 5y ago
The Linux kernel doesn't have an actual path limit. Nor does Solaris. PATH_MAX is 4096 in glibc and musl libc because setting to it to something like INT_MAX or ULONG_MAX would break a lot of existing code that uses PATH_MAX to size buffers. (Though Solaris does define it as INT_MAX, IIRC.) OTOH, because of the lack of a hard limit there's also code that relies (if accidentally) on paths longer than PATH_MAX.
- jwilk 5y ago
  Linux does have a limit, at least for some system calls: $ strace -e trace=file perl -e 'open(FH, "<", "/" x 4096)' … openat(AT_FDCWD, "//////////…"..., O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENAMETOOLONG (File name too long) +++ exited with 0 +++
  - wahern 5y ago
    I stand corrected. It seems that Linux copies the entire path into a kernel-allocated buffer (see getname and getname_flags in fs/namei.c as called by various syscalls in fs/open.c), rejecting paths longer than PATH_MAX. EDIT: And on Solaris PATH_MAX is 1024 and (AFAICT) Solaris also copies paths into kernel space. It seems I was confusing things with NL_TEXTMAX, which is INT_MAX on glibc (but not Solaris).
  - yencabulator 5y ago
    There's a limit to what path string you can make the kernel interpret. That does not limit total path length. Keep looping on mkdirat/openat and you can make very very deep trees. As opposed to your syscall that is relative to an arbitrary directory, /proc/self/mountinfo has to contain the whole absolute path to be useful.
- yrro 5y ago
  https://eklitzke.org/path-max-is-tricky https://eklitzke.org/path-max-is-tricky The Linux kernel defines upper limits for NAME_MAX (255) and PATH_MAX (4096). The glibc doesn't enforce this limit because it was originally written to run on GNU HURD which I guess doesn't have these limits. But systemd only runs on glibc on Linux. So I don't see why it doesn't at least sanity check the length of absolute paths with PATH_MAX...
ohazi 5y ago
> reduces the performance for nothing Does the code in question ever run in a tight loop (e.g. on file operations after the filesystem is mounted), or just at mount time? If it's just at mount time, "reducing performance" by one malloc vs a stack adjustment doesn't seem to me like it should be a primary concern.
- alerighi 5y ago
  We are talking about systemd, that is a core software. I would like systemd to do as few memory allocations as possible. The reason is that memory can run out, especially on embedded devices where you have for example 32Mb of RAM, and you have to properly manage the case that you run out of memory. Most programmers don't, and the program does crash in case you don't have memory available. That is bad for PID 1, because that would mean a kernel panic, that you don't want. If PID 1 does need to do that kind of stuff (as it seems), I would prefer it to fork a process and do the memory allocation in that, so if that process crashes because you are out of memory the kernel doesn't panic.
josefx 5y ago
> In Linux (or most general on any UNIX system that I saw) a path should not be longer (total) than PATH_MAX, that is typically defined to 4096 bytes. That almost sounds like the 260 character windows path limit constant used by some ancient APIs. I would assume that any API limited to that path length is dated and probably unreliable in various contexts as the wikipedia article on filesystems explicitly gives the limit as not defined for various Linux filesystems. Also given the recent talk about in kernel support for NTFS (path limit ~2^16) I assume that any historic code still relying on PATH_MAX needs to be fixed.
- alerighi 5y ago
  The filesystem can support path length even infinite (simple, make a symlink of a directory inside that directory, you have an infinite path). PATH_MAX is a limit of a path that you can pass to the various path manipulating functions, open(), unlink(), etc, or returned by getcwd() (that gives an error if path is longer than PATH_MAX, and yes there are non standard system call to go around this limit but... why?) You can however use paths longer by PATH_MAX, how? Simply chdir() PATH_MAX, then you can chdir() another PATH_MAX, then count how many software breaks... Imposing a limit on paths makes sense and should be done. 4096 bytes seems reasonable to me. Also, in that example, it wasn't even a matter of a path! They are parsing it seems only the file name, and that is defined to be NAME_MAX, that is 255 bytes, on every system and every filesystem!

wahern 5y ago

The Linux kernel doesn't have an actual path limit. Nor does Solaris. PATH_MAX is 4096 in glibc and musl libc because setting to it to something like INT_MAX or ULONG_MAX would break a lot of existing code that uses PATH_MAX to size buffers. (Though Solaris does define it as INT_MAX, IIRC.) OTOH, because of the lack of a hard limit there's also code that relies (if accidentally) on paths longer than PATH_MAX.

jwilk 5y ago
Linux does have a limit, at least for some system calls: $ strace -e trace=file perl -e 'open(FH, "<", "/" x 4096)' … openat(AT_FDCWD, "//////////…"..., O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENAMETOOLONG (File name too long) +++ exited with 0 +++
- wahern 5y ago
  I stand corrected. It seems that Linux copies the entire path into a kernel-allocated buffer (see getname and getname_flags in fs/namei.c as called by various syscalls in fs/open.c), rejecting paths longer than PATH_MAX. EDIT: And on Solaris PATH_MAX is 1024 and (AFAICT) Solaris also copies paths into kernel space. It seems I was confusing things with NL_TEXTMAX, which is INT_MAX on glibc (but not Solaris).
- yencabulator 5y ago
  There's a limit to what path string you can make the kernel interpret. That does not limit total path length. Keep looping on mkdirat/openat and you can make very very deep trees. As opposed to your syscall that is relative to an arbitrary directory, /proc/self/mountinfo has to contain the whole absolute path to be useful.
yrro 5y ago
https://eklitzke.org/path-max-is-tricky https://eklitzke.org/path-max-is-tricky The Linux kernel defines upper limits for NAME_MAX (255) and PATH_MAX (4096). The glibc doesn't enforce this limit because it was originally written to run on GNU HURD which I guess doesn't have these limits. But systemd only runs on glibc on Linux. So I don't see why it doesn't at least sanity check the length of absolute paths with PATH_MAX...

ohazi 5y ago

> reduces the performance for nothing Does the code in question ever run in a tight loop (e.g. on file operations after the filesystem is mounted), or just at mount time? If it's just at mount time, "reducing performance" by one malloc vs a stack adjustment doesn't seem to me like it should be a primary concern.

alerighi 5y ago
We are talking about systemd, that is a core software. I would like systemd to do as few memory allocations as possible. The reason is that memory can run out, especially on embedded devices where you have for example 32Mb of RAM, and you have to properly manage the case that you run out of memory. Most programmers don't, and the program does crash in case you don't have memory available. That is bad for PID 1, because that would mean a kernel panic, that you don't want. If PID 1 does need to do that kind of stuff (as it seems), I would prefer it to fork a process and do the memory allocation in that, so if that process crashes because you are out of memory the kernel doesn't panic.

josefx 5y ago

> In Linux (or most general on any UNIX system that I saw) a path should not be longer (total) than PATH_MAX, that is typically defined to 4096 bytes. That almost sounds like the 260 character windows path limit constant used by some ancient APIs. I would assume that any API limited to that path length is dated and probably unreliable in various contexts as the wikipedia article on filesystems explicitly gives the limit as not defined for various Linux filesystems. Also given the recent talk about in kernel support for NTFS (path limit ~2^16) I assume that any historic code still relying on PATH_MAX needs to be fixed.

alerighi 5y ago
The filesystem can support path length even infinite (simple, make a symlink of a directory inside that directory, you have an infinite path). PATH_MAX is a limit of a path that you can pass to the various path manipulating functions, open(), unlink(), etc, or returned by getcwd() (that gives an error if path is longer than PATH_MAX, and yes there are non standard system call to go around this limit but... why?) You can however use paths longer by PATH_MAX, how? Simply chdir() PATH_MAX, then you can chdir() another PATH_MAX, then count how many software breaks... Imposing a limit on paths makes sense and should be done. 4096 bytes seems reasonable to me. Also, in that example, it wasn't even a matter of a path! They are parsing it seems only the file name, and that is defined to be NAME_MAX, that is 255 bytes, on every system and every filesystem!