3 ms·

We wanted to switch to a Linux-based NAS. There was a major issue though: EXT4 does not support long enough directory / file names (or paths). As a result of ou

by chmars 8y ago

We wanted to switch to a Linux-based NAS. There was a major issue though: EXT4 does not support long enough directory / file names (or paths). As a result of our extensive folder structure and file naming scheme, we ran into all sorts of issues. In the end, we made the jump to Google Drive where security is probably better anyway in comparison to a local NAS. (Your mileage may vary as usual.)

e12e 8y ago
I don't think there's a path limit in ext4, but looks like individual file/folders are limited to 255 bytes, while hfs+ (on Linux) is 255 utf-16 characters, which is potentially longer. The 255 bytes limit seems common (xfs, zfs)-while reiser allows 4kb or so. What kind of naming scheme do you have that you had a problem? Long, multibyte file names?
- cesarb 8y ago
  IIRC, there's a limit that the whole path passed to system calls has to fit into a single page. For most architectures, that's 4096 bytes including the terminating nul character. (This limit arises because the kernel has to copy the passed path to kernel space before doing any work on it, and allocating more than a single page is costlier.)
  - e12e 8y ago
    Just tried, there's definitely a 255 sigle file/folder name limit with zfs - but I had no problem making very deep folder trees. However 'touch' wouldn't accept a path that was too long - 4kb looks about right. Bash cd wouldn't let mee leap all the way down in one go - but if I got down there (while ls a * ;do cd a * ;done) - I could create a file (ed: view from "top"): find -iname ok|wc 1 1 12805 I guess I've got a hard time imagining 4kb path and 255 byte path segments being a serious issue - but maybe my imagination is the problem?
- et2o 8y ago
  https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits https://en.wikipedia.org/wiki/Comparison_of_file_systems#Lim... For other's reference
pfranz 8y ago
What were you migrating from? Most common filesystems have 255 byte limit for filenames. It looks like most of them don't have a explicit limit on path length, but Linux has a limit of 4096. Any idea how Google Drive is a solution since many of these limits seem to exist on the OS side of things? https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits https://en.wikipedia.org/wiki/Comparison_of_file_systems#Lim...

by chmars 8y ago

e12e 8y ago

I don't think there's a path limit in ext4, but looks like individual file/folders are limited to 255 bytes, while hfs+ (on Linux) is 255 utf-16 characters, which is potentially longer. The 255 bytes limit seems common (xfs, zfs)-while reiser allows 4kb or so. What kind of naming scheme do you have that you had a problem? Long, multibyte file names?

cesarb 8y ago
IIRC, there's a limit that the whole path passed to system calls has to fit into a single page. For most architectures, that's 4096 bytes including the terminating nul character. (This limit arises because the kernel has to copy the passed path to kernel space before doing any work on it, and allocating more than a single page is costlier.)
- e12e 8y ago
  Just tried, there's definitely a 255 sigle file/folder name limit with zfs - but I had no problem making very deep folder trees. However 'touch' wouldn't accept a path that was too long - 4kb looks about right. Bash cd wouldn't let mee leap all the way down in one go - but if I got down there (while ls a * ;do cd a * ;done) - I could create a file (ed: view from "top"): find -iname ok|wc 1 1 12805 I guess I've got a hard time imagining 4kb path and 255 byte path segments being a serious issue - but maybe my imagination is the problem?
et2o 8y ago
https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits https://en.wikipedia.org/wiki/Comparison_of_file_systems#Lim... For other's reference

pfranz 8y ago

What were you migrating from? Most common filesystems have 255 byte limit for filenames. It looks like most of them don't have a explicit limit on path length, but Linux has a limit of 4096. Any idea how Google Drive is a solution since many of these limits seem to exist on the OS side of things? https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits https://en.wikipedia.org/wiki/Comparison_of_file_systems#Lim...