3 ms·

> I interpret this as "file systems need to have a flexible metadata mechanism instead of ad-hoc hacks". Pike isn't even talking about that, only the typical c

by klabb3 4y ago

> I interpret this as "file systems need to have a flexible metadata mechanism instead of ad-hoc hacks". Pike isn't even talking about that, only the typical config files in the home dir that clutter and affect performance of file name ops in the home dir. > (For those who object that dot files serve a purpose, I don't dispute that but counter that it's the files that serve the purpose, not the convention for their names. They could just as easily be in $HOME/cfg or $HOME/lib, which is what we did in Plan 9, which had no dot files. Lessons can be learned.) I have to agree. I can't think of any situation where hiding really helps. If the reason is clutter, the solution should be a dir.

BrainVirus 4y ago
>I can't think of any situation where hiding really helps. The irony of dot-files is that they became used for metadata, like .git directories, .htcasses files and so on. So, a solution that fakes metadata by abusing names is now used to indicate metadata at another level of the system. There is clearly something missing in file system design.
- klabb3 4y ago
  I agree, sometimes dot files are used for metadata. But say a typical code repo: lots of dirs (vendored deps, compiled libs, generated code) are in regular dirs but git uses a dot-dir. Is there a strong reason for this I'm not getting? You're not supposed to modify either of them, heck even listing can be a shit-show. I can see the use-case of "when zipping or sending a dir to a different machine, omit the metadata", but this doesn't hold true for code repos, and is sometimes even the opposite of what you want. It'd probably be better with dir name conventions like "cache", "secrets", "config" or similar.
  - BrainVirus 4y ago
    I think what you're pointing out is an example of a real problem. I'm not sure the problem has a name or is noticed by most people. But something is off. In some cases it might be that common conventions (like dotfiles) aren't really well-designed. But I think the more general issue is that the current set of abstractions (file/directory/symlink) are not sufficient to take good care of practical use cases we have for data these days, not matter how cleverly we use them. Should .git directory reside in the same directory as your source code? Maybe not. Fossil has an interesting compromise in that regard. Should compiled artefacts reside in the same directory as your source code? Almost certainly not. This seems like a bad idea from some hacky implementation decades ago. Should tools be able to distinguish between "real" files and metadata files? Almost certainly yes, except there are different types of "metadata" files, as you pointed out. I think naming conventions are simply not good enough to handle this properly. This is where FS metadata could come into play. Generally, I think if someone without prejudice examined the common problems we have with data these days and designed a "file" system from scratch, they would end up with something vastly different from our current files/folders paradigm.
    - klabb3 4y ago
      > Should .git directory reside in the same directory as your source code? Maybe not. Indeed, many dependency managers (both go mod and cargo iirc) don't vendor deps into your working dir. > I think naming conventions are simply not good enough to handle this properly. This is where FS metadata could come into play. Yeah, the ideal solution may be just that, but I'm not 100% convinced that dirname conventions and tooling can't solve the major hurdles. New file systems would take decades even if they were to take on.. And for VCS and other tools you need to account for many different file systems and can't rely on anything but largest-common-denominator features..

by klabb3 4y ago

BrainVirus 4y ago

>I can't think of any situation where hiding really helps. The irony of dot-files is that they became used for metadata, like .git directories, .htcasses files and so on. So, a solution that fakes metadata by abusing names is now used to indicate metadata at another level of the system. There is clearly something missing in file system design.

klabb3 4y ago
I agree, sometimes dot files are used for metadata. But say a typical code repo: lots of dirs (vendored deps, compiled libs, generated code) are in regular dirs but git uses a dot-dir. Is there a strong reason for this I'm not getting? You're not supposed to modify either of them, heck even listing can be a shit-show. I can see the use-case of "when zipping or sending a dir to a different machine, omit the metadata", but this doesn't hold true for code repos, and is sometimes even the opposite of what you want. It'd probably be better with dir name conventions like "cache", "secrets", "config" or similar.
- BrainVirus 4y ago
  I think what you're pointing out is an example of a real problem. I'm not sure the problem has a name or is noticed by most people. But something is off. In some cases it might be that common conventions (like dotfiles) aren't really well-designed. But I think the more general issue is that the current set of abstractions (file/directory/symlink) are not sufficient to take good care of practical use cases we have for data these days, not matter how cleverly we use them. Should .git directory reside in the same directory as your source code? Maybe not. Fossil has an interesting compromise in that regard. Should compiled artefacts reside in the same directory as your source code? Almost certainly not. This seems like a bad idea from some hacky implementation decades ago. Should tools be able to distinguish between "real" files and metadata files? Almost certainly yes, except there are different types of "metadata" files, as you pointed out. I think naming conventions are simply not good enough to handle this properly. This is where FS metadata could come into play. Generally, I think if someone without prejudice examined the common problems we have with data these days and designed a "file" system from scratch, they would end up with something vastly different from our current files/folders paradigm.
  - klabb3 4y ago
    > Should .git directory reside in the same directory as your source code? Maybe not. Indeed, many dependency managers (both go mod and cargo iirc) don't vendor deps into your working dir. > I think naming conventions are simply not good enough to handle this properly. This is where FS metadata could come into play. Yeah, the ideal solution may be just that, but I'm not 100% convinced that dirname conventions and tooling can't solve the major hurdles. New file systems would take decades even if they were to take on.. And for VCS and other tools you need to account for many different file systems and can't rely on anything but largest-common-denominator features..