3 ms·

C++'s filesystem package (part of the standard library) should take care of all that nonsense for you. It's whole point is so that you don't have to pay attent

by gumby 3y ago

C++'s filesystem package (part of the standard library) should take care of all that nonsense for you. It's whole point is so that you don't have to pay attention to format of the underlying filenames, directories, etc.

LegionMammal978 3y ago
I think some people might be wary of the std::filesystem APIs because the standard allows implementations to completely disregard TOCTOU issues internally, to the point of breaking memory safety [0]: > A file system race is the condition that occurs when multiple threads, processes, or computers interleave access and modification of the same object within a file system. Behavior is undefined if calls to functions provided by subclause [filesystems] introduce a file system race. It's not just implementation-defined behavior, but full UB! You're utterly at the mercy of your implementation to do something reasonable when it encounters a TOCTOU issue, or, for that matter, any kind of concurrent modification to a file or directory. And C++ has a long history of implementations being unreliable in their behavior when UB is encountered. [0] https://wg21.link/fs.race.behavior#1 https://wg21.link/fs.race.behavior#1
- gumby 3y ago
  The more limited C API (system calls) is also full of race opportunities. The windows file APIs are better, but not a lot better. Anyway, the problem described in the post (scanning the string twice when converting) is as race prone, or not, as the the filesystem API.
  - LegionMammal978 3y ago
    > The more limited C API (system calls) is also full of race opportunities. The windows file APIs are better, but not a lot better. Sure, there are opportunities for races, but POSIX and the Windows API limit the possible outcomes of concurrent modification, and they also provide tools (fixed file handles, etc.) for well-written programs to prevent TOCTOU bugs. Meanwhile, the std::filesystem API just throws its hands in the air and says that filesystem races are UB, period, making it unusable if the program isn't in complete control of the directory tree it operates on. > Anyway, the problem described in the post (scanning the string twice when converting) is as race prone, or not, as the the filesystem API. I don't see what you mean? The typical scenario here is, you receive some absolute or relative path from an external source, encoded in ASCII or UTF-8 or some other non-UTF-16 encoding, and you want to operate on the file or directory at that path. Your internal path string is never going to change, only the filesystem can change. So there aren't any races from scanning your internal string twice to re-encode it; races can only come from using the re-encoded path in multiple calls to the filesystem API.
kouteiheika 3y ago
> C++'s filesystem package (part of the standard library) should take care of all that nonsense for you. It's whole point is so that you don't have to pay attention to format of the underlying filenames, directories, etc. Unfortunately by default it doesn't, because the constructors which take `const char *`/`std::string` will default to the local codepage, instead of just using UTF-8. So you have to use `u8path` or C++20's `u8` string literals to get sane portable behavior. So yes, you technically don't have to pay attention to the format, but as it is typical of many features in C++ the default is wrong, and you have to manually make sure you don't write code that's subtly broken. (Or alternatively use a linter which would warn about this, but does such linter exist?)

by gumby 3y ago

LegionMammal978 3y ago

I think some people might be wary of the std::filesystem APIs because the standard allows implementations to completely disregard TOCTOU issues internally, to the point of breaking memory safety [0]: > A file system race is the condition that occurs when multiple threads, processes, or computers interleave access and modification of the same object within a file system. Behavior is undefined if calls to functions provided by subclause [filesystems] introduce a file system race. It's not just implementation-defined behavior, but full UB! You're utterly at the mercy of your implementation to do something reasonable when it encounters a TOCTOU issue, or, for that matter, any kind of concurrent modification to a file or directory. And C++ has a long history of implementations being unreliable in their behavior when UB is encountered. [0] https://wg21.link/fs.race.behavior#1 https://wg21.link/fs.race.behavior#1

gumby 3y ago
The more limited C API (system calls) is also full of race opportunities. The windows file APIs are better, but not a lot better. Anyway, the problem described in the post (scanning the string twice when converting) is as race prone, or not, as the the filesystem API.
- LegionMammal978 3y ago
  > The more limited C API (system calls) is also full of race opportunities. The windows file APIs are better, but not a lot better. Sure, there are opportunities for races, but POSIX and the Windows API limit the possible outcomes of concurrent modification, and they also provide tools (fixed file handles, etc.) for well-written programs to prevent TOCTOU bugs. Meanwhile, the std::filesystem API just throws its hands in the air and says that filesystem races are UB, period, making it unusable if the program isn't in complete control of the directory tree it operates on. > Anyway, the problem described in the post (scanning the string twice when converting) is as race prone, or not, as the the filesystem API. I don't see what you mean? The typical scenario here is, you receive some absolute or relative path from an external source, encoded in ASCII or UTF-8 or some other non-UTF-16 encoding, and you want to operate on the file or directory at that path. Your internal path string is never going to change, only the filesystem can change. So there aren't any races from scanning your internal string twice to re-encode it; races can only come from using the re-encoded path in multiple calls to the filesystem API.

kouteiheika 3y ago

> C++'s filesystem package (part of the standard library) should take care of all that nonsense for you. It's whole point is so that you don't have to pay attention to format of the underlying filenames, directories, etc. Unfortunately by default it doesn't, because the constructors which take `const char *`/`std::string` will default to the local codepage, instead of just using UTF-8. So you have to use `u8path` or C++20's `u8` string literals to get sane portable behavior. So yes, you technically don't have to pay attention to the format, but as it is typical of many features in C++ the default is wrong, and you have to manually make sure you don't write code that's subtly broken. (Or alternatively use a linter which would warn about this, but does such linter exist?)