4 ms·

This issue seems like it would be common across languages, or at least those languages that have filesystem APIs in the stdlib. My understanding is that it is f

This issue seems like it would be common across languages, or at least those languages that have filesystem APIs in the stdlib. My understanding is that it is fixed in Python 3.3, while in C++ concurrent file system access is considered UB and therefore this isn’t a bug. Do people know the state in other languages?

piyh 5y ago
UB?
- spoiler 5y ago
  It stands for "Undefined Behaviour" It's commonly used in C/C++ communities (elsewhere too, but not as often).
- tyleo 5y ago
  Undefined behavior: https://en.m.wikipedia.org/wiki/Undefined_behavior https://en.m.wikipedia.org/wiki/Undefined_behavior
- milliams 5y ago
  Undefined behaviour -- https://en.wikipedia.org/wiki/Undefined_behavior https://en.wikipedia.org/wiki/Undefined_behavior
  - tyleo 5y ago
    It appears the race was on for commenters to jump in and define undefined behavior. Your comment is similar to my own. I feel like this sort of thing can be utilized as some sort of dev honeypot.
    - remram 5y ago
      It's easy enough to delete your comment if you see older ones after you've replied. With the amount of discussion around distributed system design on here, it's pretty egregious that we can't get the simplest transactional behavior right.
      - tyleo 5y ago
        It’s not so simple. I get internet points for my comment, even if it is nearly the same as someone else. Furthermore, the “first” comment chronologically is not necessarily sorted first. Indeed, older comments are sorted below. It’s actually later comments which are more near the top when others read threads, thus they get more exposure and upvotes and stay up top. In any case, I see no harm in any of this. Bits are cheap.
        remram 5y ago
        As I said, we are not doing it right.
- bestouff 5y ago
  "Undefined Behavior", a.k.a. "don't do that". It's everything the language allows you to do but the spec (you didn't read) says you should not, otherwise anything can happen. For example you should not access an array out of bounds, nor dereference a null pointer; so the compiler is free to assume this will never happen, and if it does the generated code may do very unexpected things, like not a straight segfault but continue running with unintended behavior. There are many UB cases in C/C++ many programmers don't suspect (more subtle than dereferencing null), and this regularly introduces very hard-to-debug bugs and securities issues. Rust does (mostly) get rid of them so you can be sure that your code has no unexpected side-effects.
  - lionkor 5y ago
    > the spec (you didn't read) This, to me, reads like it implies that reading the spec shouldn't be necessary to use a language correctly. The issue with that is that the spec is the only document you need to read, understand and follow in order to write safe, cross-implementation-compatible code. When you work with low-level languages and refuse to read the spec carefully, you are to blame more than the language or the spec. > this regularly introduces very hard-to-debug bugs and securities issues Yeah, that's true. The reality is that a lot of developers come from a C-syntax language (like C#, C, Java, ...) and assume that it's all the same because it looks the same. Mostly they follow video tutorials on youtube and blog-y websites instead of standard-adhering documentation (like cppreference.com). I've seen C++ developers that, after building 10k+ line of code programs, never knew why to use `delete`, what RAII even is, and have not a clue about what a mutex does. These are fundamental education problems. Rust has the benefit of being different enough from common "first languages" that one is inclined to look at the documentation. I'm not the only one who is endlessly tired of being told that the design of C++, with its undefined behavior and "need to read the spec", is somehow at fault for developers being too stubborn, entitled or stupid to take 3 minutes to read and understand the documentation. When a Rust developer makes a security mistake, causes a crash or other unexpected behavior, the developer is likely at fault. With C++, it's always "oh c++ is just like that". It's a complete mystery to me how people cannot understand that sometimes a language with lots of power can have pitfalls that need to be avoided by the programmer.
howdydoo 5y ago
In C++, any concurrent filesystem access is undefined behavior (which seems pretty crazy to me) https://en.cppreference.com/w/cpp/filesystem https://en.cppreference.com/w/cpp/filesystem > The behavior is undefined if the calls to functions in this library introduce a file system race, that is, when multiple threads, processes, or computers interleave access and modification to the same object in a file system. --- Golang also seems vulnerable to the same issue https://github.com/golang/go/blob/d15481b8c7f5f73a8b987a0c1deea04659ed0bb0/src/os/removeall_at.go#L78 https://github.com/golang/go/blob/d15481b8c7f5f73a8b987a0c1d... Line 78 checks that the path isn't a symlink (time-of-check). Then line 97 calls openFdAt which on line 174 opens the path by name, without NOFOLLOW (time-of-use). I bet this is a pretty common vulnerability.

piyh 5y ago

UB?

spoiler 5y ago
It stands for "Undefined Behaviour" It's commonly used in C/C++ communities (elsewhere too, but not as often).
tyleo 5y ago
Undefined behavior: https://en.m.wikipedia.org/wiki/Undefined_behavior https://en.m.wikipedia.org/wiki/Undefined_behavior
milliams 5y ago
Undefined behaviour -- https://en.wikipedia.org/wiki/Undefined_behavior https://en.wikipedia.org/wiki/Undefined_behavior
- tyleo 5y ago
  It appears the race was on for commenters to jump in and define undefined behavior. Your comment is similar to my own. I feel like this sort of thing can be utilized as some sort of dev honeypot.
  - remram 5y ago
    It's easy enough to delete your comment if you see older ones after you've replied. With the amount of discussion around distributed system design on here, it's pretty egregious that we can't get the simplest transactional behavior right.
    - tyleo 5y ago
      It’s not so simple. I get internet points for my comment, even if it is nearly the same as someone else. Furthermore, the “first” comment chronologically is not necessarily sorted first. Indeed, older comments are sorted below. It’s actually later comments which are more near the top when others read threads, thus they get more exposure and upvotes and stay up top. In any case, I see no harm in any of this. Bits are cheap.
      - remram 5y ago
        As I said, we are not doing it right.
bestouff 5y ago
"Undefined Behavior", a.k.a. "don't do that". It's everything the language allows you to do but the spec (you didn't read) says you should not, otherwise anything can happen. For example you should not access an array out of bounds, nor dereference a null pointer; so the compiler is free to assume this will never happen, and if it does the generated code may do very unexpected things, like not a straight segfault but continue running with unintended behavior. There are many UB cases in C/C++ many programmers don't suspect (more subtle than dereferencing null), and this regularly introduces very hard-to-debug bugs and securities issues. Rust does (mostly) get rid of them so you can be sure that your code has no unexpected side-effects.
- lionkor 5y ago
  > the spec (you didn't read) This, to me, reads like it implies that reading the spec shouldn't be necessary to use a language correctly. The issue with that is that the spec is the only document you need to read, understand and follow in order to write safe, cross-implementation-compatible code. When you work with low-level languages and refuse to read the spec carefully, you are to blame more than the language or the spec. > this regularly introduces very hard-to-debug bugs and securities issues Yeah, that's true. The reality is that a lot of developers come from a C-syntax language (like C#, C, Java, ...) and assume that it's all the same because it looks the same. Mostly they follow video tutorials on youtube and blog-y websites instead of standard-adhering documentation (like cppreference.com). I've seen C++ developers that, after building 10k+ line of code programs, never knew why to use `delete`, what RAII even is, and have not a clue about what a mutex does. These are fundamental education problems. Rust has the benefit of being different enough from common "first languages" that one is inclined to look at the documentation. I'm not the only one who is endlessly tired of being told that the design of C++, with its undefined behavior and "need to read the spec", is somehow at fault for developers being too stubborn, entitled or stupid to take 3 minutes to read and understand the documentation. When a Rust developer makes a security mistake, causes a crash or other unexpected behavior, the developer is likely at fault. With C++, it's always "oh c++ is just like that". It's a complete mystery to me how people cannot understand that sometimes a language with lots of power can have pitfalls that need to be avoided by the programmer.

howdydoo 5y ago

In C++, any concurrent filesystem access is undefined behavior (which seems pretty crazy to me) https://en.cppreference.com/w/cpp/filesystem https://en.cppreference.com/w/cpp/filesystem > The behavior is undefined if the calls to functions in this library introduce a file system race, that is, when multiple threads, processes, or computers interleave access and modification to the same object in a file system. --- Golang also seems vulnerable to the same issue https://github.com/golang/go/blob/d15481b8c7f5f73a8b987a0c1deea04659ed0bb0/src/os/removeall_at.go#L78 https://github.com/golang/go/blob/d15481b8c7f5f73a8b987a0c1d... Line 78 checks that the path isn't a symlink (time-of-check). Then line 97 calls openFdAt which on line 174 opens the path by name, without NOFOLLOW (time-of-use). I bet this is a pretty common vulnerability.