6 ms·

Two big problems in this document: - it conflates data race protection with memory safety, and it does so inconsistently. Java and C# are mentioned as MSLs and

Two big problems in this document: - it conflates data race protection with memory safety, and it does so inconsistently. Java and C# are mentioned as MSLs and yet they totally let you race. More fundamentally, data races aren’t the thing that attackers exploit except when those data races do lead to actual memory corruption (like use after free, double free, out of bounds, access to allocator metadata etc). So it’s more precise to not mention data races freedom as a requirement for memory safety, both because otherwise languages like Java and C# don’t meet the definition despite being included in the list and because data races in the presence of memory safety are not a big deal from a security standpoint. - The document fails to mention to mention Fil-C. It would be understandable if it was mentioned with caveats (“new project”, “performance blah blah”) but not mentioning it at all is silly.

jart 1y ago
[flagged]
- jdright 1y ago
  jart putting Carmack and Musk at the same level is a bit sad and revealing, no wonder the downvotes.
  - jart 1y ago
    https://x.com/ID_AA_Carmack/status/1935353905149341968 https://x.com/ID_AA_Carmack/status/1935353905149341968
- Ygg2 1y ago
  > Memory safety is like the global warming of the software industry. So it's an insidious long term issue that challenges our systems which reward short term thinking, and will slowly crush us, if we don't do anything about it? I fully agree.
pornel 1y ago
They're not going to mention a single-person experimental project that has 900 stars on GitHub. This is meant to be a practical strategy that can be implemented nation-wide, without turning into another https://xkcd.com/2347 https://xkcd.com/2347
- pizlonator 1y ago
  > They're not going to mention a single-person experimental project that has 900 stars on GitHub. Seems like a bad way to pick technology. They do mention things like TRACTOR. Fil-C is far ahead of any project under the TRACTOR umbrella. > This is meant to be a practical strategy that can be implemented nation-wide, without turning into another https://xkcd.com/2347 https://xkcd.com/2347 The solution to that is funding the thing that is essential, rather than complaining that an essential thing is unfunded. DOD could do that
  - safercplusplus 1y ago
    Preach it brother! :) Hmm, I take it that the situation is that there are a number of vendors/providers/distros/repos who could be distributing your memory-safe builds, but are currently still distributing unsafe builds? I wonder if an organization like the Tor project [1] would be more motivated to "officially" distribute a Fil-C build, being that security is the whole point of their product. (I'm talking just their "onion router" [2], not (necessarily) the whole browser.) I could imagine that once some organizations start officially shipping Fil-C builds, adoption might accelerate. Also, have you talked to the Ladybird browser people? They seemed to be taking an interested in Fil-C. [1] https://www.torproject.org/ https://www.torproject.org/ [2] https://gitlab.torproject.org/tpo/core/tor https://gitlab.torproject.org/tpo/core/tor
    - pornel 1y ago
      Tor wants to move to Rust, and they aren't happy with their C codebase. They want to expand use of multi-threading, and C has been too fragile for that. https://blog.torproject.org/announcing-arti/ https://blog.torproject.org/announcing-arti/
burakemir 1y ago
A definition of memory safety without data race freedom may be more precise but arguably less complete. It is correct that data races in a garbage collected language are difficult to turn into exploits. The problem is that data races in C and C++ do in fact get combined with other memory safety bugs into exploits. A definition from first principles is still missing, but imagine it takes the form of "all memory access is free from UB". Then whether the pointer is in-bounds, or whether no thread is concurrently mutating the location seem to be quite similar constraints. Rust does give ways to control concurrency, eg via expressing exclusive access through &mut reference. So there is also precedent that the same mechanisms can be used to ensure validity of reference (not dangling) as well as absence of concurrent access.
- pizlonator 1y ago
  > The problem is that data races in C and C++ do in fact get combined with other memory safety bugs into exploits. Because C and C++ are not memory safe. > A definition from first principles is still missing, but imagine it takes the form of "all memory access is free from UB". Then whether the pointer is in-bounds, or whether no thread is concurrently mutating the location seem to be quite similar constraints. I think it's useful to work backwards from the languages that security folks say are "memory safe", since what they're really saying is, "I cannot use the attacks I'm familiar with against programs written in these languages". Based on that, saying "no UB" isn't enough, and only looking at memory accesses isn't enough. WebAssembly has no UB, but pointers defined to just be integers (i.e. the UB-free structured assembly semantics of a C programmer's dreams). So, attackers can do OOB and UAF data attacks within the wasm memory. The only thing attackers cannot do is control the instruction pointer or escape the wasm memory (unless the wasm embedder has a weak sandbox policy, in which case they can do both). Overall, I think that memory-safety-in-the-sense-of-wasm isn't really memory safety at all. It's too exploitable. To be memory safe like the "MSLs" that security folks speak of, you also need to consider stuff like function calls. Depending on the language, you might have to look at other stuff, too. I think that what security folks consider "memory safe" is the combination of these things: 1) Absence of UB. Every outcome is well defined. 2) Pointers (or whatever pointer-like construct your language has) can only be used to access whatever allocation the originated from (i.e. pointers carry capabilities). And it's important that these get "strongly" combined; i.e. there is no operation in the language that could be used to break a pointer's capability enforcement. Java and Fil-C both have a strong combination of (1) and (2). But, long story short, it's true that a definition of memory safety from first principles is missing in the sense that the field hasn't settled on a consensus for what the definition should be. It's controversial because you could argue that under my definition, Rust isn't memory safe (you can get to UB in Rust). And, you could argue that wasm meets (2) because "the allocation" is just "all of memory". I'm not even sure I like my own definition. At some point you have to say more words about what an allocation is.
AnthonyMouse 1y ago
> More fundamentally, data races aren’t the thing that attackers exploit except when those data races do lead to actual memory corruption (like use after free, double free, out of bounds, access to allocator metadata etc). This is absolutely not true. One of the classic data races is when you do a set of operations like this non-atomically: new_total = account.balance; new_total -= purchase_price; account.balance = new_total; Which is a huge security vulnerability because it lets people double spend. Alice buys something for $1000 and something for $1 and instead of debiting her account by $1001 it debits it by $1 because the write for the second transaction clobbers the balance reduction from the first one. Another common one is symbolic links. You check the target of a symbolic link and then access it, but between the check and the access the link changed and now you're leaking secrets or overwriting privileged data. Data races are serious vulnerabilities completely independent of memory safety.
- tialaramex 1y ago
  That filesystem example is a TOCTOU race, not a data race, and so it can happen in Rust or similar languages just the same. Although both TOCTOU races and data races are types of race condition, they are not the same thing. Many race conditions are an ordinary part of our lived experience - if you've ever thought "Oh, I need to buy more milk" and then went to a store but meanwhile your house mate, partner, colleague at work, or whatever were also out buying more milk, well, when you get back with milk there's too much milk, oops, that's a race condition, specifically a TOCTOU race - we checked the milk, then we purchased more milk, meanwhile someone else changed how much milk is there. Data races aren't like any real world experience. The way the machine actually works is too alien for us to get our heads around so we're provided with a grossly simplified "sequentially consistent" illusion when writing high level languages like C - in which things happen in some order. Data races are reality "bleeding through" if we don't follow the rules to preserve that illusion.
  - stirfish 1y ago
    >TOCTUO Time of check to time of use https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use I didn't know this, thank you
tialaramex 1y ago
> Java and C# are mentioned as MSLs and yet they totally let you race. In Java a data race means loss of sequential consistency. Humans generally don't understand programs which lack sequential consistency so a typical Java team probably can't debug the program, but the program still always has well defined behaviour - and chances are you don't want to debug the weird non-sequentially consistent behaviour anyway, you just want them to fix the data race. In C# data races are not too dangerous for trivial objects which are valid for all bit patterns. If you race an integer k, well, now k is smashed, don't think too hard about the value of k, it does have some value but it won't go well for you to try to reason about the value. For a complex object like a hash table, it's Undefined Behaviour. Meanwhile in C or C++ all data races are immediate UB, you lose, game over.
- tialaramex 1y ago
  Ugh, I guess my brain mis-fired when I wrote this originally. Sorry about that. The above C# comments (about trivial versus complex objects) actually refer to Go, a quite different garbage collected language. For C# the exact situation is not well documented, and so it's probably best not to rely on anything that you can't get a solid guarantee for, but in principle it's similar to Java.
- pizlonator 1y ago
  In Fil-C, data races have Java-like behavior. The reason why YOLO-C has the bad data race behavior is because (1) data races might lead to memory safety violations and (2) the compiler is permitted to play more fast and loose than strictly necessary. Fil-C fixes (1) by making the language memory safe. Fil-C fixes (2) by just having different policies in the compiler.
SkiFire13 1y ago
> because data races in the presence of memory safety are not a big deal from a security standpoint. Note though that data races can make otherwise memory-safe programs not actually memory safe. See for example Go
- pizlonator 1y ago
  What’s the problem with Go?
  - aw1621107 1y ago
    IIRC slices aren't updated atomically in Go so you can get data races on their components, resulting in potential UB. For what it's worth, from what I've read on here (e.g., [0]) this has yet to be exploited in a non-demonstration setting, but who knows if/when the first such exploit will appear. [0]: https://news.ycombinator.com/item?id=42043939 https://news.ycombinator.com/item?id=42043939
    - pizlonator 1y ago
      I think that just means that Go isn't completely memory safe. Or, it means that "Go is only memory safe provided you have no data races". My point is that it's weird to say that there is a notion of memory safety that is separate from the memory safety you get if you also have a story for data races. It leads to exactly the confusion in the OP: it's not clear if they're saying that memory safety subsumes data race freedom in the sense that you're memory-safe even in the presence of races (like Java or Fil-C), or that it means that memory safety subsumes data race freedom in the sense that Rust's type system handles both memory safety and data races using the same basic technique.
      - aw1621107 1y ago
        Right, I get what you're saying and think you have a point. Just wanted to expand on the (perceived?) issue with Go and data races.

jart 1y ago

[flagged]

jdright 1y ago
jart putting Carmack and Musk at the same level is a bit sad and revealing, no wonder the downvotes.
- jart 1y ago
  https://x.com/ID_AA_Carmack/status/1935353905149341968 https://x.com/ID_AA_Carmack/status/1935353905149341968
Ygg2 1y ago
> Memory safety is like the global warming of the software industry. So it's an insidious long term issue that challenges our systems which reward short term thinking, and will slowly crush us, if we don't do anything about it? I fully agree.

pornel 1y ago

They're not going to mention a single-person experimental project that has 900 stars on GitHub. This is meant to be a practical strategy that can be implemented nation-wide, without turning into another https://xkcd.com/2347 https://xkcd.com/2347

pizlonator 1y ago
> They're not going to mention a single-person experimental project that has 900 stars on GitHub. Seems like a bad way to pick technology. They do mention things like TRACTOR. Fil-C is far ahead of any project under the TRACTOR umbrella. > This is meant to be a practical strategy that can be implemented nation-wide, without turning into another https://xkcd.com/2347 https://xkcd.com/2347 The solution to that is funding the thing that is essential, rather than complaining that an essential thing is unfunded. DOD could do that
- safercplusplus 1y ago
  Preach it brother! :) Hmm, I take it that the situation is that there are a number of vendors/providers/distros/repos who could be distributing your memory-safe builds, but are currently still distributing unsafe builds? I wonder if an organization like the Tor project [1] would be more motivated to "officially" distribute a Fil-C build, being that security is the whole point of their product. (I'm talking just their "onion router" [2], not (necessarily) the whole browser.) I could imagine that once some organizations start officially shipping Fil-C builds, adoption might accelerate. Also, have you talked to the Ladybird browser people? They seemed to be taking an interested in Fil-C. [1] https://www.torproject.org/ https://www.torproject.org/ [2] https://gitlab.torproject.org/tpo/core/tor https://gitlab.torproject.org/tpo/core/tor
  - pornel 1y ago
    Tor wants to move to Rust, and they aren't happy with their C codebase. They want to expand use of multi-threading, and C has been too fragile for that. https://blog.torproject.org/announcing-arti/ https://blog.torproject.org/announcing-arti/

burakemir 1y ago

A definition of memory safety without data race freedom may be more precise but arguably less complete. It is correct that data races in a garbage collected language are difficult to turn into exploits. The problem is that data races in C and C++ do in fact get combined with other memory safety bugs into exploits. A definition from first principles is still missing, but imagine it takes the form of "all memory access is free from UB". Then whether the pointer is in-bounds, or whether no thread is concurrently mutating the location seem to be quite similar constraints. Rust does give ways to control concurrency, eg via expressing exclusive access through &mut reference. So there is also precedent that the same mechanisms can be used to ensure validity of reference (not dangling) as well as absence of concurrent access.

pizlonator 1y ago
> The problem is that data races in C and C++ do in fact get combined with other memory safety bugs into exploits. Because C and C++ are not memory safe. > A definition from first principles is still missing, but imagine it takes the form of "all memory access is free from UB". Then whether the pointer is in-bounds, or whether no thread is concurrently mutating the location seem to be quite similar constraints. I think it's useful to work backwards from the languages that security folks say are "memory safe", since what they're really saying is, "I cannot use the attacks I'm familiar with against programs written in these languages". Based on that, saying "no UB" isn't enough, and only looking at memory accesses isn't enough. WebAssembly has no UB, but pointers defined to just be integers (i.e. the UB-free structured assembly semantics of a C programmer's dreams). So, attackers can do OOB and UAF data attacks within the wasm memory. The only thing attackers cannot do is control the instruction pointer or escape the wasm memory (unless the wasm embedder has a weak sandbox policy, in which case they can do both). Overall, I think that memory-safety-in-the-sense-of-wasm isn't really memory safety at all. It's too exploitable. To be memory safe like the "MSLs" that security folks speak of, you also need to consider stuff like function calls. Depending on the language, you might have to look at other stuff, too. I think that what security folks consider "memory safe" is the combination of these things: 1) Absence of UB. Every outcome is well defined. 2) Pointers (or whatever pointer-like construct your language has) can only be used to access whatever allocation the originated from (i.e. pointers carry capabilities). And it's important that these get "strongly" combined; i.e. there is no operation in the language that could be used to break a pointer's capability enforcement. Java and Fil-C both have a strong combination of (1) and (2). But, long story short, it's true that a definition of memory safety from first principles is missing in the sense that the field hasn't settled on a consensus for what the definition should be. It's controversial because you could argue that under my definition, Rust isn't memory safe (you can get to UB in Rust). And, you could argue that wasm meets (2) because "the allocation" is just "all of memory". I'm not even sure I like my own definition. At some point you have to say more words about what an allocation is.

AnthonyMouse 1y ago

> More fundamentally, data races aren’t the thing that attackers exploit except when those data races do lead to actual memory corruption (like use after free, double free, out of bounds, access to allocator metadata etc). This is absolutely not true. One of the classic data races is when you do a set of operations like this non-atomically: new_total = account.balance; new_total -= purchase_price; account.balance = new_total; Which is a huge security vulnerability because it lets people double spend. Alice buys something for $1000 and something for $1 and instead of debiting her account by $1001 it debits it by $1 because the write for the second transaction clobbers the balance reduction from the first one. Another common one is symbolic links. You check the target of a symbolic link and then access it, but between the check and the access the link changed and now you're leaking secrets or overwriting privileged data. Data races are serious vulnerabilities completely independent of memory safety.

tialaramex 1y ago
That filesystem example is a TOCTOU race, not a data race, and so it can happen in Rust or similar languages just the same. Although both TOCTOU races and data races are types of race condition, they are not the same thing. Many race conditions are an ordinary part of our lived experience - if you've ever thought "Oh, I need to buy more milk" and then went to a store but meanwhile your house mate, partner, colleague at work, or whatever were also out buying more milk, well, when you get back with milk there's too much milk, oops, that's a race condition, specifically a TOCTOU race - we checked the milk, then we purchased more milk, meanwhile someone else changed how much milk is there. Data races aren't like any real world experience. The way the machine actually works is too alien for us to get our heads around so we're provided with a grossly simplified "sequentially consistent" illusion when writing high level languages like C - in which things happen in some order. Data races are reality "bleeding through" if we don't follow the rules to preserve that illusion.
- stirfish 1y ago
  >TOCTUO Time of check to time of use https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use I didn't know this, thank you

tialaramex 1y ago

> Java and C# are mentioned as MSLs and yet they totally let you race. In Java a data race means loss of sequential consistency. Humans generally don't understand programs which lack sequential consistency so a typical Java team probably can't debug the program, but the program still always has well defined behaviour - and chances are you don't want to debug the weird non-sequentially consistent behaviour anyway, you just want them to fix the data race. In C# data races are not too dangerous for trivial objects which are valid for all bit patterns. If you race an integer k, well, now k is smashed, don't think too hard about the value of k, it does have some value but it won't go well for you to try to reason about the value. For a complex object like a hash table, it's Undefined Behaviour. Meanwhile in C or C++ all data races are immediate UB, you lose, game over.

tialaramex 1y ago
Ugh, I guess my brain mis-fired when I wrote this originally. Sorry about that. The above C# comments (about trivial versus complex objects) actually refer to Go, a quite different garbage collected language. For C# the exact situation is not well documented, and so it's probably best not to rely on anything that you can't get a solid guarantee for, but in principle it's similar to Java.
pizlonator 1y ago
In Fil-C, data races have Java-like behavior. The reason why YOLO-C has the bad data race behavior is because (1) data races might lead to memory safety violations and (2) the compiler is permitted to play more fast and loose than strictly necessary. Fil-C fixes (1) by making the language memory safe. Fil-C fixes (2) by just having different policies in the compiler.

SkiFire13 1y ago

> because data races in the presence of memory safety are not a big deal from a security standpoint. Note though that data races can make otherwise memory-safe programs not actually memory safe. See for example Go

pizlonator 1y ago
What’s the problem with Go?
- aw1621107 1y ago
  IIRC slices aren't updated atomically in Go so you can get data races on their components, resulting in potential UB. For what it's worth, from what I've read on here (e.g., [0]) this has yet to be exploited in a non-demonstration setting, but who knows if/when the first such exploit will appear. [0]: https://news.ycombinator.com/item?id=42043939 https://news.ycombinator.com/item?id=42043939
  - pizlonator 1y ago
    I think that just means that Go isn't completely memory safe. Or, it means that "Go is only memory safe provided you have no data races". My point is that it's weird to say that there is a notion of memory safety that is separate from the memory safety you get if you also have a story for data races. It leads to exactly the confusion in the OP: it's not clear if they're saying that memory safety subsumes data race freedom in the sense that you're memory-safe even in the presence of races (like Java or Fil-C), or that it means that memory safety subsumes data race freedom in the sense that Rust's type system handles both memory safety and data races using the same basic technique.
    - aw1621107 1y ago
      Right, I get what you're saying and think you have a point. Just wanted to expand on the (perceived?) issue with Go and data races.