4 ms·

Not really buying your thesis here: Attempts to retrofit safely to C have been around a lot longer than Cheri; fil-c is just the latest and there's no obvious

by ajb 2y ago

Not really buying your thesis here: Attempts to retrofit safely to C have been around a lot longer than Cheri; fil-c is just the latest and there's no obvious reason why it should be more successful. The speed comparison with a laptop is just disingenuous. Is a device with Cheri integrated slower than one of the same class without?

pizlonator 2y ago
> Not really buying your thesis here: Attempts to retrofit safely to C have been around a lot longer than Cheri; fil-c is just the latest and there's no obvious reason why it should be more successful. Here's the difference with Fil-C: It's totally memory safe and fanatically compatible with C/C++, no B.S. The other approaches tend to either create a different language (not compatible), or tend to create a superset of C/C++ with a safe subset (not totally memory safe, only compatible in the sense that unsafe code continues to be unsafe), or just don't fully square the circle (for example SoftBound could have gotten to full compatibility and safety, but the implementation just didn't). > The speed comparison with a laptop is just disingenuous. Is a device with Cheri integrated slower than one of the same class without? Not disingenuous at all. The issue is that: - high volume silicon tends to outperform low volume silicon. Fil-C runs on x86_64 (and could run on ARM64 if I had the resources to regularly test on it). So, Fil-C runs on the high volume stuff that gets all of the best optimizations. - silicon optimization is best done incrementally on top of an already fast chip, where the software being optimized for already runs on that chip. Then it's a matter of collecting traces on that software and tuning, rather than having to think through how to optimize a fast chip from first principles. CHERI means a new register file and new instructions so it doesn't lend itself well to the incremental optimization. So, I suspect Fil-C will always be faster than CHERI. This is especially true if you consider that there are lots of possible optimizations to Fil-C that I just haven't had a chance to land yet.
- lmm 2y ago
  > Here's the difference with Fil-C: It's totally memory safe and fanatically compatible with C/C++, no B.S. The other approaches tend to either create a different language (not compatible), or tend to create a superset of C/C++ with a safe subset (not totally memory safe, only compatible in the sense that unsafe code continues to be unsafe), or just don't fully square the circle (for example SoftBound could have gotten to full compatibility and safety, but the implementation just didn't). That's exactly the kind of thing that the boosters of all those previous efforts said. But somehow it never quite worked out.
  - jandrewrogers 2y ago
    Also, it always had a material performance impact. People write C++, and to a lesser extent C, because they really, really care about performance. If they didn’t care about performance there are easier languages to use.
    - pizlonator 2y ago
      Talking about performance impact is missing the bigger picture of how languages become performant. "Really really care about performance" describes some C/C++ programmers, but definitely not all of them. Finally, Fil-C is already faster than a lot of other safe languages (definitely faster than TypeScript, yet lots of stuff ships in TypeScript). Language implementations get faster over time and young ones tend to be slow. The Fil-C implementation is young. So were all of the previous attempts at memory-safe C - usually an implementation that had years of at most a few person years of investment (because it was done in an academic setting). Young implementations tend to be slow because the optimization investment hasn't happened in anger. So, "past academic attempts were slow" is not a great reason to avoid investigating memory safe C. Performance focus is not the reason why all of the world's C/C++ code gets written. Maybe that's even a minority reason. Lots of stuff uses C/C++ because of reasons like: - It started out in C/C++ so it continues to be in C/C++. So many huge projects are in this boat. - You're critically relying on a library whose only bindings are in C/C++, or the C/C++ bindings are the most mature, or the most easy to use. - You're doing low-level systems stuff, and having pointers that you can pass to syscalls is a core part of your logic. - You want to play nice with the dynamic linking situation on the OS you're targeting. (C/C++ get dynamic linking right in a way other languages don't.) I'd guess less than half of the C/C++ code that's being written today is being written because the programmer was thinking "oh man, this'll be too slow in any other language". Finally, Fil-C is already faster than a lot of memory safe languages. It's just not as fast as Yolo-C, but I don't think you can safely bet that this will be true forever.
  - pizlonator 2y ago
    > That's exactly the kind of thing that the boosters of all those previous efforts said. But somehow it never quite worked out. No, they really didn't. Let's review some of the big cases. - SafeC: not based on a mainstream C compiler, so can't handle gcc/clang extensions (Fil-C can). Had no story for threads or shared memory (Fil-C does). Hence, not really compatible. - CCured: incompatible (cannot compile C code with it without making changes, or running their tool that tries to automate the changes - but even then, common C idioms like unions don't quite work). Didn't use a major C compiler, - SoftBound: not totally memory safe (no attempt to provide safety for linking or function calls). But at least it's highly compatible. I can list more examples. Fil-C is the first to get both compatibility and safety right.
- woodruffw 2y ago
  I've been really impressed with what you're doing with Fil-C, but: > Here's the difference with Fil-C: It's totally memory safe and fanatically compatible with C/C++, no B.S. Is this true on both counts? If I'm reading your docs right, you're essentially adding hidden capabilities to pointers. This is a great technique that gives you almost perfect machine-level compatibility by default, but it comes with the standard caveats: 1. Your type safety/confusion guards are essentially tied to pointer "color," and colors are finite. In other words, in a sufficiently large program, an attacker can still perform type confusion by finding types with overlapping colors. Not an issue in small codebases, but maybe in browser- or kernel-sized ones. 2. In terms of compatibility, I'm pretty sure this doesn't allow a handful of pretty common pointer-integer roundtrip operations, at least not without having the user/programmer reassign the capability to the pointer that's been created out of "thin air." You could argue correctly that this is a bad thing that programmers shouldn't be doing, but it's well-defined and common enough IME. (You also cited my blog's writeup of `totally_safe_transmute` as an example of something that Fil-C would prevent, but I'm not sure I agree: the I/O effect in that example means that the program could thwart the runtime checks themselves. Of course, it's fair to say that /proc/self/mem is a stunt anyways.)
  - pizlonator 2y ago
    My type safety and confusion guards have nothing to do with colors of any kind. There are no finite colors to run out of. Fil-C totally allows pointer to integer round tripping in many cases, if the compiler can see it’s safe. I’m citing unsafe transmute as something that Fil-C doesn’t prevent. It’s not something that memory safety prevents.
    - woodruffw 2y ago
      > My type safety and confusion guards have nothing to do with colors of any kind. There are no finite colors to run out of. I'm having trouble seeing where the type confusion protection properties come from, then. I read through your earlier (I think?) design that involved isoheaps and it made sense in that context, but the newer stuff (in `gimso_semantics.md` and `invisicap.txt`) seems to mostly be around bounds checking instead. Apologies if I'm missing something obvious. > I’m citing unsafe transmute as something that Fil-C doesn’t prevent. It’s not something that memory safety prevents. I think the phrasing is confusing, because this is what the manifesto says: > No program accepted by the Fil-C compiler can possibly go on to escape out of the Fil-C type system. This to me suggests that Fil-C's type system detects I/O effects, but I take it that wasn't the intended suggestion.
- amluto 2y ago
  I read the Fil-C overview, and I was confused by one thing: how does Fil-C handle integer-to-pointer conversions? Rust has the new strict provenance API that is somewhat explicitly designed to avoid a need to materialize a pointer capability from just an integer, but C and C++ have no such thing. So if the code does: int deref(uintptr_t p) { return *(int*)p; } Does this fail unconditionally? Or is there some trick by which it can succeed if p is valid? And, if the latter is the case, then how is memory safety preserved? edit: I found zptrtable and this construct: https://github.com/pizlonator/pizlonated-quickjs/commit/258a4a291fd0f080614e5b345528478c31e51705#diff-45f1ae674139f993bf8a99c382c1ba4863272a6fec2f492d76d7ff1b2cfcfbe2R5181 https://github.com/pizlonator/pizlonated-quickjs/commit/258a... The latter seems to indicate that: (int*)((uintptr_t)p + 1) is at least a semi-reliable way to increment p by one. I guess this is a decent way to look like C and to keep a widely-used pattern functional. Rust’s with_addr seems like a more explicit and less magical way to accomplish the same thing. If Fil-C really takes off, would you want to add something like with_addr? Is allowing the pair of conversions on the same line of code something that can be fully specified and can be guaranteed to compile correctly such that it never accidentally produces a pointer with no capability?
  - pizlonator 2y ago
    Your deref function will fail, yeah. The pair of conversions is guaranreed to always produce a pointer with a capability. That’s how I implemented it and it’s low-tech enough that it could be specified.
    - amluto 2y ago
      How far can the pair of conversions be pushed? Will this work: (int*)(f((uintptr_t)p)) Does it matter if f is inline? Could someone implement Rust’s with_addr as: (int*)((uintptr_t)p, addr) FWIW, I kind of like zptrtable, and I think Fil-C sounds awesome. And I’m impressed that you were able to port large code bases with as few changes as it seems to have taken.
      - pizlonator 2y ago
musicale 2y ago
> Attempts to retrofit safely to C have been around a lot longer than Cheri; fil-c is just the latest and there's no obvious reason why it should be more successful. It is true that memory-safe C compilers have existed for decades and have seen minimal adoption. However, improvements to clang/llvm could yield wider impact and benefit than previous efforts, since they may be supported in a widely used C toolchain. -fbounds-safety is another change that may see more adoption if it makes it into mainline clang/llvm https://clang.llvm.org/docs/BoundsSafetyAdoptionGuide.html https://clang.llvm.org/docs/BoundsSafetyAdoptionGuide.html

by ajb 2y ago

pizlonator 2y ago

> Not really buying your thesis here: Attempts to retrofit safely to C have been around a lot longer than Cheri; fil-c is just the latest and there's no obvious reason why it should be more successful. Here's the difference with Fil-C: It's totally memory safe and fanatically compatible with C/C++, no B.S. The other approaches tend to either create a different language (not compatible), or tend to create a superset of C/C++ with a safe subset (not totally memory safe, only compatible in the sense that unsafe code continues to be unsafe), or just don't fully square the circle (for example SoftBound could have gotten to full compatibility and safety, but the implementation just didn't). > The speed comparison with a laptop is just disingenuous. Is a device with Cheri integrated slower than one of the same class without? Not disingenuous at all. The issue is that: - high volume silicon tends to outperform low volume silicon. Fil-C runs on x86_64 (and could run on ARM64 if I had the resources to regularly test on it). So, Fil-C runs on the high volume stuff that gets all of the best optimizations. - silicon optimization is best done incrementally on top of an already fast chip, where the software being optimized for already runs on that chip. Then it's a matter of collecting traces on that software and tuning, rather than having to think through how to optimize a fast chip from first principles. CHERI means a new register file and new instructions so it doesn't lend itself well to the incremental optimization. So, I suspect Fil-C will always be faster than CHERI. This is especially true if you consider that there are lots of possible optimizations to Fil-C that I just haven't had a chance to land yet.

lmm 2y ago
> Here's the difference with Fil-C: It's totally memory safe and fanatically compatible with C/C++, no B.S. The other approaches tend to either create a different language (not compatible), or tend to create a superset of C/C++ with a safe subset (not totally memory safe, only compatible in the sense that unsafe code continues to be unsafe), or just don't fully square the circle (for example SoftBound could have gotten to full compatibility and safety, but the implementation just didn't). That's exactly the kind of thing that the boosters of all those previous efforts said. But somehow it never quite worked out.
- jandrewrogers 2y ago
  Also, it always had a material performance impact. People write C++, and to a lesser extent C, because they really, really care about performance. If they didn’t care about performance there are easier languages to use.
  - pizlonator 2y ago
    Talking about performance impact is missing the bigger picture of how languages become performant. "Really really care about performance" describes some C/C++ programmers, but definitely not all of them. Finally, Fil-C is already faster than a lot of other safe languages (definitely faster than TypeScript, yet lots of stuff ships in TypeScript). Language implementations get faster over time and young ones tend to be slow. The Fil-C implementation is young. So were all of the previous attempts at memory-safe C - usually an implementation that had years of at most a few person years of investment (because it was done in an academic setting). Young implementations tend to be slow because the optimization investment hasn't happened in anger. So, "past academic attempts were slow" is not a great reason to avoid investigating memory safe C. Performance focus is not the reason why all of the world's C/C++ code gets written. Maybe that's even a minority reason. Lots of stuff uses C/C++ because of reasons like: - It started out in C/C++ so it continues to be in C/C++. So many huge projects are in this boat. - You're critically relying on a library whose only bindings are in C/C++, or the C/C++ bindings are the most mature, or the most easy to use. - You're doing low-level systems stuff, and having pointers that you can pass to syscalls is a core part of your logic. - You want to play nice with the dynamic linking situation on the OS you're targeting. (C/C++ get dynamic linking right in a way other languages don't.) I'd guess less than half of the C/C++ code that's being written today is being written because the programmer was thinking "oh man, this'll be too slow in any other language". Finally, Fil-C is already faster than a lot of memory safe languages. It's just not as fast as Yolo-C, but I don't think you can safely bet that this will be true forever.
- pizlonator 2y ago
  > That's exactly the kind of thing that the boosters of all those previous efforts said. But somehow it never quite worked out. No, they really didn't. Let's review some of the big cases. - SafeC: not based on a mainstream C compiler, so can't handle gcc/clang extensions (Fil-C can). Had no story for threads or shared memory (Fil-C does). Hence, not really compatible. - CCured: incompatible (cannot compile C code with it without making changes, or running their tool that tries to automate the changes - but even then, common C idioms like unions don't quite work). Didn't use a major C compiler, - SoftBound: not totally memory safe (no attempt to provide safety for linking or function calls). But at least it's highly compatible. I can list more examples. Fil-C is the first to get both compatibility and safety right.
woodruffw 2y ago
I've been really impressed with what you're doing with Fil-C, but: > Here's the difference with Fil-C: It's totally memory safe and fanatically compatible with C/C++, no B.S. Is this true on both counts? If I'm reading your docs right, you're essentially adding hidden capabilities to pointers. This is a great technique that gives you almost perfect machine-level compatibility by default, but it comes with the standard caveats: 1. Your type safety/confusion guards are essentially tied to pointer "color," and colors are finite. In other words, in a sufficiently large program, an attacker can still perform type confusion by finding types with overlapping colors. Not an issue in small codebases, but maybe in browser- or kernel-sized ones. 2. In terms of compatibility, I'm pretty sure this doesn't allow a handful of pretty common pointer-integer roundtrip operations, at least not without having the user/programmer reassign the capability to the pointer that's been created out of "thin air." You could argue correctly that this is a bad thing that programmers shouldn't be doing, but it's well-defined and common enough IME. (You also cited my blog's writeup of `totally_safe_transmute` as an example of something that Fil-C would prevent, but I'm not sure I agree: the I/O effect in that example means that the program could thwart the runtime checks themselves. Of course, it's fair to say that /proc/self/mem is a stunt anyways.)
- pizlonator 2y ago
  My type safety and confusion guards have nothing to do with colors of any kind. There are no finite colors to run out of. Fil-C totally allows pointer to integer round tripping in many cases, if the compiler can see it’s safe. I’m citing unsafe transmute as something that Fil-C doesn’t prevent. It’s not something that memory safety prevents.
  - woodruffw 2y ago
    > My type safety and confusion guards have nothing to do with colors of any kind. There are no finite colors to run out of. I'm having trouble seeing where the type confusion protection properties come from, then. I read through your earlier (I think?) design that involved isoheaps and it made sense in that context, but the newer stuff (in `gimso_semantics.md` and `invisicap.txt`) seems to mostly be around bounds checking instead. Apologies if I'm missing something obvious. > I’m citing unsafe transmute as something that Fil-C doesn’t prevent. It’s not something that memory safety prevents. I think the phrasing is confusing, because this is what the manifesto says: > No program accepted by the Fil-C compiler can possibly go on to escape out of the Fil-C type system. This to me suggests that Fil-C's type system detects I/O effects, but I take it that wasn't the intended suggestion.
amluto 2y ago
I read the Fil-C overview, and I was confused by one thing: how does Fil-C handle integer-to-pointer conversions? Rust has the new strict provenance API that is somewhat explicitly designed to avoid a need to materialize a pointer capability from just an integer, but C and C++ have no such thing. So if the code does: int deref(uintptr_t p) { return *(int*)p; } Does this fail unconditionally? Or is there some trick by which it can succeed if p is valid? And, if the latter is the case, then how is memory safety preserved? edit: I found zptrtable and this construct: https://github.com/pizlonator/pizlonated-quickjs/commit/258a4a291fd0f080614e5b345528478c31e51705#diff-45f1ae674139f993bf8a99c382c1ba4863272a6fec2f492d76d7ff1b2cfcfbe2R5181 https://github.com/pizlonator/pizlonated-quickjs/commit/258a... The latter seems to indicate that: (int*)((uintptr_t)p + 1) is at least a semi-reliable way to increment p by one. I guess this is a decent way to look like C and to keep a widely-used pattern functional. Rust’s with_addr seems like a more explicit and less magical way to accomplish the same thing. If Fil-C really takes off, would you want to add something like with_addr? Is allowing the pair of conversions on the same line of code something that can be fully specified and can be guaranteed to compile correctly such that it never accidentally produces a pointer with no capability?
- pizlonator 2y ago
  Your deref function will fail, yeah. The pair of conversions is guaranreed to always produce a pointer with a capability. That’s how I implemented it and it’s low-tech enough that it could be specified.
  - amluto 2y ago
    How far can the pair of conversions be pushed? Will this work: (int*)(f((uintptr_t)p)) Does it matter if f is inline? Could someone implement Rust’s with_addr as: (int*)((uintptr_t)p, addr) FWIW, I kind of like zptrtable, and I think Fil-C sounds awesome. And I’m impressed that you were able to port large code bases with as few changes as it seems to have taken.
    - pizlonator 2y ago

musicale 2y ago

> Attempts to retrofit safely to C have been around a lot longer than Cheri; fil-c is just the latest and there's no obvious reason why it should be more successful. It is true that memory-safe C compilers have existed for decades and have seen minimal adoption. However, improvements to clang/llvm could yield wider impact and benefit than previous efforts, since they may be supported in a widely used C toolchain. -fbounds-safety is another change that may see more adoption if it makes it into mainline clang/llvm https://clang.llvm.org/docs/BoundsSafetyAdoptionGuide.html https://clang.llvm.org/docs/BoundsSafetyAdoptionGuide.html