8 ms·

Fil-C: A memory-safe C implementation

283 points by chmaynard 1y ago

kragen 1y ago
Either Fil-C or a different implementation of the same idea seems essential to me. A great deal of software has been written in C, and without some way of running it, we lose access to that intellectual heritage. But pervasive security vulnerabilities mean that the traditional "YOLO" approach to C compilation is a bad idea for software that has to handle untrusted input, such as Web browsing or email. Pizlo seems to have found an astonishingly cheap way to do the necessary pointer checking, which hopefully I will be able to understand after more study. (The part I'm still confused about is how InvisiCaps work with memcpy.) tialaramex points out that we shouldn't expect C programmers to be excited about Fil-C. The point tialaramex mentions is "DWIM", like, accessing random memory and executing in constant time, but I think most C programmers won't be willing to take a 4× performance hit. After all, if they wanted to be using a slow language, they wouldn't be writing their code in C. But I think that's the wrong place to look for interest: Fil-C's target audience is users of C programs, not authors of C programs. We want the benefits of security and continued usage of existing working codebases, without having to pay the cost to rewrite everything in Rust or TypeScript or whatever. And for many of us, much of the time, the performance hit may be acceptable.
- nielsbot 1y ago
  I like to share this every time there's a post about memory safe C: Apple has a memory-safer C compiler/variant they use to compile their boot loaders: https://saaramar.github.io/iBoot_firebloom/ https://saaramar.github.io/iBoot_firebloom/
  - astrange 1y ago
    A descendent of this is in clang as -fbounds-safety.
    - pjmlp 1y ago
      That is basically clang getting something like MSVC SAL, the more the merrier.
  - kragen 1y ago
    Yeah, fat pointers are definitely a viable approach, but a lot of the existing C code that is the main argument for Fil-C assumes it can put a pointer in a long. (Most of the C code that assumed you could put it in an int has been flushed out by now, but that was a big problem when the Alpha came out.) I'm guessing that the amount of existing C code in Apple's bootloader is minimal, maybe 1000 lines, not the billions of lines you can compile with Fil-C.
    - 1718627440 1y ago
      Why not assume it fits into an uintptr_t, which is the type for exactly this purpose?
    - matthewfcarlson 1y ago
      You’re off by a few orders of magnitude. I’ll grant you, what is the bootloader becomes a very complex question. Even if you scope it to just “what is the code physically etched into the chip as the mask ROM” (secureROM) you’re talking hundreds of thousands. If you’re talking about all the code that runs before the kernel starts executing you’re talking hundreds of millions.
      - kragen 1y ago
        No, I was only talking about the pre-existing C code that wasn't written for the bootloader, which therefore might have incompatibilities with fat pointers you had to hunt down and fix. Also I'm really skeptical about your "hundreds of millions" number, even if we're talking about all the code that runs before the kernel starts. How do you figure? The entire Linux kernel doesn't contain a hundred millions of lines of code, and that includes all the drivers for network cards, SCSI controllers, and multiport serial boards that nobody's made in 30 years, plus ports to Alpha, HP PA-RISC, Loongson, Motorola 68000, and another half-dozen architectures. All of that contains maybe 30 million lines. glibc is half a million. Firefox 140.4.0esr is 33 million. You're saying that the bootloader is six times the size of Firefox? Are you really suggesting that tens of gigabytes of source code are compiled into the bootloader? That would make the bootloader at least a few hundred megabytes of executable code, probably gigabytes, wouldn't it?
  - conradev 1y ago
    and the author of Fil-C worked on that!
    - kragen 1y ago
      Oh, somehow I missed that connection!
  - pizlonator 1y ago
    That was my idea and I wrote a good chunk of the compiler and runtime.
    - kragen 1y ago
      (For those who didn't make the connection, pizlonator also wrote Fil-C.)
    - frumplestlatz 1y ago
      [flagged]
      - pizlonator 1y ago
        Sounds like you weren't there
        frumplestlatz 1y ago
        [flagged]
        deleted 1y ago
        [deleted]
        jibal 1y ago
        That breaks HN rules.
        kragen 1y ago
        (But revealing your identity would not.)
      - actionfromafar 1y ago
        You don't want to - so you did it anyway?
    - geocar 1y ago
  - EPWN3D 1y ago
    You don't even need to reverse it. It's in the public clang, and I'm working on helping my team adopt it in some test cases. And it's not just the bounds-checking that's great -- it makes a bunch of C anti-patterns much harder, and it makes you think a lot harder about pointer ownership and usage. Really a great addition to the language, and it's source-compatible with empty macro-definitions (with two exceptions).
    - kragen 1y ago
      Interesting! How do you get started?
      - EPWN3D 1y ago
        -fbounds-safety is the flag. https://clang.llvm.org/docs/BoundsSafety.html https://clang.llvm.org/docs/BoundsSafety.html
    - pizlonator 1y ago
      > It's in the public clang I think you’re thinking of something else
      - debugnik 1y ago
        It or something similar has apparently been upstreamed to clang as -fbounds-safety. I don't know if they're the same, but the RFC on -fbounds-safety does give some credit to the author of Fil-C, who also took credit for firebloom on this thread as well.
        pizlonator 1y ago
        Not the same
        debugnik 1y ago
        Thanks for clarifying, I had found a past HN comment stating they are the same.
  - pjmlp 1y ago
    Not for much longer, replacing it with Swift is one of the reasons why Swift Embedded subset came to be. "Why Embedded Swift" https://youtu.be/LqxbsADqDI4?t=144 https://youtu.be/LqxbsADqDI4?t=144
- TuxSH 1y ago
  Also this is _de facto_ limited to userspace application for the mainstream OSes if my understanding is correct. Reading Fil-C website's "InvisiCaps by example" page, I see that "Laundering Integers As Pointers" is disallowed. This essentially disqualifies Fil-C for low-level work, which makes for a substantial part of C programs. (int2ptr for MMIO/pre-allocated memory is in theory UB, in practice just fine as long as you don't otherwise break aliasing rules (and lifetime rules in C++) - as the compiler will fail to track provenance at least once). But that isn't really what Fil-C is aimed at - the value is, as you implied, in hardening userspace applications.
  - kragen 1y ago
    Yes, I think that's reasonable. I imagine you wouldn't have to extend Fil-C very much to sneak some memory-mapped I/O addresses into your program, but maybe having the garbage collector pause the program in the middle of an interrupt handler would have other bad effects. Like, if you were generating a video signal, you'd surely get display glitches.
  - mbrock 1y ago
    Check out this document to see how the Fil-C ports of Python and Perl and so on work: https://github.com/mbrock/filnix/blob/main/ports/analysis.md https://github.com/mbrock/filnix/blob/main/ports/analysis.md This is still within the userspace application realm but it's good to know that Fil-C does have explicit capability-preserving operations (`zxorptr`, `zretagptr`, etc) to do e.g. pointer tagging, and special support for mapping pointers to integer table indices and back (`zptrtable`, etc).
  - pizlonator 1y ago
    It’s not so fundamental of a limitation. Fil-C already allows memory mapped I/O in the form of mmap. The only thing missing that is needed for kernel level MMIO is a way to forge a capability. I don’t allow that right now, but that’s mostly a policy decision. It also falls out from the fact that InvisiCaps optimize the lower by having it double as a pointer to the top of the capability. That’s also not fundamental; it’s an implementation choice. It’s true that InvisiCaps will always disallow int to ptr casts, in the sense that you get a pointer with no capability. You’d want MMIO code to have some intrinsic like `zunsafe_forge_ptr` that clearly calls out what’s happening and then you’d use that wherever you define your memory mapped registers.
- nickpsecurity 1y ago
  Softbound + CETS was one of the old attempts I liked: https://people.cs.rutgers.edu/~santosh.nagarakatte/softbound/ https://people.cs.rutgers.edu/~santosh.nagarakatte/softbound... CCured was another: https://people.eecs.berkeley.edu/~necula/Papers/ccured_popl02.pdf https://people.eecs.berkeley.edu/~necula/Papers/ccured_popl0...
  - kragen 1y ago
    Good to see you back! I hope you're doing okay; I know you ran into some real trouble a couple of years ago.
    - nickpsecurity 1y ago
      Up to my eyeballs in trouble. Working two jobs, too, where I can't practice tech skills. Fortunately, I have the peace of Jesus Christ in hopeless situations. He might change them at any time, too. I still try to squirrel away a little time to learn from and contribute to the tech community. Mostly here and on AI subs like r/mlscaling. The best stuff that I can't work on or even curate right now I'm saving in case time or funding open up in the future. Worst case, I can pass it onto bright minds with the skills to build it. Like before in security. You were doing neat things with satellites and bootstrapping research last I checked. What interesting jobs or tech are you digging into now?
      - kragen 1y ago
        Mostly I've just been struggling to keep my head above water, but I've been experimenting with submilliwatt autonomous personal computing, LuaJIT, and radically simplified windowing systems. Planning to visit the US next year for the first time in almost 20 years.
- Ygg2 1y ago
  > but I think most C programmers won't be willing to take a 4× performance hit. At a 4x performance hit, you might as well use C# or Go. > Fil-C's target audience is users of C programs, not authors of C programs. Sure, but then they don't get it for free. There is a perf penalty from GC. Plus you need all the original sources, right? > we lose access to that intellectual heritage. Declining usage of C is going to make you lose intellectual heritage[1]. A language no one can read or write is a dead language, regardless if you can translate it to English or not. [1] And that is outside Rust's or Zig's influence. It's an old language from when people thought you can trust the programmer. Which may well be the case if only people using it were Bell Labs engineers. It's got UB up the wazoo, no safety, and no sane package manager.
  - mbrock 1y ago
    > you might as well use C# or Go Except, uh, you can't use C# or Go to run a program written in C/C++. Oh, you mean we can solve all our problems by simply rewriting all legacy software? Right, I forgot!
    - einsteinx2 1y ago
      > Oh, you mean we can solve all our problems by simply rewriting all legacy software? Isn’t that Rust’s raison d'être? (Just kidding…mostly)
      - Ygg2 1y ago
        Pretty sure that's a programmer's raison d'être. Not invented here (NIH) syndrom predates Rust.
        einsteinx2 1y ago
        Fair point, though it does seem to be more common in the Rust community to the point that it’s become a bit of a meme. Related funny anecdote, I recently saw a Show HN post title that made sure to mention the thing was written in Rust, but forgot to mention what it actually did. Priorities lol. I’m sure it sounds like I’m a Rust hater or something, but I’m really not. I like a lot of the new Rust tools being created and use a few myself (ripgrep, fd, and bat immediately come to mind but I’m sure there are more I’m using). I just find the almost religious fervor of the Rust community amusing.
mbrock 1y ago
I'm working on packaging Fil-C for Nix, as well as integrating Fil-C as a toolchain in Nix so you can build any Nix package with Fil-C. https://github.com/mbrock/filnix https://github.com/mbrock/filnix It's working. It builds tmux, nethack, coreutils, Perl, Tcl, Lua, SQLite, and a bunch of other stuff. Binary cache on https://filc.cachix.org https://filc.cachix.org so you don't have to wait 40 minutes for the Clang fork to build. If you have Nix with flakes on a 64-bit Linux computer, you can run nix run github:mbrock/filnix#nethack right now!
- kragen 1y ago
  That's very exciting! Thank you!
- yjftsjthsd-h 1y ago
  That is super cool, and I will probably start running it on at least a test box shortly. How does python work? Of course I can just add filc.python to my system, but if I `python3 -m pip install whatever` will it just rebuild any C modules with the fil-c compiler?
  - mbrock 1y ago
    Right now Python simply does not work at all. I was trying last night just to get the compiler to build, but ran into some Fil-C panics at the last moment when it tries to "freeze" some modules? But it should work. My idea is to move towards defining Fil-C as a "platform", meaning it would have its own ABI value, so it would look like a target called "x86_64-unknown-linux-filc", and then the magic of Nixpkgs is that it can instantiate a full realm called "pkgsCross.filc" which automatically has every package, whose packages are built on your ordinary non-FilC platform as a cross compilation. When that works—I hope it works, I probably need help to get it working—you should be able to use all the Nixpkgs packaging helpers like pkgs.mkShell { packages = [ (pkgsCross.filc.python3.withPackages (pypkgs: with pypkgs; [ pandas requests ])) ]; } This is already how Nixpkgs can have a pkgsCross.musl that builds everything with Musl as a cross compilation. So it seems reasonable and possible. It should be of wide interest for Nix users, since it would suddenly allow memory safety for whole swaths of the C/C++ packages, it would let NixOS use memory safe builds for core services like OpenSSHd, and so on... I should probably try to gather some interest among people who are truly familiar with Nixpkgs cross compilation...
nextaccountic 1y ago
Somewhat related, safe C++ proposal is not being continued https://news.ycombinator.com/item?id=45234460 https://news.ycombinator.com/item?id=45234460
dmitrygr 1y ago
TLDR: 4x slowdown in the normal case the performance overhead of this approach for most programs makes them run about four times more slowly
- pizlonator 1y ago
  > TLDR: 4x slowdown in the normal case 4x slower isn't the normal case. 4x is at the upper end of the overheads you'll see.
  - geraldog 1y ago
    That's good to know! C is immensely powerful, portable and probably as fast as you can go without hand-coding in the architecture-specific assembly. Most of the world's information systems (our cyberstructure) rely directly or indirectly on C. And don't get me wrong, I'm a great enthusiast of the idea of sticking to memory-safe languages like Rust from now on. The hard truth is will live with legacy C code, period. Pizlo's heroic effort bridges the gap so to speak, it kind of sandboxes userspace C in a way that inherently adds memory safety to legacy code. There are only a few corner cases now that can't be bothered by any slow-down vis-a-vis unsafe C, and the great majority of code across every industry would benefit much more from the reduced surface of exposure.
  - thw_9a83c 1y ago
    > 4x is at the upper end of the overheads This is already quite impressive. Many automatic memory managed languages have more than 4x worst-case slowdown. E.g. Google-backed Golang is ~1.5× to ~4× slower than optimized C++. I suppose there are many ways to further speed-up Fil-C if more resources were given to the project.
    - hyghjiyhu 1y ago
      This seems likely. I would assume that the vast majority of memory access can be statically proven safe if you throw enough money at the problem, meaning the bookkeeping can be eliminated.
EverydayBalloon 1y ago
[dead]
braggerxyz 1y ago
Every time I read C and memory safety, I just think Golang. Especially for user space
- thomasmg 1y ago
  Go programs are not fully memory-safe if they use multiple treads, due to possible data races with fat pointers: https://news.ycombinator.com/item?id=44672003 https://news.ycombinator.com/item?id=44672003
  - pjmlp 1y ago
    Still much better than C will ever be, even if I disagree with some of its design decisions.
- pjmlp 1y ago
  Yes, think of something like Inferno, but Limbo now is AOT compiled instead of JITed. However there are also kernel like commercial projects in Go, and apparently the related TamaGo fork might eventually get upstreamed into the reference implementation. https://www.withsecure.com/en/solutions/innovative-security-hardware/usb-armory https://www.withsecure.com/en/solutions/innovative-security-...
ridiculous_fish 1y ago
Extraordinary project. I had several questions which I believe I have answered for myself (pizlonator please correct if wrong): 1. How do we prevent loading a bogus lower through misaligned store or load? Answer: Misaligned pointer load/stores are trapped; this is simply not allowed. 2. How are pointer stores through a pointer implemented (e.g. `*(char **)p = s`) - does the runtime have to check if *p is "flight" or "heap" to know where to store the lower? Answer: no. Flight (i.e. local) pointers whose address is taken are not literally implemented as two adjacent words; rather the call frame is allocated with the same object layout as a heap object. The flight pointer is its "intval" and its paired "lower" is at the same offset in the "aux" allocation (presumably also allocated as part of the frame?). 3. How are use-after-return errors prevented? Say I store a local pointer in a global variable and then return. Later, I call a new function which overwrites the original frame - can't I get a bogus `lower` this way? Answer: no. Call frames are allocated by the GC, not the usual C stack. The global reference will keep the call frame alive. That leads to the following program, which definitely should not work, and yet does. ~Amazing~ Unbelievable: #include <stdio.h> char *bottles[100]; __attribute__((noinline)) void beer(int count) { char buf[64]; sprintf(buf, "%d bottles of beer on the wall", count); bottles[count] = buf; } int main(void) { for (int i=0; i < 100; i++) beer(i); for (int i=99; i >= 0; i--) puts(bottles[i]); }
- jibal 1y ago
  Hmmm ... there's a danger here that people will test their programs compiled with Fil-C and think that they are safe to compile with a "normal" compiler. I would hope for an option to flag any undefined behavior.
Panzerschrek 1y ago
Great effort, but I find the whole idea somewhat flawed. If one needs speed, he can't use this C implementation, because it's several times slower. If speed isn't important, why not just using a memory safe language? And if both are important, why not using Rust? Recompiling existing software written in C using Fil-C isn't also a great idea, since some modifications are likely needed, at least for fixing bugs found with usage of Fil-C. And after these bugs are fixed, why continue using Fil-C?
- mrunix 1y ago
  Most software works flawlessly without modifications on Fil-C, the performance isn't *that* bad and there are applications where security is more important than performance (for example military applications)
- veltas 1y ago
  You're glossing over the "just rewrite everything in Rust" part.
- Filligree 1y ago
  Safety isn’t really optional. Up until now, we’ve had no choice but to run C and accept the bugs; now we’re allowed to also run it safely. Yes, this means that C is one of the slower languages around. That’s fine; computers are fast. If you want to write high-performance code, there are plenty of faster languages to do it with.
  - pjmlp 1y ago
    There were several options, but the industry decided to bet on UNIX and C as main inspiration going forwards from the 1980's. "A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law." -- C.A.R Hoare's "The 1980 ACM Turing Award Lecture" Guess what he means by "1980 language designers and users have not learned this lesson".
cjk 1y ago
I worked with the author of Fil-C at Apple while on the Safari team, and he's easily one of the brightest folks I've had the pleasure of knowing. Fil-C looks extremely cool.
gnabgib 1y ago
No discussion, but just on the front page last week (31 points) https://news.ycombinator.com/item?id=45655519 https://news.ycombinator.com/item?id=45655519 Previous discussion: 2025 Safepoints and Fil-C (87 points, 1 month ago, 44 comments) https://news.ycombinator.com/item?id=45258029 https://news.ycombinator.com/item?id=45258029 2025 Fil's Unbelievable Garbage Collector (603 points, 2 months ago, 281 comments) https://news.ycombinator.com/item?id=45133938 https://news.ycombinator.com/item?id=45133938 2024 The Fil-C Manifesto: Garbage In, Memory Safety Out (13 points, 17 comments) https://news.ycombinator.com/item?id=39449500 https://news.ycombinator.com/item?id=39449500
synergy20 1y ago
posted multiple times, x86 only last time I checked
- lambdaone 1y ago
  All the more reason to make it portable. I wonder if this can be implemented via LLVM?
  - kragen 1y ago
    It is implemented via LLVM.
- pizlonator 1y ago
  Yeah because I’m limiting my test matrix. There’s nothing about how Fil-C is designed that constrains it to x86_64. It doesn’t strongly rely on x86’s memory model. It doesn’t strongly rely on 64-bit. I’m focusing on one OS and arch until I have more contributors and so more bandwidth to track bugs across a wider set of platforms.
  - khamidou 1y ago
    Hey Filip – while we're talking about memory architecture, have you started looking at ARM's EMTE extension (e.g https://security.apple.com/blog/memory-integrity-enforcement/ https://security.apple.com/blog/memory-integrity-enforcement...)? Could it eventually replace invisicaps?
    - pizlonator 1y ago
      No. MTE is probabilistic. Fil-C is deterministic.
  - erichocean 1y ago
    If you're taking votes, I would vote for AArch64 next. :D
    - 1y ago

283 points by chmaynard 1y ago

Either Fil-C or a different implementation of the same idea seems essential to me. A great deal of software has been written in C, and without some way of running it, we lose access to that intellectual heritage. But pervasive security vulnerabilities mean that the traditional "YOLO" approach to C compilation is a bad idea for software that has to handle untrusted input, such as Web browsing or email. Pizlo seems to have found an astonishingly cheap way to do the necessary pointer checking, which hopefully I will be able to understand after more study. (The part I'm still confused about is how InvisiCaps work with memcpy.) tialaramex points out that we shouldn't expect C programmers to be excited about Fil-C. The point tialaramex mentions is "DWIM", like, accessing random memory and executing in constant time, but I think most C programmers won't be willing to take a 4× performance hit. After all, if they wanted to be using a slow language, they wouldn't be writing their code in C. But I think that's the wrong place to look for interest: Fil-C's target audience is users of C programs, not authors of C programs. We want the benefits of security and continued usage of existing working codebases, without having to pay the cost to rewrite everything in Rust or TypeScript or whatever. And for many of us, much of the time, the performance hit may be acceptable.

nielsbot 1y ago
I like to share this every time there's a post about memory safe C: Apple has a memory-safer C compiler/variant they use to compile their boot loaders: https://saaramar.github.io/iBoot_firebloom/ https://saaramar.github.io/iBoot_firebloom/
- astrange 1y ago
  A descendent of this is in clang as -fbounds-safety.
  - pjmlp 1y ago
    That is basically clang getting something like MSVC SAL, the more the merrier.
- kragen 1y ago
  Yeah, fat pointers are definitely a viable approach, but a lot of the existing C code that is the main argument for Fil-C assumes it can put a pointer in a long. (Most of the C code that assumed you could put it in an int has been flushed out by now, but that was a big problem when the Alpha came out.) I'm guessing that the amount of existing C code in Apple's bootloader is minimal, maybe 1000 lines, not the billions of lines you can compile with Fil-C.
  - 1718627440 1y ago
    Why not assume it fits into an uintptr_t, which is the type for exactly this purpose?
  - matthewfcarlson 1y ago
    You’re off by a few orders of magnitude. I’ll grant you, what is the bootloader becomes a very complex question. Even if you scope it to just “what is the code physically etched into the chip as the mask ROM” (secureROM) you’re talking hundreds of thousands. If you’re talking about all the code that runs before the kernel starts executing you’re talking hundreds of millions.
    - kragen 1y ago
      No, I was only talking about the pre-existing C code that wasn't written for the bootloader, which therefore might have incompatibilities with fat pointers you had to hunt down and fix. Also I'm really skeptical about your "hundreds of millions" number, even if we're talking about all the code that runs before the kernel starts. How do you figure? The entire Linux kernel doesn't contain a hundred millions of lines of code, and that includes all the drivers for network cards, SCSI controllers, and multiport serial boards that nobody's made in 30 years, plus ports to Alpha, HP PA-RISC, Loongson, Motorola 68000, and another half-dozen architectures. All of that contains maybe 30 million lines. glibc is half a million. Firefox 140.4.0esr is 33 million. You're saying that the bootloader is six times the size of Firefox? Are you really suggesting that tens of gigabytes of source code are compiled into the bootloader? That would make the bootloader at least a few hundred megabytes of executable code, probably gigabytes, wouldn't it?
- conradev 1y ago
  and the author of Fil-C worked on that!
  - kragen 1y ago
    Oh, somehow I missed that connection!
- pizlonator 1y ago
  That was my idea and I wrote a good chunk of the compiler and runtime.
  - kragen 1y ago
    (For those who didn't make the connection, pizlonator also wrote Fil-C.)
  - frumplestlatz 1y ago
    [flagged]
    - pizlonator 1y ago
      Sounds like you weren't there
      - frumplestlatz 1y ago
        [flagged]
        deleted 1y ago
        [deleted]
        jibal 1y ago
        That breaks HN rules.
        kragen 1y ago
        (But revealing your identity would not.)
    - actionfromafar 1y ago
      You don't want to - so you did it anyway?
  - geocar 1y ago
- EPWN3D 1y ago
  You don't even need to reverse it. It's in the public clang, and I'm working on helping my team adopt it in some test cases. And it's not just the bounds-checking that's great -- it makes a bunch of C anti-patterns much harder, and it makes you think a lot harder about pointer ownership and usage. Really a great addition to the language, and it's source-compatible with empty macro-definitions (with two exceptions).
  - kragen 1y ago
    Interesting! How do you get started?
    - EPWN3D 1y ago
      -fbounds-safety is the flag. https://clang.llvm.org/docs/BoundsSafety.html https://clang.llvm.org/docs/BoundsSafety.html
  - pizlonator 1y ago
    > It's in the public clang I think you’re thinking of something else
    - debugnik 1y ago
      It or something similar has apparently been upstreamed to clang as -fbounds-safety. I don't know if they're the same, but the RFC on -fbounds-safety does give some credit to the author of Fil-C, who also took credit for firebloom on this thread as well.
      - pizlonator 1y ago
        Not the same
        debugnik 1y ago
        Thanks for clarifying, I had found a past HN comment stating they are the same.
- pjmlp 1y ago
  Not for much longer, replacing it with Swift is one of the reasons why Swift Embedded subset came to be. "Why Embedded Swift" https://youtu.be/LqxbsADqDI4?t=144 https://youtu.be/LqxbsADqDI4?t=144
TuxSH 1y ago
Also this is _de facto_ limited to userspace application for the mainstream OSes if my understanding is correct. Reading Fil-C website's "InvisiCaps by example" page, I see that "Laundering Integers As Pointers" is disallowed. This essentially disqualifies Fil-C for low-level work, which makes for a substantial part of C programs. (int2ptr for MMIO/pre-allocated memory is in theory UB, in practice just fine as long as you don't otherwise break aliasing rules (and lifetime rules in C++) - as the compiler will fail to track provenance at least once). But that isn't really what Fil-C is aimed at - the value is, as you implied, in hardening userspace applications.
- kragen 1y ago
  Yes, I think that's reasonable. I imagine you wouldn't have to extend Fil-C very much to sneak some memory-mapped I/O addresses into your program, but maybe having the garbage collector pause the program in the middle of an interrupt handler would have other bad effects. Like, if you were generating a video signal, you'd surely get display glitches.
- mbrock 1y ago
  Check out this document to see how the Fil-C ports of Python and Perl and so on work: https://github.com/mbrock/filnix/blob/main/ports/analysis.md https://github.com/mbrock/filnix/blob/main/ports/analysis.md This is still within the userspace application realm but it's good to know that Fil-C does have explicit capability-preserving operations (`zxorptr`, `zretagptr`, etc) to do e.g. pointer tagging, and special support for mapping pointers to integer table indices and back (`zptrtable`, etc).
- pizlonator 1y ago
  It’s not so fundamental of a limitation. Fil-C already allows memory mapped I/O in the form of mmap. The only thing missing that is needed for kernel level MMIO is a way to forge a capability. I don’t allow that right now, but that’s mostly a policy decision. It also falls out from the fact that InvisiCaps optimize the lower by having it double as a pointer to the top of the capability. That’s also not fundamental; it’s an implementation choice. It’s true that InvisiCaps will always disallow int to ptr casts, in the sense that you get a pointer with no capability. You’d want MMIO code to have some intrinsic like `zunsafe_forge_ptr` that clearly calls out what’s happening and then you’d use that wherever you define your memory mapped registers.
nickpsecurity 1y ago
Softbound + CETS was one of the old attempts I liked: https://people.cs.rutgers.edu/~santosh.nagarakatte/softbound/ https://people.cs.rutgers.edu/~santosh.nagarakatte/softbound... CCured was another: https://people.eecs.berkeley.edu/~necula/Papers/ccured_popl02.pdf https://people.eecs.berkeley.edu/~necula/Papers/ccured_popl0...
- kragen 1y ago
  Good to see you back! I hope you're doing okay; I know you ran into some real trouble a couple of years ago.
  - nickpsecurity 1y ago
    Up to my eyeballs in trouble. Working two jobs, too, where I can't practice tech skills. Fortunately, I have the peace of Jesus Christ in hopeless situations. He might change them at any time, too. I still try to squirrel away a little time to learn from and contribute to the tech community. Mostly here and on AI subs like r/mlscaling. The best stuff that I can't work on or even curate right now I'm saving in case time or funding open up in the future. Worst case, I can pass it onto bright minds with the skills to build it. Like before in security. You were doing neat things with satellites and bootstrapping research last I checked. What interesting jobs or tech are you digging into now?
    - kragen 1y ago
      Mostly I've just been struggling to keep my head above water, but I've been experimenting with submilliwatt autonomous personal computing, LuaJIT, and radically simplified windowing systems. Planning to visit the US next year for the first time in almost 20 years.
Ygg2 1y ago
> but I think most C programmers won't be willing to take a 4× performance hit. At a 4x performance hit, you might as well use C# or Go. > Fil-C's target audience is users of C programs, not authors of C programs. Sure, but then they don't get it for free. There is a perf penalty from GC. Plus you need all the original sources, right? > we lose access to that intellectual heritage. Declining usage of C is going to make you lose intellectual heritage[1]. A language no one can read or write is a dead language, regardless if you can translate it to English or not. [1] And that is outside Rust's or Zig's influence. It's an old language from when people thought you can trust the programmer. Which may well be the case if only people using it were Bell Labs engineers. It's got UB up the wazoo, no safety, and no sane package manager.
- mbrock 1y ago
  > you might as well use C# or Go Except, uh, you can't use C# or Go to run a program written in C/C++. Oh, you mean we can solve all our problems by simply rewriting all legacy software? Right, I forgot!
  - einsteinx2 1y ago
    > Oh, you mean we can solve all our problems by simply rewriting all legacy software? Isn’t that Rust’s raison d'être? (Just kidding…mostly)
    - Ygg2 1y ago
      Pretty sure that's a programmer's raison d'être. Not invented here (NIH) syndrom predates Rust.
      - einsteinx2 1y ago
        Fair point, though it does seem to be more common in the Rust community to the point that it’s become a bit of a meme. Related funny anecdote, I recently saw a Show HN post title that made sure to mention the thing was written in Rust, but forgot to mention what it actually did. Priorities lol. I’m sure it sounds like I’m a Rust hater or something, but I’m really not. I like a lot of the new Rust tools being created and use a few myself (ripgrep, fd, and bat immediately come to mind but I’m sure there are more I’m using). I just find the almost religious fervor of the Rust community amusing.

mbrock 1y ago

I'm working on packaging Fil-C for Nix, as well as integrating Fil-C as a toolchain in Nix so you can build any Nix package with Fil-C. https://github.com/mbrock/filnix https://github.com/mbrock/filnix It's working. It builds tmux, nethack, coreutils, Perl, Tcl, Lua, SQLite, and a bunch of other stuff. Binary cache on https://filc.cachix.org https://filc.cachix.org so you don't have to wait 40 minutes for the Clang fork to build. If you have Nix with flakes on a 64-bit Linux computer, you can run nix run github:mbrock/filnix#nethack right now!

kragen 1y ago
That's very exciting! Thank you!
yjftsjthsd-h 1y ago
That is super cool, and I will probably start running it on at least a test box shortly. How does python work? Of course I can just add filc.python to my system, but if I `python3 -m pip install whatever` will it just rebuild any C modules with the fil-c compiler?
- mbrock 1y ago
  Right now Python simply does not work at all. I was trying last night just to get the compiler to build, but ran into some Fil-C panics at the last moment when it tries to "freeze" some modules? But it should work. My idea is to move towards defining Fil-C as a "platform", meaning it would have its own ABI value, so it would look like a target called "x86_64-unknown-linux-filc", and then the magic of Nixpkgs is that it can instantiate a full realm called "pkgsCross.filc" which automatically has every package, whose packages are built on your ordinary non-FilC platform as a cross compilation. When that works—I hope it works, I probably need help to get it working—you should be able to use all the Nixpkgs packaging helpers like pkgs.mkShell { packages = [ (pkgsCross.filc.python3.withPackages (pypkgs: with pypkgs; [ pandas requests ])) ]; } This is already how Nixpkgs can have a pkgsCross.musl that builds everything with Musl as a cross compilation. So it seems reasonable and possible. It should be of wide interest for Nix users, since it would suddenly allow memory safety for whole swaths of the C/C++ packages, it would let NixOS use memory safe builds for core services like OpenSSHd, and so on... I should probably try to gather some interest among people who are truly familiar with Nixpkgs cross compilation...

nextaccountic 1y ago

Somewhat related, safe C++ proposal is not being continued https://news.ycombinator.com/item?id=45234460 https://news.ycombinator.com/item?id=45234460

dmitrygr 1y ago

TLDR: 4x slowdown in the normal case the performance overhead of this approach for most programs makes them run about four times more slowly

pizlonator 1y ago
> TLDR: 4x slowdown in the normal case 4x slower isn't the normal case. 4x is at the upper end of the overheads you'll see.
- geraldog 1y ago
  That's good to know! C is immensely powerful, portable and probably as fast as you can go without hand-coding in the architecture-specific assembly. Most of the world's information systems (our cyberstructure) rely directly or indirectly on C. And don't get me wrong, I'm a great enthusiast of the idea of sticking to memory-safe languages like Rust from now on. The hard truth is will live with legacy C code, period. Pizlo's heroic effort bridges the gap so to speak, it kind of sandboxes userspace C in a way that inherently adds memory safety to legacy code. There are only a few corner cases now that can't be bothered by any slow-down vis-a-vis unsafe C, and the great majority of code across every industry would benefit much more from the reduced surface of exposure.
- thw_9a83c 1y ago
  > 4x is at the upper end of the overheads This is already quite impressive. Many automatic memory managed languages have more than 4x worst-case slowdown. E.g. Google-backed Golang is ~1.5× to ~4× slower than optimized C++. I suppose there are many ways to further speed-up Fil-C if more resources were given to the project.
  - hyghjiyhu 1y ago
    This seems likely. I would assume that the vast majority of memory access can be statically proven safe if you throw enough money at the problem, meaning the bookkeeping can be eliminated.

EverydayBalloon 1y ago

[dead]

braggerxyz 1y ago

Every time I read C and memory safety, I just think Golang. Especially for user space

thomasmg 1y ago
Go programs are not fully memory-safe if they use multiple treads, due to possible data races with fat pointers: https://news.ycombinator.com/item?id=44672003 https://news.ycombinator.com/item?id=44672003
- pjmlp 1y ago
  Still much better than C will ever be, even if I disagree with some of its design decisions.
pjmlp 1y ago
Yes, think of something like Inferno, but Limbo now is AOT compiled instead of JITed. However there are also kernel like commercial projects in Go, and apparently the related TamaGo fork might eventually get upstreamed into the reference implementation. https://www.withsecure.com/en/solutions/innovative-security-hardware/usb-armory https://www.withsecure.com/en/solutions/innovative-security-...

ridiculous_fish 1y ago

Extraordinary project. I had several questions which I believe I have answered for myself (pizlonator please correct if wrong): 1. How do we prevent loading a bogus lower through misaligned store or load? Answer: Misaligned pointer load/stores are trapped; this is simply not allowed. 2. How are pointer stores through a pointer implemented (e.g. `*(char **)p = s`) - does the runtime have to check if *p is "flight" or "heap" to know where to store the lower? Answer: no. Flight (i.e. local) pointers whose address is taken are not literally implemented as two adjacent words; rather the call frame is allocated with the same object layout as a heap object. The flight pointer is its "intval" and its paired "lower" is at the same offset in the "aux" allocation (presumably also allocated as part of the frame?). 3. How are use-after-return errors prevented? Say I store a local pointer in a global variable and then return. Later, I call a new function which overwrites the original frame - can't I get a bogus `lower` this way? Answer: no. Call frames are allocated by the GC, not the usual C stack. The global reference will keep the call frame alive. That leads to the following program, which definitely should not work, and yet does. ~Amazing~ Unbelievable: #include <stdio.h> char *bottles[100]; __attribute__((noinline)) void beer(int count) { char buf[64]; sprintf(buf, "%d bottles of beer on the wall", count); bottles[count] = buf; } int main(void) { for (int i=0; i < 100; i++) beer(i); for (int i=99; i >= 0; i--) puts(bottles[i]); }

jibal 1y ago
Hmmm ... there's a danger here that people will test their programs compiled with Fil-C and think that they are safe to compile with a "normal" compiler. I would hope for an option to flag any undefined behavior.

Panzerschrek 1y ago

Great effort, but I find the whole idea somewhat flawed. If one needs speed, he can't use this C implementation, because it's several times slower. If speed isn't important, why not just using a memory safe language? And if both are important, why not using Rust? Recompiling existing software written in C using Fil-C isn't also a great idea, since some modifications are likely needed, at least for fixing bugs found with usage of Fil-C. And after these bugs are fixed, why continue using Fil-C?

mrunix 1y ago
Most software works flawlessly without modifications on Fil-C, the performance isn't *that* bad and there are applications where security is more important than performance (for example military applications)
veltas 1y ago
You're glossing over the "just rewrite everything in Rust" part.
Filligree 1y ago
Safety isn’t really optional. Up until now, we’ve had no choice but to run C and accept the bugs; now we’re allowed to also run it safely. Yes, this means that C is one of the slower languages around. That’s fine; computers are fast. If you want to write high-performance code, there are plenty of faster languages to do it with.
- pjmlp 1y ago
  There were several options, but the industry decided to bet on UNIX and C as main inspiration going forwards from the 1980's. "A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law." -- C.A.R Hoare's "The 1980 ACM Turing Award Lecture" Guess what he means by "1980 language designers and users have not learned this lesson".

cjk 1y ago

I worked with the author of Fil-C at Apple while on the Safari team, and he's easily one of the brightest folks I've had the pleasure of knowing. Fil-C looks extremely cool.

gnabgib 1y ago

No discussion, but just on the front page last week (31 points) https://news.ycombinator.com/item?id=45655519 https://news.ycombinator.com/item?id=45655519 Previous discussion: 2025 Safepoints and Fil-C (87 points, 1 month ago, 44 comments) https://news.ycombinator.com/item?id=45258029 https://news.ycombinator.com/item?id=45258029 2025 Fil's Unbelievable Garbage Collector (603 points, 2 months ago, 281 comments) https://news.ycombinator.com/item?id=45133938 https://news.ycombinator.com/item?id=45133938 2024 The Fil-C Manifesto: Garbage In, Memory Safety Out (13 points, 17 comments) https://news.ycombinator.com/item?id=39449500 https://news.ycombinator.com/item?id=39449500

synergy20 1y ago

posted multiple times, x86 only last time I checked

lambdaone 1y ago
All the more reason to make it portable. I wonder if this can be implemented via LLVM?
- kragen 1y ago
  It is implemented via LLVM.
pizlonator 1y ago
Yeah because I’m limiting my test matrix. There’s nothing about how Fil-C is designed that constrains it to x86_64. It doesn’t strongly rely on x86’s memory model. It doesn’t strongly rely on 64-bit. I’m focusing on one OS and arch until I have more contributors and so more bandwidth to track bugs across a wider set of platforms.
- khamidou 1y ago
  Hey Filip – while we're talking about memory architecture, have you started looking at ARM's EMTE extension (e.g https://security.apple.com/blog/memory-integrity-enforcement/ https://security.apple.com/blog/memory-integrity-enforcement...)? Could it eventually replace invisicaps?
  - pizlonator 1y ago
    No. MTE is probabilistic. Fil-C is deterministic.
- erichocean 1y ago
  If you're taking votes, I would vote for AArch64 next. :D
  - 1y ago