10 ms·

A Note on Fil-C

241 points by signa11 1y ago

computersuck 1y ago
403's until you go to https://graydon2.dreamwidth.org/ https://graydon2.dreamwidth.org/ first
- AceJohnny2 1y ago
  I only got a captcha prompt on the direct link. Perhaps you have something that disables the captcha, and thus got 403'd?
- cesarb 1y ago
  403 until you go to https://graydon2.dreamwidth.org/ https://graydon2.dreamwidth.org/ first with JavaScript enabled and temporarily allow third-party scripts from awswaf.com.
pizlonator 1y ago
This is super kind and awesome, I'm seriously flattered!
- keyle 1y ago
  Great work for the community. This has obviously been 'rust'ling some feathers, as it challenges some of the arguments laid past; but once the dust settles, it is a major net benefit to the community. I hope you get financed and can support other platforms than linux again.
  - goku12 1y ago
    > This has obviously been 'rust'ling some feathers, I'm a Rust user and a fan. But memory safe C is actually an exciting prospect. I was hoping that the rise of Rust would encourage others to prioritize memory safety and come up with approaches that are much more ergonomic to the developers. > as it challenges some of the arguments laid past Genuinely curious. What are the assumptions you have in mind that Fil-C challenges? (This isn't a rhetorical question. I'm just trying to understand memory safety concepts better.) > but once the dust settles, it is a major net benefit to the community. Agreed, this is big! If Fil-C can fulfill its promise to make old C code memory safe, it will be a massive benefit to the world. God knows how many high-consequnce bugs and vulnerabilities hide in those.
    - vanderZwan 1y ago
      > I was hoping that the rise of Rust would encourage others to prioritize memory safety and come up with approaches that are much more ergonomic to the developers. That's the end-goal right? I don't write Rust code myself, but I'm glad its existence means there's safer code out there now, and like you I have been looking forward to seeing shifts in safety expectations. I'm not surprised that it's happening so slowly though.
    - pjmlp 1y ago
      Same here, I don't have any use for Rust, and am perfectly fine with automatic resource management languages (regardless of the approach). However, Rust has been quite successful making more developers think about less known type systems, besides affine types, there is also linear types, effects, dependent types, prof systems. And we as industry aren't going to throw away the millions and millions of stuff that was written in C, C++ and less extent Objective-C, thus efforts like Fil-C are quite welcomed.
      - dfawcus 1y ago
        Now that suggests an idea: Objective-Fil-C !
- brucehoult 1y ago
  Did you know each other at Apple? Long long ago, in 2009, Graydon was my official on-boarding mentor when I joined the Mozilla Javascript team. Rust already existed then but, as he notes, was quite different then. For one thing, it was GC'd, like Fil-C. Which I like -- I write a lot of my C/C++ code using Boehm GC, have my own libraries designed knowing GC is there, etc.
  - pizlonator 1y ago
    Yeah we interacted at Apple
kardos 1y ago
I suppose /some/ performance loss is inevitable. But this could be quite a game changer. As more folks play with it, performing benchmarks, etc -- it should reveal which C idioms incur the most/least performance hits under Fil-C. So with some targetted patching of C code, we may end up with a rather modest price for the memory safety
- pizlonator 1y ago
  And I'm not done optimizing. The perf will get better. Rust and Yolo-C will always be faster, but right now we can't know what the difference will be. Top optimization opportunities: - InvisiCaps 2.0. While implementing the current capability model, when I was about 3/4 of the way done with the rewrite, I realized that if I had done it differently I would have avoided two branch+compares on every pointer load. That's huge! I just haven't had the appetite for doing yet another rewrite recently. But I'll do it eventually. - ABI. Right now, Fil-C uses a binary interface that relies on lowering to what ELF is capable of. This introduces a bunch of overhead on every global variable access and every function call. All of this goes away if Fil-C gets its own object file format. That's a lot of work, but it will happen in Fil-C gets more adoption. - Better abstract interpreter. Fil-C already has an abstract interpreter in the compiler, but it's not nearly as smart as it could be. For example, it doesn't have octagon domain yet. Giving it octagon domain will dramatically improve the performance of loops. - More intrinsics. Right now, a lot of libc functions that are totally memory safe but are implemented in assembly are implemented in plain Fil-C instead right now, just because of how the libc ports happened to work out. Like, say you call some <math.h> function that takes doubles and returns doubles - it's going to be slower in Fil-C today because you'll end up in the generic C code version compiled with Fil-C. No good reason for this! It's just grunt work to fix! - The calling convention itself is trash right now - it involves passing things through a thread-local buffer. It's less trashy than the calling convention I started out with (that allocated everything in the heap lmao), but still. There's nothing fundamentally preventing a Fil-C register-based calling convention, but it would take a decent amount of work to implement. There are probably other perf optimization opportunities that I'm either forgetting right now or that haven't been found yet. It's still early days!
  - 1y ago
jsrcout 1y ago
Getting a "not available in your state" page, does anyone have an archive? I've only recently tried out fil-c and hope to use it in some work projects.
- dieggsy 1y ago
  https://web.archive.org/web/20251107024022/https://graydon2.dreamwidth.org/320265.html https://web.archive.org/web/20251107024022/https://graydon2....
pertymcpert 1y ago
I can see the value this brings vs regular C, but I’m less clear on what this brings on top of -fbounds-safety
- pizlonator 1y ago
  -fbounds-safety is awesome! Here’s what Fil-C gives you that -fbounds-safety doesn’t: - Fil-C gives you comprehensive memory safety while -fbounds-safety just covers bounds. For example, Fil-C panics on use after free and has well defined semantics on ptr-int type confusion. - -fbounds-safety requires you to modify your code. Fil-C makes unmodified C/C++ code memory safe. FWIW, I worked on -fbounds-safety and I still think it’s a good idea. :-)
thomasmg 1y ago
Yes, safety got more important, and it's great to support old C code in a safe way. The performance drop and specially the GC of Fil-C do limit the usage however. I read there are some ideas for Fil-C without GC; I would love to hear more about that! But all existing programming languages seem to have some disadvange: C is fast but unsafe. Fil-C is C compatible but requires GC, more memory, and is slower. Rust is fast, uses little memory, but us verbose and hard to use (borrow checker). Python, Java, C# etc are easy to use, concise, but, like Fil-C, require tracing GC and so more memory, and are slow. I think the 'perfect' language would be as concise as Python, statically typed, not require tracing GC like Swift (use reference counting), support some kind of borrow checker like Rust (for the most performance critical sections). And leverage the C ecosystem, by transpiling to C. And so would run on almost all existing hardware, and could even be used in the kernel.
- spijdar 1y ago
  Nim fits most of those descriptors, and it’s become my favorite language to use. Like any language, it’s still a compromise, but it sits in a really nice spot in terms of compromises, at least IMO. Its biggest downsides are all related to its relative “obscurity” (compared to the other mentioned languages) and resulting small ecosystem.
  - tptacek 1y ago
    The advantage of Fil-C is that it's C, not some other language. For the problem domain it's most suited to, you'd do C/C++, some other ultra-modern memory-safe C/C++ system, or Rust.
  - thomasmg 1y ago
    I agree. Nim is memory safe, concise, and fast. In my view, Nim lacks a very clear memory management strategy: it supports ARC, ORC, manual (unsafe) allocation, move semantics. Maybe supporting viewer options would be better? Usually, adding things that are lacking is easier than removing features, specially if the community is small and if you don't want to alienate too many people.
- sakompella 1y ago
aw1621107 1y ago
One thing I've been wondering recently about Fil-C - why now? And I don't mean that in a dismissive way at all, I'm genuinely curious about the history. Was there some relatively recent fundamental breakthrough or other change that prevented a Fil-C-like approach from being viable before? Was it a matter of finding the right approach/implementation (i.e., a "software" problem), or is there something about modern hardware which makes the approach impractical otherwise? Something else?
- pizlonator 1y ago
  I’ve been thinking about this problem since 2004. Here’s a rough timeline: - 2004-2018: I had ideas of how to do it but I thought the whole premise (memory safe C) was idiotic. - 2018-2023: I no longer thought the premise was idiotic but I couldn’t find a way to do it that would result in fanatical compatibility. - 2023-2024: early Fil-C versions that were much less compatible and much less performant - end of 2024: InvisiCaps breakthrough that gives current fanatical compatibility and “ok” performance. It’s a hard problem. Lots of folks have tried to find a way to do it. I’ve tried many approaches before finding the current one.
  - remexre 1y ago
    Beyond the Git history, is there any write-up of the different capability designs you've gone with? I'm interested in implementing a safe low-level language with less static information around than C has (e.g. no static pointer-int distinction), but I'd rather keep around the ability to restrict capabilities to only refer to subobjects than have the same compatibility guarantees Invisicaps provide, so I was hoping to look into Monocaps (or maybe another design, if there's one that might fit better).
    - pizlonator 1y ago
      I summarize past attempts in https://fil-c.org/invisicaps https://fil-c.org/invisicaps
  - aw1621107 1y ago
samdoesnothing 1y ago
It's always seemed obvious to me that it would be better to make C safer than it would be to rewrite the billions of lines of C that run all our digital infrastructure. Of course that will get pushback from people who care more about rewriting it in a specific language, but pragmatically it's the obvious solution. Nice to see stuff like Fil-C proving it's possible, and if the performance gap can get within 10% (which seems very possible) it would be a no-brainer.
- IshKebab 1y ago
  Memory safety isn't the only benefit of rewriting C code in Rust. IMO it's maybe not even the biggest. For example you also get a far stronger type system (leading to fewer logic bugs) and modern tooling.
- pornel 1y ago
  It depends how much the C software is "done" vs being updated and extended. Some legacy projects need a rewrite/rearchitecting anyway (even well-written battle-tested code may stop meeting requirements simply due to the world changing around it). It also doesn't have to be a complete all-at-once rewrite. Plain C can easily co-exist with other languages, and you can gradually replace it by only writing new code in another language.
wakawaka28 1y ago
[flagged]
- drnick1 1y ago
  And people who just don't like like the Rust "style" and would rather write new software in a familiar language with all the features like classic OOP they they are used to.
  - timeon 1y ago
    There are many languages in this space to choose from.
  - zozbot234 1y ago
    You can use classic OOP in Rust (even implementation inheritance, via the generic typestate pattern!) It's just not a good idea.
    - wakawaka28 1y ago
      When I heard that writing a linked list in Rust is a challenging problem, I knew it wasn't for me lol.
- hitekker 1y ago
  From my skim, I didn't see anything mean spirited. So far as I've seen, Graydon is not a zealot and he doesn't play political games. It was a shame to lose his guiding hand
  - wakawaka28 1y ago
    I'm not familiar with the guy and I can't say that this post is obviously mean. You certainly seem to think very well of him, which might cloud your judgement about what is actually in the post. It just comes off as soapboxing for Rust under the pretense of saying Fil-C is great. It's like reading a review for a movie that says "This movie was good for X, Y, Z reasons but this OTHER movie is at least as good, if not better." The man is certainly entitled to an opinion but I didn't get anything from it besides "I like Fil-C but Rust is still solving this problem a way I like better."
antirez 1y ago
"But almost all programs have paths that crash, and perhaps the density of crashes will be tolerable." This is a very odd statement. Mature C programs written by professional coders (Redis is a good example) basically never crash in the experience of users. Crashing, in such programs, is a rare occurrence mostly obtained by attackers on purpose, looking for code paths that generate a memory error that - if the program is used as it should - are never reached. This does not mean that C code never segfaults: it happens, especially when developed without care and the right amount of testing. But the code that is the most security sensitive, like C Unix servers, is high quality and crashes are mostly a security problem and a lot less a stability problem.
- zozbot234 1y ago
  How many "mature C programs" try to recover in a usable way when malloc() returns NULL? That's a crash - a well-behaved one (no UB involved) hence not one that would be sought by most attackers other than a mere denial of service - but still a crash.
  - okanat 1y ago
    On 64-bit systems (esp Linux ones) malloc almost never returns NULL but keeps overallocating (aka overcommiting). You don't get out of memory errors / kills until you access it.
    - sibellavia 1y ago
      Exactly. Also, it is extremely rare.
  - 1718627440 1y ago
    > when malloc() returns NULL? That's a crash - a well-behaved one (no UB involved) Wrong, dereferencing a NULL pointer is UB.
    - sph 1y ago
      Which on UNIXes is a crash because the zero page is unmapped so you get a SIGSEGV
testdelacc1 1y ago
One divide when it comes to using Fil-C is C as an application (git) vs C as a library from another language (libgit2). Suppose we assume that many C applications aren’t performance sensitive and can easily take a 2-4x performance hit without noticing. Browsers and OS internals being obvious exceptions. The ideal candidates are like the ones djb writes, and he’s already a convert to Fil-C. sudo, sshd, curl - all seem like promising candidates. But as far as I can tell, Fil-C doesn’t work for C libraries that can be called from elsewhere. Even if it could be made to work, the reason other languages like Python or Node use C libraries is for speed. If they were ok with it being 2-4x slower, they would just write ordinary Python or Javascript. C (and C++) are fundamentally important because of their use in performance sensitive contexts like operating systems, browsers and libraries. If we’re restricting Fil-C to pure C/C++ applications that aren’t performance sensitive, that might still be very important and useful, but it’s a small slice of the large C/C++ pie. Also, it’s a great tool for an existing C application, certainly. A performance hit in exchange for security is a reasonable trade off while making a battle hardened application work. But for a new application, would someone choose Fil-C over other performant GC languages like Go or Java or C#? I’d be keen to hear why. Still, I want to stress - this is a great project and it’ll generate a lot of value.
- zozbot234 1y ago
  Why can't it work? You need to assume that the C library is only ever passed well-behaved pointers and callbacks in order to avoid invoking UB that it can't know about - but other than that it's just a matter of marshaling from the usual C ABI to the Fil-C ABI, which should be doable.
  - testdelacc1 1y ago
    I’m assuming the calling program is a GC language like Python or Node (the most popular run times by far), but the same holds with other popular languages like Ruby. Why would a GC language call out to slow code, that runs its own separate GC. Now you have two GCs running, neither of which knows about the other. I’m not declaring it’s impossible, I’m asking why someone would want to do this. An example: GitHub’s entire business revolves around calling libgit2 (C) from Ruby. Are they more likely to slow down libgit2 and make it substantially more complex by running 2 GCs side by side, or are they going to risk accept any potential unsafety in regular C? It’s 100% the latter, I’ll bet on that.
Surac 1y ago
I am very excited about this. Thanks to HN I see such things. To bad normal media is not any longer interested in anything not AI. On Topic: I am quite sceptical about all this rusting only because we can. Going rust makes the amount of programmers willing to look at the code quite small. A way to add this static testing to c will on the other hand open up the whole c community to a needed thing: memory
- zozbot234 1y ago
  Garbage collection is GOFAI (the LISP folks came up with it) and of course GOFAI is AI.
- timeon 1y ago
  As article points out, this does not solve all the things Rust does (apart for memory/performance, things like point 3.). So new code would be preferable in something like Rust (and some other PLs). However lot of existing code is in C and most of it will stay in C. So Fil-C seems to be really useful here.
torginus 1y ago
There's something off-putting about how Rust people communicate. Like you're talking to people selling something, or people with ulterior motives. They love talikng about memory safety as that's the only thing standing between humanity having secure and reliable software. They love external audiences, as their brand of fearmongering, virtue signaling and offering of panacea-like solutions resonates with a certain kind of risk-averse decision maker, who tend to in power in uncomfortable numbers. First of all, Rust's 'fearless concurrency' largely boils down to 'no concurrency' - Rust has about as much concurrency as Javascript - you can copy objects between threads, but not share memory beyond that, with certain libraries allowing some escape hatches in Rust's case. Additionally the case for aliasing control leading to better and safer code that's easier to reason about just isn't really true in practice - it's so rare that, for example your function can accidentally can alias memory in strictly typed languages - and when it does it's kind of intentional. The concern pretty much only manifests in memcpy, as the ugly hack of 'strict aliasing' - assuming 2 pointers of incompatible types pointing to different bits of memory - works very well in practice. It even helps with situations people complain about in Rust, like when object A and B (both mutably borrowed) take a reference to a some common service object (which is super common) - but that sort of code simply doesn't compile in Rust. All in all, I don't dislike Rust as it is but the project's tendency to do activism trying to bully its technical skeptics into submission (which is unfortunately what all activism really is - when you lost the argument, try to louder than the other guy and paint him as reprehensible) - they focused on fixing the technical issues. There has been research into ownership schemes, some exist that are less restrictive than Rust's while offering the same safety guarantees. In my personal opinion Rust is not done on the conceptual level - by 'done' I mean Rust serving its purpose as the language it claims to be. Maybe there will be Rust 2.0 which will overhaul the ownership system completely or maybe there'll be another language that will do what Rust does but better. Edit: I wish I could claim I'm some sort of tin-foil conspiracy theorist, but I'm commenting under an article written by one of the key people behind Rust, and it reeks of this attitude.
- filleduchaos 1y ago
  > First of all, Rust's 'fearless concurrency' largely boils down to 'no concurrency' - Rust has about as much concurrency as Javascript - you can copy objects between threads, but not share memory beyond that ...this is just blatantly false? Like it is false to the extent that I am confused as to what you could even possibly be talking about - I don't know how anyone who has actually written non-trivial programs in both languages could come to the conclusion that they have the same memory model (or much of anything in common when it comes to threads, really).
truth_seeker 1y ago
THERE IS NO ALTERNATIVE TO ADOPTION OF THOUGHTFUL DESING AND PRINCIPLES.
- truth_seeker 1y ago
  Downvotes to my comment proves Truth is offendig most of the time :) cheers, anyway

A Note on Fil-C

241 points by signa11 1y ago

computersuck 1y ago

403's until you go to https://graydon2.dreamwidth.org/ https://graydon2.dreamwidth.org/ first

AceJohnny2 1y ago
I only got a captcha prompt on the direct link. Perhaps you have something that disables the captcha, and thus got 403'd?
cesarb 1y ago
403 until you go to https://graydon2.dreamwidth.org/ https://graydon2.dreamwidth.org/ first with JavaScript enabled and temporarily allow third-party scripts from awswaf.com.

pizlonator 1y ago

This is super kind and awesome, I'm seriously flattered!

keyle 1y ago
Great work for the community. This has obviously been 'rust'ling some feathers, as it challenges some of the arguments laid past; but once the dust settles, it is a major net benefit to the community. I hope you get financed and can support other platforms than linux again.
- goku12 1y ago
  > This has obviously been 'rust'ling some feathers, I'm a Rust user and a fan. But memory safe C is actually an exciting prospect. I was hoping that the rise of Rust would encourage others to prioritize memory safety and come up with approaches that are much more ergonomic to the developers. > as it challenges some of the arguments laid past Genuinely curious. What are the assumptions you have in mind that Fil-C challenges? (This isn't a rhetorical question. I'm just trying to understand memory safety concepts better.) > but once the dust settles, it is a major net benefit to the community. Agreed, this is big! If Fil-C can fulfill its promise to make old C code memory safe, it will be a massive benefit to the world. God knows how many high-consequnce bugs and vulnerabilities hide in those.
  - vanderZwan 1y ago
    > I was hoping that the rise of Rust would encourage others to prioritize memory safety and come up with approaches that are much more ergonomic to the developers. That's the end-goal right? I don't write Rust code myself, but I'm glad its existence means there's safer code out there now, and like you I have been looking forward to seeing shifts in safety expectations. I'm not surprised that it's happening so slowly though.
  - pjmlp 1y ago
    Same here, I don't have any use for Rust, and am perfectly fine with automatic resource management languages (regardless of the approach). However, Rust has been quite successful making more developers think about less known type systems, besides affine types, there is also linear types, effects, dependent types, prof systems. And we as industry aren't going to throw away the millions and millions of stuff that was written in C, C++ and less extent Objective-C, thus efforts like Fil-C are quite welcomed.
    - dfawcus 1y ago
      Now that suggests an idea: Objective-Fil-C !
brucehoult 1y ago
Did you know each other at Apple? Long long ago, in 2009, Graydon was my official on-boarding mentor when I joined the Mozilla Javascript team. Rust already existed then but, as he notes, was quite different then. For one thing, it was GC'd, like Fil-C. Which I like -- I write a lot of my C/C++ code using Boehm GC, have my own libraries designed knowing GC is there, etc.
- pizlonator 1y ago
  Yeah we interacted at Apple

kardos 1y ago

I suppose /some/ performance loss is inevitable. But this could be quite a game changer. As more folks play with it, performing benchmarks, etc -- it should reveal which C idioms incur the most/least performance hits under Fil-C. So with some targetted patching of C code, we may end up with a rather modest price for the memory safety

pizlonator 1y ago
And I'm not done optimizing. The perf will get better. Rust and Yolo-C will always be faster, but right now we can't know what the difference will be. Top optimization opportunities: - InvisiCaps 2.0. While implementing the current capability model, when I was about 3/4 of the way done with the rewrite, I realized that if I had done it differently I would have avoided two branch+compares on every pointer load. That's huge! I just haven't had the appetite for doing yet another rewrite recently. But I'll do it eventually. - ABI. Right now, Fil-C uses a binary interface that relies on lowering to what ELF is capable of. This introduces a bunch of overhead on every global variable access and every function call. All of this goes away if Fil-C gets its own object file format. That's a lot of work, but it will happen in Fil-C gets more adoption. - Better abstract interpreter. Fil-C already has an abstract interpreter in the compiler, but it's not nearly as smart as it could be. For example, it doesn't have octagon domain yet. Giving it octagon domain will dramatically improve the performance of loops. - More intrinsics. Right now, a lot of libc functions that are totally memory safe but are implemented in assembly are implemented in plain Fil-C instead right now, just because of how the libc ports happened to work out. Like, say you call some <math.h> function that takes doubles and returns doubles - it's going to be slower in Fil-C today because you'll end up in the generic C code version compiled with Fil-C. No good reason for this! It's just grunt work to fix! - The calling convention itself is trash right now - it involves passing things through a thread-local buffer. It's less trashy than the calling convention I started out with (that allocated everything in the heap lmao), but still. There's nothing fundamentally preventing a Fil-C register-based calling convention, but it would take a decent amount of work to implement. There are probably other perf optimization opportunities that I'm either forgetting right now or that haven't been found yet. It's still early days!
- 1y ago

jsrcout 1y ago

Getting a "not available in your state" page, does anyone have an archive? I've only recently tried out fil-c and hope to use it in some work projects.

dieggsy 1y ago
https://web.archive.org/web/20251107024022/https://graydon2.dreamwidth.org/320265.html https://web.archive.org/web/20251107024022/https://graydon2....

pertymcpert 1y ago

I can see the value this brings vs regular C, but I’m less clear on what this brings on top of -fbounds-safety

pizlonator 1y ago
-fbounds-safety is awesome! Here’s what Fil-C gives you that -fbounds-safety doesn’t: - Fil-C gives you comprehensive memory safety while -fbounds-safety just covers bounds. For example, Fil-C panics on use after free and has well defined semantics on ptr-int type confusion. - -fbounds-safety requires you to modify your code. Fil-C makes unmodified C/C++ code memory safe. FWIW, I worked on -fbounds-safety and I still think it’s a good idea. :-)

thomasmg 1y ago

Yes, safety got more important, and it's great to support old C code in a safe way. The performance drop and specially the GC of Fil-C do limit the usage however. I read there are some ideas for Fil-C without GC; I would love to hear more about that! But all existing programming languages seem to have some disadvange: C is fast but unsafe. Fil-C is C compatible but requires GC, more memory, and is slower. Rust is fast, uses little memory, but us verbose and hard to use (borrow checker). Python, Java, C# etc are easy to use, concise, but, like Fil-C, require tracing GC and so more memory, and are slow. I think the 'perfect' language would be as concise as Python, statically typed, not require tracing GC like Swift (use reference counting), support some kind of borrow checker like Rust (for the most performance critical sections). And leverage the C ecosystem, by transpiling to C. And so would run on almost all existing hardware, and could even be used in the kernel.

spijdar 1y ago
Nim fits most of those descriptors, and it’s become my favorite language to use. Like any language, it’s still a compromise, but it sits in a really nice spot in terms of compromises, at least IMO. Its biggest downsides are all related to its relative “obscurity” (compared to the other mentioned languages) and resulting small ecosystem.
- tptacek 1y ago
  The advantage of Fil-C is that it's C, not some other language. For the problem domain it's most suited to, you'd do C/C++, some other ultra-modern memory-safe C/C++ system, or Rust.
- thomasmg 1y ago
  I agree. Nim is memory safe, concise, and fast. In my view, Nim lacks a very clear memory management strategy: it supports ARC, ORC, manual (unsafe) allocation, move semantics. Maybe supporting viewer options would be better? Usually, adding things that are lacking is easier than removing features, specially if the community is small and if you don't want to alienate too many people.
sakompella 1y ago

aw1621107 1y ago

One thing I've been wondering recently about Fil-C - why now? And I don't mean that in a dismissive way at all, I'm genuinely curious about the history. Was there some relatively recent fundamental breakthrough or other change that prevented a Fil-C-like approach from being viable before? Was it a matter of finding the right approach/implementation (i.e., a "software" problem), or is there something about modern hardware which makes the approach impractical otherwise? Something else?

pizlonator 1y ago
I’ve been thinking about this problem since 2004. Here’s a rough timeline: - 2004-2018: I had ideas of how to do it but I thought the whole premise (memory safe C) was idiotic. - 2018-2023: I no longer thought the premise was idiotic but I couldn’t find a way to do it that would result in fanatical compatibility. - 2023-2024: early Fil-C versions that were much less compatible and much less performant - end of 2024: InvisiCaps breakthrough that gives current fanatical compatibility and “ok” performance. It’s a hard problem. Lots of folks have tried to find a way to do it. I’ve tried many approaches before finding the current one.
- remexre 1y ago
  Beyond the Git history, is there any write-up of the different capability designs you've gone with? I'm interested in implementing a safe low-level language with less static information around than C has (e.g. no static pointer-int distinction), but I'd rather keep around the ability to restrict capabilities to only refer to subobjects than have the same compatibility guarantees Invisicaps provide, so I was hoping to look into Monocaps (or maybe another design, if there's one that might fit better).
  - pizlonator 1y ago
    I summarize past attempts in https://fil-c.org/invisicaps https://fil-c.org/invisicaps
- aw1621107 1y ago

samdoesnothing 1y ago

It's always seemed obvious to me that it would be better to make C safer than it would be to rewrite the billions of lines of C that run all our digital infrastructure. Of course that will get pushback from people who care more about rewriting it in a specific language, but pragmatically it's the obvious solution. Nice to see stuff like Fil-C proving it's possible, and if the performance gap can get within 10% (which seems very possible) it would be a no-brainer.

IshKebab 1y ago
Memory safety isn't the only benefit of rewriting C code in Rust. IMO it's maybe not even the biggest. For example you also get a far stronger type system (leading to fewer logic bugs) and modern tooling.
pornel 1y ago
It depends how much the C software is "done" vs being updated and extended. Some legacy projects need a rewrite/rearchitecting anyway (even well-written battle-tested code may stop meeting requirements simply due to the world changing around it). It also doesn't have to be a complete all-at-once rewrite. Plain C can easily co-exist with other languages, and you can gradually replace it by only writing new code in another language.

wakawaka28 1y ago

[flagged]

drnick1 1y ago
And people who just don't like like the Rust "style" and would rather write new software in a familiar language with all the features like classic OOP they they are used to.
- timeon 1y ago
  There are many languages in this space to choose from.
- zozbot234 1y ago
  You can use classic OOP in Rust (even implementation inheritance, via the generic typestate pattern!) It's just not a good idea.
  - wakawaka28 1y ago
    When I heard that writing a linked list in Rust is a challenging problem, I knew it wasn't for me lol.
hitekker 1y ago
From my skim, I didn't see anything mean spirited. So far as I've seen, Graydon is not a zealot and he doesn't play political games. It was a shame to lose his guiding hand
- wakawaka28 1y ago
  I'm not familiar with the guy and I can't say that this post is obviously mean. You certainly seem to think very well of him, which might cloud your judgement about what is actually in the post. It just comes off as soapboxing for Rust under the pretense of saying Fil-C is great. It's like reading a review for a movie that says "This movie was good for X, Y, Z reasons but this OTHER movie is at least as good, if not better." The man is certainly entitled to an opinion but I didn't get anything from it besides "I like Fil-C but Rust is still solving this problem a way I like better."

antirez 1y ago

"But almost all programs have paths that crash, and perhaps the density of crashes will be tolerable." This is a very odd statement. Mature C programs written by professional coders (Redis is a good example) basically never crash in the experience of users. Crashing, in such programs, is a rare occurrence mostly obtained by attackers on purpose, looking for code paths that generate a memory error that - if the program is used as it should - are never reached. This does not mean that C code never segfaults: it happens, especially when developed without care and the right amount of testing. But the code that is the most security sensitive, like C Unix servers, is high quality and crashes are mostly a security problem and a lot less a stability problem.

zozbot234 1y ago
How many "mature C programs" try to recover in a usable way when malloc() returns NULL? That's a crash - a well-behaved one (no UB involved) hence not one that would be sought by most attackers other than a mere denial of service - but still a crash.
- okanat 1y ago
  On 64-bit systems (esp Linux ones) malloc almost never returns NULL but keeps overallocating (aka overcommiting). You don't get out of memory errors / kills until you access it.
  - sibellavia 1y ago
    Exactly. Also, it is extremely rare.
- 1718627440 1y ago
  > when malloc() returns NULL? That's a crash - a well-behaved one (no UB involved) Wrong, dereferencing a NULL pointer is UB.
  - sph 1y ago
    Which on UNIXes is a crash because the zero page is unmapped so you get a SIGSEGV

testdelacc1 1y ago

One divide when it comes to using Fil-C is C as an application (git) vs C as a library from another language (libgit2). Suppose we assume that many C applications aren’t performance sensitive and can easily take a 2-4x performance hit without noticing. Browsers and OS internals being obvious exceptions. The ideal candidates are like the ones djb writes, and he’s already a convert to Fil-C. sudo, sshd, curl - all seem like promising candidates. But as far as I can tell, Fil-C doesn’t work for C libraries that can be called from elsewhere. Even if it could be made to work, the reason other languages like Python or Node use C libraries is for speed. If they were ok with it being 2-4x slower, they would just write ordinary Python or Javascript. C (and C++) are fundamentally important because of their use in performance sensitive contexts like operating systems, browsers and libraries. If we’re restricting Fil-C to pure C/C++ applications that aren’t performance sensitive, that might still be very important and useful, but it’s a small slice of the large C/C++ pie. Also, it’s a great tool for an existing C application, certainly. A performance hit in exchange for security is a reasonable trade off while making a battle hardened application work. But for a new application, would someone choose Fil-C over other performant GC languages like Go or Java or C#? I’d be keen to hear why. Still, I want to stress - this is a great project and it’ll generate a lot of value.

zozbot234 1y ago
Why can't it work? You need to assume that the C library is only ever passed well-behaved pointers and callbacks in order to avoid invoking UB that it can't know about - but other than that it's just a matter of marshaling from the usual C ABI to the Fil-C ABI, which should be doable.
- testdelacc1 1y ago
  I’m assuming the calling program is a GC language like Python or Node (the most popular run times by far), but the same holds with other popular languages like Ruby. Why would a GC language call out to slow code, that runs its own separate GC. Now you have two GCs running, neither of which knows about the other. I’m not declaring it’s impossible, I’m asking why someone would want to do this. An example: GitHub’s entire business revolves around calling libgit2 (C) from Ruby. Are they more likely to slow down libgit2 and make it substantially more complex by running 2 GCs side by side, or are they going to risk accept any potential unsafety in regular C? It’s 100% the latter, I’ll bet on that.

Surac 1y ago

I am very excited about this. Thanks to HN I see such things. To bad normal media is not any longer interested in anything not AI. On Topic: I am quite sceptical about all this rusting only because we can. Going rust makes the amount of programmers willing to look at the code quite small. A way to add this static testing to c will on the other hand open up the whole c community to a needed thing: memory

zozbot234 1y ago
Garbage collection is GOFAI (the LISP folks came up with it) and of course GOFAI is AI.
timeon 1y ago
As article points out, this does not solve all the things Rust does (apart for memory/performance, things like point 3.). So new code would be preferable in something like Rust (and some other PLs). However lot of existing code is in C and most of it will stay in C. So Fil-C seems to be really useful here.

torginus 1y ago

There's something off-putting about how Rust people communicate. Like you're talking to people selling something, or people with ulterior motives. They love talikng about memory safety as that's the only thing standing between humanity having secure and reliable software. They love external audiences, as their brand of fearmongering, virtue signaling and offering of panacea-like solutions resonates with a certain kind of risk-averse decision maker, who tend to in power in uncomfortable numbers. First of all, Rust's 'fearless concurrency' largely boils down to 'no concurrency' - Rust has about as much concurrency as Javascript - you can copy objects between threads, but not share memory beyond that, with certain libraries allowing some escape hatches in Rust's case. Additionally the case for aliasing control leading to better and safer code that's easier to reason about just isn't really true in practice - it's so rare that, for example your function can accidentally can alias memory in strictly typed languages - and when it does it's kind of intentional. The concern pretty much only manifests in memcpy, as the ugly hack of 'strict aliasing' - assuming 2 pointers of incompatible types pointing to different bits of memory - works very well in practice. It even helps with situations people complain about in Rust, like when object A and B (both mutably borrowed) take a reference to a some common service object (which is super common) - but that sort of code simply doesn't compile in Rust. All in all, I don't dislike Rust as it is but the project's tendency to do activism trying to bully its technical skeptics into submission (which is unfortunately what all activism really is - when you lost the argument, try to louder than the other guy and paint him as reprehensible) - they focused on fixing the technical issues. There has been research into ownership schemes, some exist that are less restrictive than Rust's while offering the same safety guarantees. In my personal opinion Rust is not done on the conceptual level - by 'done' I mean Rust serving its purpose as the language it claims to be. Maybe there will be Rust 2.0 which will overhaul the ownership system completely or maybe there'll be another language that will do what Rust does but better. Edit: I wish I could claim I'm some sort of tin-foil conspiracy theorist, but I'm commenting under an article written by one of the key people behind Rust, and it reeks of this attitude.

filleduchaos 1y ago
> First of all, Rust's 'fearless concurrency' largely boils down to 'no concurrency' - Rust has about as much concurrency as Javascript - you can copy objects between threads, but not share memory beyond that ...this is just blatantly false? Like it is false to the extent that I am confused as to what you could even possibly be talking about - I don't know how anyone who has actually written non-trivial programs in both languages could come to the conclusion that they have the same memory model (or much of anything in common when it comes to threads, really).

truth_seeker 1y ago

THERE IS NO ALTERNATIVE TO ADOPTION OF THOUGHTFUL DESING AND PRINCIPLES.

truth_seeker 1y ago
Downvotes to my comment proves Truth is offendig most of the time :) cheers, anyway