21 ms·
The provenance memory model for C
- zombot 1y agoDoes C allow Unicode identifiers now, or is that pseudo code? The code snippets also contain `&`, so something definitely went wrong with the transcoding to HTML.
- qsort 1y agoQuoting cppreference: An identifier is an arbitrarily long sequence of digits, underscores, lowercase and uppercase Latin letters, and Unicode characters specified using \u and \U escape notation(since C99), of class XID_Continue(since C23). A valid identifier must begin with a non-digit character (Latin letter, underscore, or Unicode non-digit character(since C99)(until C23), or Unicode character of class XID_Start)(since C23)). Identifiers are case-sensitive (lowercase and uppercase letters are distinct). Every identifier must conform to Normalization Form C.(since C23) In practice depends on the compiler.
- dgrunwald 1y agoBut the source character set remains implementation-defined, so compilers do not have to directly support unicode names, only the escape notation. Definitely a questionable choice to throw off readers with unicode weirdness in the very first code example.
- qsort 1y agoIf it were up to me, anything outside the basic character set in a source file would be a syntax error, I'm simply reporting what the spec says.
- ncruces 1y agoI use unicode for math in comments, and think makes certain complicated formulas far more readable.
- kzrdude 1y agoI've just been learning pinyin notation, so now i think the variable řₚ should have a value that first goes down a bit and then up.
- zelphirkalt 1y agoI am not sure it is a good idea to mix such specific phonetic script ideas about diacritic marks with the behavior of the program over time. Even considering the shape, it does not align with the idea of first down a little, then up a lot.
- kzrdude 1y agoTo be sure, it's a joke. Mostly trying to joke at the expense of these excessively complicated variable names (that are only there because it's pseudocode) :) And yeah, the chinese tone in practice does not align with the idea of "down a little up a lot" either. It depends on context...
- guipsp 1y agoWhat a "basic character set" is depends on locale
- qsort 1y agohttps://en.cppreference.com/w/c/language/charset.html https://en.cppreference.com/w/c/language/charset.html
- account42 1y agoAnything except US-ASCII in source code outside comments and string constants should be a syntax error.
- guipsp 1y agoYou are aware other languages exist? Some of which don't even use the Latin script?
- Y_Y 1y agoWhat; like APL‽
- nottorp 1y agoDunno about the OP but I'm very aware as I'm not an english speaker. I still don't want anything as unpredictable as Unicode in my code. How many different encodings will display as the same variable name and how is the compiler supposed to decide? If you're thinking of comments and user facing strings, the OP already excluded those.
- cryptonector 1y agoThe language and compiler & linker should reject Zalgo in identifiers, and they should reject confusable script mixes in identifiers, but otherwise they treat all equivalent strings as equivalent. To make it easier on the linker compilers should normalize all symbols to one common form (e.g., NFC).
- 1y ago
- unwind 1y agoI can't even view the post, I just get some kind of content management system-like with the page as JSON or something, in pink-on-white. I'm super confused. :| The answer to your question seems to (still) be "no".
- pjmlp 1y agoBesides the sibling comment on C23, it does work fine on GCC. https://godbolt.org/z/qKejzc1Kb https://godbolt.org/z/qKejzc1Kb Whereas clang loudly complains, https://godbolt.org/z/qWrccWzYW https://godbolt.org/z/qWrccWzYW
- Y_Y 1y agoImplementation-defined until C99, explicitly possible via UCNs aince c99, possible with explicit encoding since C23, but literals are still implementation defined.
- tialaramex 1y agoPresumably this was converted from markdown or similar and the conversion partly failed or the input was broken. From the PVI section onward it seems to recover, but if the author sees this please fix and re-convert your post. [Edited, nope, there are more errors further in the text, this needed proper proofreading before it was posted, I can somewhat struggle through because I already know this topic but if this was intended to introduce newcomers it's probably very confusing]
- gustedt 1y agoThe problem is that wordpress changes these things once you edit in some part. I will probably regenerate the whole.
- lioeters 1y agoLooks like a code block didn't get closed properly, before this phrase: > the functions `recip` and `recip⁺` and not equivalent Several paragraphs after this got swallowed by the code block. Edit: Oh, I didn't realize the article is by the author of the book, Modern C. I've seen it recommended in many places. > The C23 edition of Modern C is now available for free download from https://hal.inria.fr/hal-02383654 https://hal.inria.fr/hal-02383654
- johnisgood 1y agoIt is a great book. I prefer the second edition, not the latest one though with what I call "bloated C".
- laqq3 1y agoI'm wondering if you could elaborate? I'd be curious to hear more about "bloated C" and the differences between the 2nd and 3rd edition.
- lioeters 1y agoI was curious about this too, and found some discussion related to the topic of "bloated C" when the 3rd edition was announced. The C23 edition of Modern C - https://news.ycombinator.com/item?id=41850017 https://news.ycombinator.com/item?id=41850017 Like this comment: > Wow, the use of attributes like [[__unsequenced__]], [[maybe_unused]] and [[noreturn]] throughout the book is really awful. It seems pretty pedantic of the author to litter all the code examples with something that is mostly optional. Or this one: > Personally this just makes C much more complicated for me, and I choose C when I want simplicity. If I want complicated, I would just pick C++ which I typically would never want. Examples of what people consider "bloat" in newer C standards: _BitInt(N), guard, defer, auto, constexpr, nullptr _generic, typeof, restrict, syntax based tls
- johnisgood 1y agoYeah, I have a comment that I cannot find right now, they mention many of these things as well. They are all C++-esque, i.e. bloat, in my opinion. Edit: oh, you actually did quote me, too: https://news.ycombinator.com/item?id=41854897 https://news.ycombinator.com/item?id=41854897 In any case, thank you.
- gavinray 1y agoAlso of interest to folks looking at this might be TySan, the recently-merged LLVM Type-Based Aliasing sanitizer: https://clang.llvm.org/docs/TypeSanitizer.html https://clang.llvm.org/docs/TypeSanitizer.html https://www.phoronix.com/news/LLVM-Merge-TySan-Type-Sanitizer https://www.phoronix.com/news/LLVM-Merge-TySan-Type-Sanitize...
- aengelke 1y agoIt's probably worth noting that TySan currently only catches aliasing violations that LLVM would be able to exploit. For some types, e.g. unions, Clang doesn't emit accurate type-based aliasing information and therefore TySan won't catch these.
- flohofwoe 1y agoWhich is fine I think, considering that union type punning is legal in C (and even in C++ where union type punning is UB I have never seen it break - theoretically it might of course).
- uecker 1y agoThe problem might be that Clang does not even implement type-based aliasing correctly. So I assume it checks its broken rules, instead of the one specified in the C standard.
- jvanderbot 1y agoI love Rust, but I miss C. If C can be updated to make it generally socially acceptable for new projects, I'd happily go back for some decent subset of things I do. However, there's a lot of anxiety and even angst around using C in production code.
- mikewarot 1y agoIf you can stomach the occasional Begin and End, and a far less confusing pointer syntax, Pascal might be the language for you. Free Pascal has some great string handling, so you never have to worry about allocating and freeing them, and they can store gigabytes of text, even Unicode. ;-)
- jvanderbot 1y agoIf my fellow devs cringe at C, imagine their reaction to Pascal
- mikewarot 1y agoC has all the things to hate in a programming language CaSe Sensitivity Weird pointer syntax Lack of a separate assignment token Null terminated strings Macros - the evil scourge of the universe On the plus side, it's installed everywhere, and it's not indent sensitive
- jvanderbot 1y agoAt this point, you're talking to someone who isn't here
- ioasuncvinvaer 1y agoExcept for null terminated strings these don't seem like mayor issues to me. Can you elaborate?
- 1718627440 1y ago> Lack of a separate assignment token What does that mean?
- briandw 1y agoThe code blocks are very difficult to read on this page. I had ChatGPT O3 rewrite this in a more accessible format. https://chatgpt.com/share/68629096-0624-8005-846f-7c0d655061ac https://chatgpt.com/share/68629096-0624-8005-846f-7c0d655061...
- cenobyte 1y agoSo much better. Thank you!
- b0a04gl 1y agoprovenance model basically turns memory back into a typed value. finally malloc wont just be a dumb number generator, it'll act more like a capability issuer. and access is not 'is this address in range' anymore, but “does this pointer have valid provenance”. way more deterministic, decouples gcc -wall
- HexDecOctBin 1y agoWill this create more nasal demons? I always disable strict aliasing, and it's not clear to me after reading the whole article whether provenance is about making sane code illegal, or making previously illegal sane code legal.
- jcranmer 1y agoAll C compilers have some notion of pointer provenance embedded in them, and this is true going back decades. The problem is that the documented definitions of pointer provenance (which generally amount to "you must somehow have a data dependency from the original object definition (e.g., malloc)") aren't really upheld by the optimizer, and the effective definition of the optimizer is generally internally inconsistent because people don't think about side effects of pointer-to-integer conversion. The one-past-the-end pointer being equal (but of different provenance) to a different object is a particular vexatious case. The definition given in TS6010 is generally the closest you'll get to a formal description of the behavior that optimizers are already generally following, except for cases that are clearly agreed to be bugs. The biggest problem is that it makes pointer-to-int an operation with side effects that need to be preserved, and compilers today generally fail to preserve those side effects (especially when pointer-to-int conversion happens more as an implicit operation). The practical effect of provenance--that you can't magic a pointer to an object out of thin air--has always been true. This is largely trying to clarify what it means to actually magic a pointer out of thin air; it's not a perfect answer, but it's the best answer anyone's come up with to date.
- layer8 1y agoThis is basically a formalization of the general understanding one already had when reading the C standard thoroughly 25 years ago. At least I was nodding along throughout the article. It cleans up the parts where the standard was too imprecise and handwavy.
- deleted 1y ago[deleted]
- cenobyte 1y agoPlease fix the code in your post.
- eqvinox 1y agoUsing the "register" storage class feels really alien for C code written in 2025…
- flohofwoe 1y agoIt has a slightly different meaning now, instead of hinting to the compiler that the variable should be placed in a register it now means that it is illegal to take the address of the variable (e.g. cannot create a pointer from it): https://www.godbolt.org/z/eEYf5c59f https://www.godbolt.org/z/eEYf5c59f Might be useful in some situations although I currently can't think of any :)
- eqvinox 1y agoI mean, yeah, but that function is really only an aid for the programmer in self-enforcing that rule; the compiler already knows whether the address of the variable is taken anywhere, and behave as is useful if it isn't taken anywhere… Doesn't feel particularly valuable to have that "help" from the compiler against "accidentally" taking the address of a variable… I mean, how do you even accidentally do that?
- flohofwoe 1y agoI guess you're not a fan of 'const' either? ;)
- eqvinox 1y agoI am a fan of "const", because it is useful in expressing API constraints and behavior. Contrast against putting "register" on a function parameter being useless because either it's passed by value, then it's a copy anyway and register is meaningless to the caller, or it's a pointer, in which case it again does nothing because you already have a pointer (and something somewhere is very confused.)
- smcameron 1y agoUgh. Are unicode variable names allowed in C now? That's horrific.
- 1over137 1y agoHorrific? You might not think so if your (human) language used a different alphabet.
- ajross 1y agoLittle to no source code is written for single (human) language development teams. Sure, everyone would like the ability to write source code in their native language. That's natural. Literally no one, anywhere, wants to be forced to read source written in a language they can't read (or more specifically in this case: written in glyphs they can't even produce on their keyboard). That idea, for almost everyone, seems "horrific", yeah. So a lingua franca is a firm requirement for modern software development outside of extremely specific environments (FSB malware authors probably don't care about anyone else reading their cyrillic variable names, etc...). Must it be ASCII-encoded English? No. But that's what the market has picked and most people seem happy enough with it.
- OkayPhysicist 1y ago> Little to no source code is written for single (human) language development teams. This is blatantly false. I'd posit that a solid 90% of all source code written is done so by single, co-located teams (a substantial portion of which are teams of 1). That certainly fits the bill for most companies I've worked at.
- eqvinox 1y agoYes but also no. The thing about software is that 90% of it is not culturally bound. If you're writing, say, some tax reporting tool, a grammar reference, or something religious… sure, it makes sense to write that in your language. So, yeah, C should support that. However, everything else, from spreadsheet software to CAD tools to OS kernels to JavaScript frameworks is universal across cultures and languages. And for better or for worse (I'm not a native English speaker either), the world has gone with English for a lot of code commons. And the thing with the examples in that post isn't about supporting language diversity, it's math symbols which are noone's native language. And you pretty much can't type them on any keyboard. Which really makes it a rather poor flex IMHO. Did the author reconfigure their keyboard layout for that specific math use case? It can't generically cover "all of math" either. Or did they copy&paste it around? That's just silly. […could some of the downvoters explain why they're downvoting?]
- Joker_vD 1y ago> Here the term "same representation and alignment" covers for example the possibility to look at [...] one would be a structure and the other would be another structure that sits at the beginning of the first. Does it? It is quite simple for a struct A that has struct B as its first member to have radically different alignment: struct B { char x; }; struct A { struct B b; long long y; }; Also, accidentally coinciding pointers are nothing "rare" because all objects are allowed to be treated as 1-element arrays: so any pointer to an e.g. struct field is also a pointer one-past the previous field of this struct; also, malloc() allocations easily may produce "touching" objects. So thanks for allowing implementations to not have padding between almost every two objects, I guess.
- layer8 1y agoThis is about the representation and alignment of the pointer object, not about the object being pointed to. And C requires struct pointer types to all have the same representation and alignment. This is generally necessary due to the possibility of having pointers to opaque struct declarations in a translation unit. Regarding your second point, if I understand the model correctly, there is only an ambiguity in pointer provenance if the adjacent objects are independent "storage instances", i.e. separately malloc'ed objects or separate variables on the stack — not between fields of the same struct.
- dsp_person 1y agoif ((Π⁻ < Π) && (Π < Π⁺)) { I spent way too long trying to figure this out as C code instead of if ((Π⁻ < Π) && (Π < Π⁺)) {
- gustedt 1y agoRandomly introduced translation errors from markdown to wordpress-internal should be fixed, now. Sorry for the incovenience!
- cryptonector 1y agoThere are some grammar errors here and there, but TFA is very nice. Thank you for your hard work!
- nikic 1y agoAt least at a skim, what this specifies for exposure/synthesis for reads/writes of the object representation is concerning. One of the consequences is that dead integer loads cannot be eliminated, as they may have an exposure side effect. I guess C might be able to get away with it due to the interaction with strict aliasing rules. Still quite surprised that they are going against consensus here (and reduces the likelihood that these semantics will get adopted by implementers).
- uecker 1y ago(Never mind, I misread you comment at first.) Yes, the representation access needs to be discussed... I took a couple of years to publish this document. More important would be if the ptr2int exposure could be implemented.
- comex 1y ago> I guess C might be able to get away with it due to the interaction with strict aliasing rules. But not for char-typed accesses. And even for larger types, I think you would have to worry about the combo of first memcpying from pointer-typed memory to integer-typed memory, then loading the integer. If you eliminate dead integer loads, then you would have to not eliminate the memcpy.
- nikic 1y agoThat's a great point. I initially thought we could assume no exposure for loads with non-pointer-compatible TBAA, but you are right that this is not correct if the memory has been laundered through memcpy.
- uecker 1y agoYou can still eliminate the memcpy of if you mark the pointer exposed at this point.
- ben0x539 1y agoCan you say more about what the consensus is that this is going against?
- hinkley 1y ago> Unfortunately no C compiler can do this optimization automatically: > The functions recip and recip⁺ and not equivalent. This is one of those examples of how optimizing code can improve legibility, robustness, or both. The first implementation allows for side effects to change the outcome of the function. But the problem is that the code is not written expecting someone to modify the values in the middle of the loop. It's incorrect behavior, and you're paying a performance penalty for it to boot. Functional Core code tends not to have this problem, in that we pass in a snapshot of data and it either gets an answer or an error. I've seen too much code that checks 3 times if a user is either still logged in or has permission to do a task, and not one of them was set up to deal with one answer for the first call and a different one for any of the subsequent ones. They just go into undefined behavior.
- jaisio 1y agoThe root cause of all this is that C programs are not much more than glorified assembly programs. Any effort to retrofit higher level reasoning will always be defeated somebody doing some dirty pointer tricks. This can only be solved by more abstract ways to express programs which necessarily restricts the bare metal dirty things one can do. But what you gain is that the compiler will easily be able to do lots of things which a C compiler can't do or only with a lot of headache. The kind of stuff this article is about is really trying to solve the wrong problem IMO.
- RossBencina 1y agoAfter reading the fine article I'm left wondering what if you implement your own heterogeneous allocation scheme on top of malloc? (e.g. TLSF) In this case all of your objects will belong to the same malloced storage region, and you will compute object offsets using raw pointers, but I'd expect provenance to potentially treat each returned object to behave as if it were allocated from a separate disjoint storage. I guess my question is: does this provenance model allow for recursive nesting of allocators with a separate notion of "storage" at each level?
- f33d5173 1y agoThe compiler knows about malloc, and hence knows that the pointer returned by malloc won't alias any other pointer. Your compiler might support some attribute to mark a function as behaving like malloc in this respect. Otherwise the compiler will be forced to assume the return value could alias any other pointer.
- cryptonector 1y agoIMO there should be attributes for declaring allocators. Or builtin functions that have the effect of marking their callers with such attributes (e.g., an `__allocated()` function to say a pointer is indeed now to be considered a pointer to a new storage allocation, with a given size and optional type, and a `__freed()` function to say that a pointer is indeed now to be considered a dangling pointer to a deallocated object.
- nixpulvis 1y agoAs a bit of an aside, the example XOR doubly linked list example given here is super cool.
- cryptonector 1y ago:thank you: This is great. I wonder what u/pizlonator thinks of it.
- Measter 1y agoIn the section about the ambiguous provenance from synthesising pointers, it's explained that the compiler will infer the correct provenance from usage. Would it not be worth having some way for the programmer to inform the compiler directly, with something analogous to Rust's Strict Provenance ptr::with_addr? To convert it to C syntax, it's a function with roughly this signature: void* with_addr(void* ptr, uintptr_t addr) Where the returned pointer has the address of `addr` and the provenance of `ptr`.
- uecker 1y agoThe proposal is mostly designed this way to make sure existing code is valid. One could add something "with_addr", but I am not convinced that it is really worth it.
- charleslmunger 1y agoThis is doable via this trick: https://github.com/protocolbuffers/protobuf/blob/ae0129fcd0128b411c968d9a21eb93a5d0253ddf/upb/wire/eps_copy_input_stream.h#L241 https://github.com/protocolbuffers/protobuf/blob/ae0129fcd01...
- cryptonector 1y agoI'd also like to have builtin functions and/or function attributes for designating allocation and deallocation. malloc() and free() (and realloc()) should not be special because of their names -- they should be special because of their declared attributes or their derived attributes given their internals.