3 ms·
Array bounds errors in C, the number 1 problem, can be mitigated by adding this proposal to fix C: https://www.digitalmars.com/articles/C-biggest-mistake.html
by WalterBright 3y ago
Array bounds errors in C, the number 1 problem, can be mitigated by adding this proposal to fix C:
https://www.digitalmars.com/articles/C-biggest-mistake.html https://www.digitalmars.com/articles/C-biggest-mistake.html
I don't really understand the resistance to it. It does not affect existing code at all, it's simple, it's easy to implement, and its worth has been proven in D. (It's one of D's most popular features.)
- woodruffw 3y agoDo you have a link showing that the C standards body or working groups are resistant to this? My understanding is that they're currently more generally open to major security-oriented improvements than they ever have been in the past; it seems like now would be a good time for someone to put a slice/fat pointer proposal in front of them.
- WalterBright 3y agoEvery time I bring it up, people object to it, and nobody seems interested in proposing it to the Standards body. So what is my motivation in this, if I'm unwilling to make the effort to do a formal proposal? I know a lot about C, having implemented a C compiler from soup to nuts, and spent much of my career working with it. I have lot of experience with it, and with array overflows, both of my own doing and debugging other peoples' code. Very high on my list of reasons to do D was fixing that problem. I (along with Jan Knepper) developed the D array scheme. It works. It works great. It nails the problem so thoroughly that buffer overflow problems in D are very rare, because there just is little to no reason to use pointers to arrays in D. (I.e. buffer overflow problems in D happen when you don't use arrays.) Interestingly, it actually speeds up your code. How is that magic possible? It's because 0-terminated strings need to be constantly scanned to find the length, and arrays just know their length. It reduces memory consumption too! Because you can slice any string - not necessary to reallocate it. I've seen other extensions to C to solve the array overflow problem. They're all terrible, which is why one has to bludgeon people into using them. My proposal is simple, works, efficient, and people take to it like a duck to water. This problem also can only be solved at the Standards level, otherwise people will not use it. Writing an official proposal is a significant time sink, and I have enough to do working on D.
- pjmlp 3y ago50 years ignoring security issues, including Dennis own proposal for fat pointers.
- yeputons 3y agoI don't see how it mitigates anything. It merely adds a warning for _all_ older code and provides shiny new syntax exactly equivalent to an already popular practice of passing array and its length next to each other. All array-out-of bounds errors that were happening before will happen with the new syntax as well, won't they?
- woodruffw 3y agoIn theory, a new first-class slice type could only provide a “checked” interface, meaning that accesses through it would only succeed if in bounds. In practice, C doesn’t have great idioms for error handling, and so a proposal here would also probably need to explain how the slice type would either reuse existing idioms or introduce a new idiom (like Rust’s use of sum types). That makes the scope much larger, which in turn might explain any hesitancy from the C WGs to accept something like this (although I’m not actually aware of any).
- mjevans 3y agoProbably a warning flag that notes any non-compile time bounds checked references. Now that you mention it, some method of forcing bounds checks in more limited circumstances might be nice for performance, that or a cleaner method of expressly disabling such for areas with critical performance and where the input is considered trusted or already validated.
- WalterBright 3y agoThe array can be simply converted to a pointer to get unchecked access.
- WalterBright 3y agoArray bounds errors in D are treated as "go directly to jail, do not pass Go do not collect $200" errors. I.e. they are not recoverable, so no need for error handling. Many people have debated this and wanted such errors to be recoverable, but the principle in play is that the program is in an undefined state if there are such errors, and should not be allowed to continue as we wouldn't want it to launch nuclear missiles.
- jsd1982 3y agoWith your proposal, how is the dimension determined from `char a[...]` syntax? Does it act like an implicit struct with standardized names for the size_t dimension and char* pointer fields?
- WalterBright 3y agoThere are options. The D way is to have a `.length` suffix, which makes it act like a struct field. (And `.ptr` if you want to access the pointer part directly.) Another way is to have a special construct, like C's _Generic, to provide the access.
- mjevans 3y agoThat, and please 'fix struct packing'. For a moment I considered, 'oh just pass a struct around as the logical tuple of this data instead' and then I remembered they don't have to conform to any particular signature. extern void foo(size_t dim, char a[dim]); struct vecStr {size_t dim; char a[dim];}; // Not guaranteed ABI compatibility with above extern void foo(struct vecStr); It would be really nice if, formally, structs mostly happened to work as expected (if ordered reasonably, behave as packed/smallest) and if there were language, not just some compiler, supported prefix attributes that enforced either packed (smallest), fastest, or if some other methods are needed later those. Possibly manually specifying a target alignment. Also at this point far too much code depends on signed integers as the default, but it would be nice if for added clarity stdint.h and similar had complimentary sintX_t equivalents to uintX_t even though sint8_t would just be a syntax alias for int8_t (preferably the reverse).
- eschaton 3y agoI expect that there are probably a few significant points of technical resistance: 1. Introducing a new syntactic token (`..`) when there are already a lot of those in C. 2. Transforming an argument `char foo[..]` to the argument pair `const size_t foo_dim, char * const foo` leaves out a ton of existing APIs that order their arguments differently or breaks their binary compatibility. 3. It doesn’t accommodate APIs where both an element size and element count are passed, e.g. `fread(3)`. C is a long-lived language so modifications like this have to work within a very large existing body of code. They can be made to work—blocks have worked out wonderfully, for example, by essentially saying they’re only for new APIs—but this type of extension requires a lot of consideration beyond “we could tweak the syntax and get this feature.”
- WalterBright 3y ago1. `..` doesn't conflict with anything else in C. 2. APIs designed around pointers will not work with this proposal. But it would provide motivation for those APIs to change, and of course it will take time. Just like it took time for C function prototyping to take hold. 3. That's right, it doesn't. > C is a long-lived language so modifications like this have to work within a very large existing body of code It doesn't break existing code in any way. However, code would have to be modified to take advantage of it, sure. Just like _Generic.
- eschaton 3y agoAnd then none of the benefits are available to existing code without a rewrite. That’s why some solutions to this involve applying attributes to function prototype arguments and variable declarations instead, so you don’t have to rewrite everything to discover and prevent bugs.
- WalterBright 3y ago> And then none of the benefits are available to existing code without a rewrite. That's right. > That’s why some solutions to this involve applying attributes to function prototype arguments and variable declarations instead, so you don’t have to rewrite everything to discover and prevent bugs. In every case I've seen you have to rewrite the code to use bounds checking. Note that the proposed arrays can be trivially converted to pointers, so you can still interact with pointer interfaces, and even length+pointer interfaces, without rewriting those interfaces. I submit that the rewrite to using the proposed arrays is a lot easier than rewriting to use the explicit length+pointer.
- Gibbon1 3y ago> I don't really understand the resistance to it ISO and WG14 are European organizations and thus culturally act like guilds. And you're not in it that's why. My suggestion for improving C is add first class types and boxing and unboxing to phat pointers.
- ljosifov 3y agoIndeed, you don't understand. Your proposal is excellent and it would in time heal a major C pain point. However - and I'm sorry to say - your politics game is non-existent, I'm afraid. Not to be super cynical, but: the politically astute way to go about it would've been to 1) not make your proposal public to all; but 2) implant the idea in the heads of few influential committee members; then 3) wait for them to "come up with a new and excellent idea" to you, at which point you'd exclaim "what a marvelous idea". It's not even this is some exception on part of the committee members - the behavior of the members is but the norm in humans groups. And we see this too in the replies to your comment in this thread and past comments on this subject (e.g. https://news.ycombinator.com/item?id=17585357 https://news.ycombinator.com/item?id=17585357) - random people will bend backwards, not understand, misunderstand, make fools of themselves, come up with most ridiculous excuses for reasons, anything but acknowledge someone else's good idea. Maybe it's part and parcel of thoroughly thrashing an idea to see if it stands. But maybe gone too far in this case. On a more cheerful note - I don't know how and when, I missed the event, and it may well be that it happened by accident - but C got the ability to access N-dim (N>1) array elements with dimensions known at run time, not only at compile time - learned about it in https://jorengarenar.github.io/blog/vla-usecases https://jorengarenar.github.io/blog/vla-usecases : int n = ..., m = ...; float (*arr)[n][m] = malloc(sizeof *arr); // `n` and `m` are tun-time variables with dimensions if (arr) { // (*arr)[i][j] = ...; free(arr); }
- WalterBright 3y agoYou're right, I'm not very political. A similar thing happened when I submitted a formal proposal for `static if` (another popular D feature). It was vehemently rejected. After some time passed, another person submitted more or less the same thing, and it was accepted into the Standard. It doesn't matter to me if C accepts the proposal to fix array overflows or not, as I don't write significant code in C any more (all my existing C code I still use was converted to D).
- ljosifov 3y agoAh - so not a one off. Socially - person A pushes for change through various stages, spending time, effort and own personal social credit, persuading very opinionated people along the way to support the change. If successful, then person B, the originator of the idea, gets most of the glory for the change. Person A would need to be more self-less, ego-less, willing to take pain for no personal gain, than average - so not very likely to happen.