13 ms·
Reallocarray() in OpenBSD: Integer Overflow Detection for Free
- SEJeff 12y agoI wonder if GNU libc will ever get something like this.
- vyqfi 12y agoI don't get the fuzz. This is like the 3rd or 4th submission about this function. And all it does is testing that > ((SIZE_MAX / nmemb) >= size) before doing the size_t multiplication. That's no rocket science. Anyone who's worried about his size_ts overflowing should be able to come up with this.
- quotemstr 12y agoI don't understand why more people don't define local helper functions. I've worked on FOSS codebases with hundreds of invocations of strncpy, each carefully followed with an additional NUL-termination check to work around strncpy's well-known problem. Why wouldn't you just write a little wrapper?
- PythonicAlpha 12y agoIn my opinion the C library was one of C's most best innovations. But there are also some dark spots. strncpy is one of it -- and one of the things, where the C library is gravely outdated today.
- drv 12y agoProgrammers just use the wrong tool for the job - arguably because strncpy is poorly named. strncpy is intended for filling zero-padded fixed-size strings, not copying normal single-zero-terminated strings. I think the main benefit of adding functions like reallocarray is that they make the right tool for the job available by default. Plain realloc for array resizing is a timebomb in the same way as strncpy for normal strings - it will work (by accident) until it doesn't, and when it fails, it's likely to be an exploitable security hole.
- tribaal 12y agoThat's very neat, and also a surprisingly concise bit of code!
- dezgeg 12y agoUnfortunately, having this kind of function doesn't fix the real problem, which is those certain kinds of programmers who won't let a single machine cycle be wasted. Here's just a completely random example from the Linux kernel: https://lkml.org/lkml/2014/8/19/194 https://lkml.org/lkml/2014/8/19/194 > You are implying that such an overflow could arise in this code. > This is false. [...] > bNumEndpoints is an unsigned 8-bit integer. So instead of changing the code code to be obviously correct (wrt. multiplication overflow), we'll instead keep the old code whose correctness becomes clear only after the programmer has looked up the types of variables involved. This mindset is exactly why vulnerabilities in C code are still common. It really needs to stop.
- btown 12y agoWon't the wrapper be inlined and the condition branch-predicted? In that case, you literally aren't letting a single machine cycle be wasted.
- gizmo686 12y agoWell, you might waste a cycle on the first run (or any-run after the branch has been flushed from the branch prediction table), as well as wasting cycles elsewhere from clobbering the table. One solution to both of these problems I have thought about, but do not work at a low enough level to know if it is already a thing, is an opcode for unlikely branches. Sementiclly, this would behave like a normal jmp instruction, but the CPU branch predictor would always assume that the jump is not taken, and not bother keeping track of it. In this particular case, it would also involve a way of signalling to the compiler (or the compiler inferring), but for loops it could be usefull. Of course, in this case it is also possible that the compiler can infer that the branch can never be taken and remove it, so you really do not loose anything.
- dezgeg 12y ago> One solution to both of these problems I have thought about, but do not work at a low enough level to know if it is already a thing, is an opcode for unlikely branches. x86 actually does have jump instructions with branch prediction hints, but surprisingly modern CPUs will completely ignore those hints. Perhaps they never really caused large enough performance improvements to be worthwhile, or something. > In this particular case, it would also involve a way of signalling to the compiler (or the compiler inferring), [...] This functionality does exist, and Linux does make use of them. See the macros likely() and unlikely() at http://lxr.free-electrons.com/source/include/linux/compiler.h#L131 http://lxr.free-electrons.com/source/include/linux/compiler....
- psgbg 12y agoThis is an amazing post. So, if this is available in libbsd it means that is also available for linux too, I'm right?
- PythonicAlpha 12y agoSure a good thing, but for me also a good example, why I changed from C/C++ to a scripting language. In application development, you seldom have the need for maximum speed in all your coding. And always worrying about such things as what is addressed by this small enhancement, is a pain. In most scripting languages, such details are just falling away. I still program in C/C++ -- but just for very speed-intensive "hot-spot" functions, in form of Python enhancement modules. And in those, I have the time and patience to look for all these implementation details. All the rest, I do in Python.
- bstamour 12y agoYou won't need this enhancement in C++ if you just use `std::vector` or `std::array`.
- PythonicAlpha 12y agoI was talking about Python enhancement modules -- and no, I won't consider C++ a good replacement for Python.
- bstamour 12y ago> In application development, you seldom have the need for maximum speed in all your coding. And always worrying about such things as what is addressed by this small enhancement, is a pain. That's what my comment was addressing: In C++ with vector, map, etc, you don't need to remember to safely realloc manually, because the containers do it for you automatically.
- PythonicAlpha 12y agoMight be, but in C++ there are still enough problems lingering, that using Python is worthwhile. Many workarounds have been forged in C++, but still there are so many problems lingering and to use C++ right, you really must be a master of a language that is 3 times as difficult to learn than most other programming languages.
- 12y ago
- alexlarsson 12y agoOr you could use glib which has this exact function (g_realloc_n) since a long time.
- tedunangst 12y agoI think you mean g_try_realloc_n.
- alexlarsson 12y agoTrue
- Demiurge 12y agoI have only had to use C for a small amount of work over the years, so I have a naive question. This seems like something very essential, why is it that this function took more than 30 years to add?
- tedunangst 12y agoFor a long time the prevailing attitude was "don't make mistakes." Then after it became apparent some problems were both common and serious, people argued (and continue to do so) that if a function doesn't solve every problem, it's not worth adding. Whataboutism.
- foxhill 12y agoam i the only one that thinks this is a total non-issue? if you're going to overflow a size_t, you're already in trouble. realloc is used for.. um, reallocating data. it's probably grown many times before you come close to exhausting 32 bits of size (assuming i386). if you are coming close to 4gb allocations, you're already likely running out memory. as such, you're program is probably going to be killed for using too much memory before you are in any risk of overflowing size_t. similarly, in a 64bit runtime.. well, you will certainly be killed by the kernel before you come close to overflowing size_t. if you're reallocing based on untrusted user input.. well, again, you're already in trouble. "fixing" realloc will only make problems occur earlier (which i suppose is good). EDIT: what i really should have said, is "make problems occur somewhere else". finally, returning NULL for failure.. well, malloc doesn't fail. most (sane) codes don't check the return value of malloc because it doesn't make sense to. the kernel doesn't allocate memory until first access, at which point you're going to have trouble regardless. colour me sceptical, but this seems like a fix to a non-existent problem.
- dezgeg 12y agoDefence in depth. This single overflow check could very well be the difference between denial-of-service-via-OOM and heap overflow. Should we then abandon ASLR or PIE or other defensive measures as well, just because some code is "already in trouble" due to a bad strcpy or something similar?
- foxhill 12y ago> This single overflow check could very well be the difference between denial-of-service-via-OOM and heap overflow OOM in general cannot be used for remote code execution. heap overflow can. > Should we then abandon ASLR or PIE or other defensive measures as well, just because some code is "already in trouble" due to a bad strcpy or something similar certainly not, ASLR is so general that it doesn't make sense to not do it. more importantly, they're entirely transparent to the programmer, and can be used on existing codes.
- dezgeg 12y ago
- kazinator 12y agoThe multiplication factor is often some small constant, like 2 in the case of an array grown by doubling. It's worth making reallocarray an inline function to handle a few constant values: inline void *reallocarray(void *old, size_t x, size_t y) { switch (y) { case 1: return realloc(old, x); ... case 2: // drastically simplified overflow check ... default: // perhaps a switch (x) with mirror cases here return realloc_array_general(old, x, y); } } The 1 case occurs when programmers are following some coding convention whereby malloc calls are replaced with this "single allocator function to bind them all": char *buf = reallocarray(size, 1); When the function is inlined, the non-matching cases turn to dead code since the switch expression is constant.
- fdej 12y agoThis would slow down the general case where the size is variable, but you could combine it with __builtin_constant_p in GCC.
- kazinator 12y agoI just looked at some of my recent code where I'm using realloc. It's obvious that this approach is insufficient, because it assumes there is only one multiplication. I have situations where an array is being resized. The size is expressed in terms of elements, rather than bytes. So two multiplications are going on: the size is increased first, and then in the realloc call, the argument expression has a multiplication by the element size also. An overflow checking realloc won't help if the first multiplication oveflows. If I make this version instead: void *realloc_array(void *old, size_t existing_size, size_t resize_factor, size_t elsize); that's no good either because I want the new size also if everything went well. I need something like: void *realloc_array(void *old, size_t existing_size, size_t resize_factor, size_t elsize, size_t *p_newsize_out); Okay, now we're talking. Resize my vector old, which contains existing_size elements, by a factor of resize_factor. The individual elements have size elsize. If all goes well (non-NULL is returned), give me the new size (in elements, not bytes) in the location *p_newsize_out.
- kazinator 12y agoIssue: the function should be treating nmemb == 0 or size == 0 as errors rather than a feature. The way it is, you can do /* obfuscated way of doing free(ptr); */ realloc_array(ptr, 0, whatever); There is no need to carry on the tradition that realloc serves as a complete memory allocator in a single function. The program could be pass nmemb or size as zero due to a bug, rather than the intent to free. Or as a compromise, zero could be accepted for nmemb (zero sized array), but not for size (the base elements must have nonzero sizes).