5 ms·

What I don't understand is why some programmers invest so much energy in using realloc. When you know that for most cases realloc actually does a new alloc and

by acqq 11y ago

What I don't understand is why some programmers invest so much energy in using realloc. When you know that for most cases realloc actually does a new alloc and copy of the existing content, but has the corners of undefined or unwanted behavior, why not doing it in your own function in a way that you fully (for your needs) control?

klodolph 11y ago
You certainly could wrap realloc(), but besides the bizarre/unnecessary behavior with realloc(ptr, 0), it behaves exactly as you would expect: on failure it returns NULL and doesn't touch your pointer, on success it returns the new pointer. It doesn't seem like it requires any more "energy" than using malloc() and programming in C to begin with.
acconsta 11y ago
For large allocations, realloc can remap virtual memory instead of doing a naive copy: http://blog.httrack.com/blog/2014/04/05/a-story-of-realloc-and-laziness/ http://blog.httrack.com/blog/2014/04/05/a-story-of-realloc-a...
- acqq 11y ago
  That's exactly when I'd use realloc. Looking who wrote the text, I also respect cperciva and believe he must have some good reasons and I'd be glad read which use cases he had, more than just "what's wrong with different reallocs." Because I'm not surprised that the corner cases aren't to everybody's (or even anybody's) satisfaction. It's C. Less is more and all that.
- cperciva 11y ago
  And for small allocations, realloc can expand or shrink an allocation in-place, because it knows where it is placing other memory allocations.
  - acqq 11y ago
    Yes. If the allocation block is 16 bytes for example, the string growing from 10 to 11, 12, 13 etc could still be on the same place. But in which use cases are frequent reallocs actually needed, so much that you can recognize the performance impact? I'd really like to know, as I personally never had such problems. When the single allocations were too expensive I've just used some kind of memory pool. For small stuff realloc is still more expensive than just a few instructions on average when some pool is used.
    - cperciva 11y ago
      The classic example of frequent reallocs is this perl code: $x .= $_ while (<>); I believe this has been fixed now, but perl used to realloc for each append operation, which resulted in O(N^2) time complexity if realloc didn't operate in-place.
      - acqq 11y ago
        But I wouldn't expect of you to assume that a "every time realloc" should be an optimal and at the same time a portable solution in such a case?
        cperciva 11y ago
        No, I don't. If you look at the code in question, every time I expand an allocation I at least double it.
        acqq 11y ago
        I'd expect that such growth seldom allows the realloc to remain in-place (unless it's the last thing allocated before and already in a big preallocated chunk of the allocator)? Have you observed what you then get in your program? Which allocator is used underneath?
        cperciva 11y ago
theseoafs 11y ago
Because it's nice to be able to use standard library C functions. It's so rare that you can actually use something in the stdlib rather than in an external library, and people are generally eager not to rewrite functionality which ostensibly should be provided by the standard library.
deleted 11y ago
[deleted]
cbsmith 11y ago
Well, it shouldn't have corners of undefined or unwanted behaviour. That's kind of the point. Obviously though, there is a huge advantage to having a single layer of heap management and letting the heap management algorithm have the best insight in to how memory is being used and needed. Rolling your own realloc on top of the heap manager is as likely to create new inefficiencies as remove them.
- acqq 11y ago
  You don't have to "roll your own" for the cases that are always defined. That means, when NULLs and 0 aren't involved, realloc works, call it from your "myrealloc" function. For 0 and NULL, make sure that your "myrealloc" function handles them exactly as you'd like. Then call your function everywhere in your code. I believed it's "Programming 101." > it shouldn't have corners It's not how C traditionally worked. Almost every function was knowingly made to be not "too good." I see modern programmers expect something else, what even 100 times slower languages not always give. Special cases have different possible treatments, and as soon as the exist nobody can make something that would please everyone.
  - cbsmith 11y ago
    > You don't have to "roll your own" for the cases that are always defined. That means, when NULLs and 0 aren't involved, realloc works, call it from your "myrealloc" function. For 0 and NULL, make sure that your "myrealloc" function handles them exactly as you'd like. Ah, I misunderstood what you meant by "doing it in your own function". Yes, wrapping the semantics will work, though doing it right and without imposing some overhead takes a bit of work. > It's not how C traditionally worked. Almost every function was knowingly made to be not "too good." I see modern programmers expect something else, what even 100 times slower languages not always give. Special cases have different possible treatments, and as soon as the exist nobody can make something that would please everyone. As the original post pointed out, traditionally it didn't work like this. This is how it has evolved. I did use poor semantics though, as "undefined behaviour" has a specific meaning in C that is of course how the language was defined. What I meant was "ambiguous or unwanted behaviour". Having a clear behaviour for a particular case, even if it is just returning an error code, is fine by me.

by acqq 11y ago

klodolph 11y ago

You certainly could wrap realloc(), but besides the bizarre/unnecessary behavior with realloc(ptr, 0), it behaves exactly as you would expect: on failure it returns NULL and doesn't touch your pointer, on success it returns the new pointer. It doesn't seem like it requires any more "energy" than using malloc() and programming in C to begin with.

acconsta 11y ago

For large allocations, realloc can remap virtual memory instead of doing a naive copy: http://blog.httrack.com/blog/2014/04/05/a-story-of-realloc-and-laziness/ http://blog.httrack.com/blog/2014/04/05/a-story-of-realloc-a...

acqq 11y ago
That's exactly when I'd use realloc. Looking who wrote the text, I also respect cperciva and believe he must have some good reasons and I'd be glad read which use cases he had, more than just "what's wrong with different reallocs." Because I'm not surprised that the corner cases aren't to everybody's (or even anybody's) satisfaction. It's C. Less is more and all that.
cperciva 11y ago
And for small allocations, realloc can expand or shrink an allocation in-place, because it knows where it is placing other memory allocations.
- acqq 11y ago
  Yes. If the allocation block is 16 bytes for example, the string growing from 10 to 11, 12, 13 etc could still be on the same place. But in which use cases are frequent reallocs actually needed, so much that you can recognize the performance impact? I'd really like to know, as I personally never had such problems. When the single allocations were too expensive I've just used some kind of memory pool. For small stuff realloc is still more expensive than just a few instructions on average when some pool is used.
  - cperciva 11y ago
    The classic example of frequent reallocs is this perl code: $x .= $_ while (<>); I believe this has been fixed now, but perl used to realloc for each append operation, which resulted in O(N^2) time complexity if realloc didn't operate in-place.
    - acqq 11y ago
      But I wouldn't expect of you to assume that a "every time realloc" should be an optimal and at the same time a portable solution in such a case?
      - cperciva 11y ago
        No, I don't. If you look at the code in question, every time I expand an allocation I at least double it.
        acqq 11y ago
        I'd expect that such growth seldom allows the realloc to remain in-place (unless it's the last thing allocated before and already in a big preallocated chunk of the allocator)? Have you observed what you then get in your program? Which allocator is used underneath?
        cperciva 11y ago

theseoafs 11y ago

Because it's nice to be able to use standard library C functions. It's so rare that you can actually use something in the stdlib rather than in an external library, and people are generally eager not to rewrite functionality which ostensibly should be provided by the standard library.

deleted 11y ago

[deleted]

cbsmith 11y ago

Well, it shouldn't have corners of undefined or unwanted behaviour. That's kind of the point. Obviously though, there is a huge advantage to having a single layer of heap management and letting the heap management algorithm have the best insight in to how memory is being used and needed. Rolling your own realloc on top of the heap manager is as likely to create new inefficiencies as remove them.

acqq 11y ago
You don't have to "roll your own" for the cases that are always defined. That means, when NULLs and 0 aren't involved, realloc works, call it from your "myrealloc" function. For 0 and NULL, make sure that your "myrealloc" function handles them exactly as you'd like. Then call your function everywhere in your code. I believed it's "Programming 101." > it shouldn't have corners It's not how C traditionally worked. Almost every function was knowingly made to be not "too good." I see modern programmers expect something else, what even 100 times slower languages not always give. Special cases have different possible treatments, and as soon as the exist nobody can make something that would please everyone.
- cbsmith 11y ago
  > You don't have to "roll your own" for the cases that are always defined. That means, when NULLs and 0 aren't involved, realloc works, call it from your "myrealloc" function. For 0 and NULL, make sure that your "myrealloc" function handles them exactly as you'd like. Ah, I misunderstood what you meant by "doing it in your own function". Yes, wrapping the semantics will work, though doing it right and without imposing some overhead takes a bit of work. > It's not how C traditionally worked. Almost every function was knowingly made to be not "too good." I see modern programmers expect something else, what even 100 times slower languages not always give. Special cases have different possible treatments, and as soon as the exist nobody can make something that would please everyone. As the original post pointed out, traditionally it didn't work like this. This is how it has evolved. I did use poor semantics though, as "undefined behaviour" has a specific meaning in C that is of course how the language was defined. What I meant was "ambiguous or unwanted behaviour". Having a clear behaviour for a particular case, even if it is just returning an error code, is fine by me.