5 ms·

How do you avoid the need for dynamically growing arrays? For instance, if you are doing any sort of IO, you often don’t know the size of the input beforehand.

by slaymaker1907 8y ago

How do you avoid the need for dynamically growing arrays? For instance, if you are doing any sort of IO, you often don’t know the size of the input beforehand. That either forces you to use malloc or do some strange low-level manipulations of the stack.

jandrese 8y ago
Or you read a fixed block at a time. This gets awkward if you need to stitch together the end of the previous block with the new one but it does have the advantage of being fast, especially if you fix the block size to the block size of the underlying filesystem (or a multiple).
- s_m_t 8y ago
  +1, maybe it is because I'm not programming the most complicated things in the world, but almost all of my data structures are fixed size arrays (or are stored in them). In my experience, dynamic memory management simply isn't needed unless you are doing something very very complicated.
- Too 8y ago
  Hehe, the lost art of not having everything in RAM at the same time. Async APIs are getting better and better all the time but it seems like streaming APIs hasn't gotten the same love, even though it should be easier than ever now as it's just an application of async primitives.
iforgotpassword 8y ago
Yeah, use the heap: realloc() is your friend. Bit often times with io you just work in chunks, e.g. read up to X KB from a socket and then write it to file.
jstimpfle 8y ago
Something like this is all you'll need: https://gist.github.com/jstimpfle/562b2c3e9fe537e378351bb9d5be8cdb https://gist.github.com/jstimpfle/562b2c3e9fe537e378351bb9d5... Explanation: A little macro that adds the correct element size. Of course, you can opt to add it by hand as well, or put it in the run-time data structures and only set it once at initilization time. I personally think these approaches are error-prone / ugly, though. There is also the stretchy-buffer approach from Sean Barret, where you don't need to explicitly allocate bookkeeping data (capacity value in my example). That one puts the bookkeeping data and count in front of the actual data. I prefer the explicit approach, though, and I MUCH prefer having an explicit count for growing vector-like things. Because I hate when I have parallel arrays and each one has its own count. Having a shared count variable for parallel arrays is much clearer.
- doty 8y ago
  BTW that code has a security vulnerability in it: the size computation can overflow and you can end up shrinking your buffer, and you will end up smashing your heap, and then bad things.
  - jstimpfle 8y ago
    I know. I don't care. Partly because it's only an example and the requirements are not spec'ed out. Partly because you have potential overflows basically everywhere, if you disregard the context. Let's just say the caller is assumed to make sure there will be no overflow. (Or just add a no-overflow assertion - I don't care).
    - deleted 8y ago
      [deleted]

jandrese 8y ago

Or you read a fixed block at a time. This gets awkward if you need to stitch together the end of the previous block with the new one but it does have the advantage of being fast, especially if you fix the block size to the block size of the underlying filesystem (or a multiple).

s_m_t 8y ago
+1, maybe it is because I'm not programming the most complicated things in the world, but almost all of my data structures are fixed size arrays (or are stored in them). In my experience, dynamic memory management simply isn't needed unless you are doing something very very complicated.
Too 8y ago
Hehe, the lost art of not having everything in RAM at the same time. Async APIs are getting better and better all the time but it seems like streaming APIs hasn't gotten the same love, even though it should be easier than ever now as it's just an application of async primitives.

iforgotpassword 8y ago

Yeah, use the heap: realloc() is your friend. Bit often times with io you just work in chunks, e.g. read up to X KB from a socket and then write it to file.

jstimpfle 8y ago

Something like this is all you'll need: https://gist.github.com/jstimpfle/562b2c3e9fe537e378351bb9d5be8cdb https://gist.github.com/jstimpfle/562b2c3e9fe537e378351bb9d5... Explanation: A little macro that adds the correct element size. Of course, you can opt to add it by hand as well, or put it in the run-time data structures and only set it once at initilization time. I personally think these approaches are error-prone / ugly, though. There is also the stretchy-buffer approach from Sean Barret, where you don't need to explicitly allocate bookkeeping data (capacity value in my example). That one puts the bookkeeping data and count in front of the actual data. I prefer the explicit approach, though, and I MUCH prefer having an explicit count for growing vector-like things. Because I hate when I have parallel arrays and each one has its own count. Having a shared count variable for parallel arrays is much clearer.

doty 8y ago
BTW that code has a security vulnerability in it: the size computation can overflow and you can end up shrinking your buffer, and you will end up smashing your heap, and then bad things.
- jstimpfle 8y ago
  I know. I don't care. Partly because it's only an example and the requirements are not spec'ed out. Partly because you have potential overflows basically everywhere, if you disregard the context. Let's just say the caller is assumed to make sure there will be no overflow. (Or just add a no-overflow assertion - I don't care).
  - deleted 8y ago
    [deleted]