2 ms·
I've been writing a toy `wordcount` recently, and it seems like if I wanted to support inputs much larger than the ~5GB file I'm testing against, or inputs that
by anonymoushn 4y ago
I've been writing a toy `wordcount` recently, and it seems like if I wanted to support inputs much larger than the ~5GB file I'm testing against, or inputs that contain a lot more unique strings per input file size, I would need to realloc, but I would not need to free.
- woodruffw 4y agoIs that `wordcount` in Zig? My understanding (which could be wrong) is that reallocation in Zig would leave the old buffer "alive" (from the allocator's perspective) if it couldn't be expanded, meaning that you'd eventually OOM if a large enough contiguous region couldn't be found.
- anonymoushn 4y agoIt's in zig but I just call mmap twice at startup to get one slab of memory for the whole file plus all the space I'll need. I am not sure whether Zig's GeneralPurposeAllocator or PageAllocator currently use mremap or not, but I do know that when realloc is not implemented by a particular allocator, the Allocator interface provides it as alloc + memcpy + free. So I think I would not OOM. In safe builds when using GeneralPurposeAllocator, it might be possible to exhaust the address space by repeatedly allocating and freeing memory, but I wouldn't expect to run into this on accident.
- woodruffw 4y agoThat's interesting, thanks for the explanation!
- dundarious 4y agoThey don't (at least the GPA's defaulting backing allocator is the page_allocator, which doesn't). https://github.com/ziglang/zig/blob/master/lib/std/heap.zig https://github.com/ziglang/zig/blob/master/lib/std/heap.zig