12 ms·
I'm not 100% sure how Zig allocators work but it looks like the arena memory is getting re-used without zeroing the memory? With slight memory corruption freed
by gizmo 2y ago
I'm not 100% sure how Zig allocators work but it looks like the arena memory is getting re-used without zeroing the memory? With slight memory corruption freed memory from a previous request can end up leaking. That's not great.
Even if you don't have process isolation between workers (which is generally what you want) then you can still put memory arenas far apart in virtual memory, make use of inaccessible guard pages, and take other precautions to prevent catastrophic memory corruption.
- eknkc 2y agoI guess you could place a zeroing allocator wrapper in between the arena and it's underlying allocator. That would write zero to anything that's getting freed. Arena deinit will free anything allocated from the underlying allocator so upon completion of each request, used memory would be zeroed before returned back to the main allocator. And that handler signature would still be the same. Which is the he whole point of this article so, yay.
- nurpax 2y agoThe same can happen with C malloc/free too.
- KerrAvon 2y agoThat’s not specific to Zig — local heap allocators generally don’t zero deallocated memory — that’s a significant, unnecessary performance hit. If you need data to be isolated when memory is corrupt, you need it to be isolated always.
- 10000truths 2y agomemset is the golden example of an easily pipelined, parallelized, predictable CPU operation - any semi-modern CPU couldn't ask for easier work to do. Zeroing 8 KB of memory is very cheap. If we use a modern Xeon chip as an example, an AVX2 store has a throughput of 2 instructions / cycle. Doing that 256 times for 8 KB totals 128 cycles, plus a few extra cycles to account for the latency of issuing the first instruction and the last store to the L1 cache. With a 2 GHz clock frequency, it still takes less than 70 nanoseconds. For comparison, an integer divide has a worst-case latency of 90ish cycles, or 45ish nanoseconds.
- celrod 2y agoThis memory is now the least recently used in the L1 cache, despite being freed by the allocator, meaning it probably isn't being used again. If it was freed after already being removed from the L1 cache, then you also need to evict other L1 cache contents and wait for it to be read into L1 so you can write to it. 128 cycles is a generous estimate, and ignores the costs to the rest of the program.
- 10000truths 2y agoThe worker is already reading/writing to the buffer memory to service each incoming HTTP request, whether the memory is zeroed or not. The side effects on the CPU cache are insubstantial.
- astrange 2y agoYou can use non-temporal writes to avoid this, and some CPUs have an instruction that zeroes a cache line. It's not expensive to do this.
- celrod 2y agoNontemporal writes are substantially slower, e.g. with avx512 you can do 1 64 byte nontemporal write every 5 or so clock cycles. That puts you at >= 640 cycles for 8 KiB. https://uops.info/html-instr/VMOVNTPS_M512_ZMM.html https://uops.info/html-instr/VMOVNTPS_M512_ZMM.html
- astrange 2y agoWell, the point of a non-temporal write kind of is that you don't care how fast it is. (Since if it was being read again anytime soon, you'd want it in the cache.) But yes, it can be an over-optimization.
- alexchamberlain 2y agoThis might be a stupid question, but why isn't zeroing 8KB of memory a single instruction? It must be so common as to be worthy that all the layers of memory (and indirection) to understand that.
- keybored 2y agoDeinit in O(1) seems to be a big attraction of arenas.
- foota 2y agoO(1) is nice, but I feel like avoiding walking a bunch of data structures is maybe most important.
- elvircrn 2y agoAny papers/blogs/SO answers covering this?
- saagarjha 2y agoWhat are you looking for? Bump allocators are quite simple, compared to typical allocators at least.
- foota 2y agoI don't have anything for you, but if you have some normally allocated hierarchal data structures in order to free them you'll have to go through their members, chase pointers, etc., to figure out the addresses to free, then call free on them in sequence. That's all going to be a lot more expensive than just memsetting a bunch of data to zero, which you can do at whatever the speed of your cores memory bandwidth is.
- josephg 2y agoYep. And you often don’t even need to zero the data. Generally, no paper or SO answer will tell you where your program spends its time. Learn to use profiling tools, and experiment with stuff like this. Try out arenas. Benchmark before and after and see what kind of real performance difference it makes in your own program.
- jedisct1 2y agoZig allocators can be composed, so adding zeroization would be trivial.
- samatman 2y agoI once spent an utterly baffling afternoon trying to figure out why my benchmark for a reverse iteration across a rope data structure in Julia was finishing way too fast. I was perf tuning it, and while it would have been lovely if my implementation was actually 50 times faster than reverse iterating a native String type, I didn't buy it. Finally figured it out: I flipped a sign in the reverse iterator, so it was allocating a bunch of memory and immediately hitting the margin of the Vector, and returning it with most of the bytes undefined. Why didn't I catch it sooner? Well, I kept running the benchmark, which allocated a reverse buffer for the String version, which GC released, then I ran the buggy code... and the GC picked up the recently freed correct data and handed it back to me! Oops. Of course, if you want to avoid that risk in Zig, you just write a ZeroOnFreeAllocator, which zeros out your memory when you free it. It's a drop in replacement for anything which needs an allocator, job done.
- tapirl 2y agoIf needed, you should zero memory on allocation succeeds, instead of zeroing it after it is freed.
- alexchamberlain 2y agoGenerally, you 0 on free in secure environments to avoid leaking secrets from 1 section of knowledge to the next. ie a request may contain a password, which the next request should not have access to.
- tapirl 2y agoGood reason. But I think it is not the responsibility of memory allocators to do the zero work. It is what the application code should do.
- alexchamberlain 2y agoDepends where you draw the line. An arena allocator per request needs to be managed at least by an app framework, if not the application. It's all layers of abstraction, and one of those layers needs to 0 memory.
- tapirl 2y agoThe arena allocator implementation for general uses absolutely should not do the zero work. This is specific use case, which can be implemented in an app-specific custom allocator.
- alexchamberlain 2y agoThat's not what I said. My point was that an arena allocator has to be managed at a relatively high level. Similarly, an allocator responsible for 0 on free would be managed at a similar level. They are orthogonal concepts as you say, but there's no reason 0 on free can't be managed by an allocator.
- saagarjha 2y agoGuard pages are not enough to prevent memory corruption across requests.
- hansvm 2y agoIn my Zig servers I'm using a similar arena-based (with resetting) strategy. It's not as bad as you'd imagine: The current alloc implementation memsets under the hood. There are ongoing discussions about the right way to remove that performance overhead, but safety comes first. Any sane implementation has an arena per request and per connection anyway, not shared between processes. You don't have bonkers aliasing bugs because the OS would have panicked before handing out that memory. Zig has a lot of small features designed to make memory corruption an unhappy code path. I've had one corruption bug out of a lot of Zig code the last few years. It was from a misunderstanding of async (a classic stack pointer leak disguised by a confusion over async syntax). It's not an issue since async is gone from the language, and that sort of thing is normally turned into a compiler error anyway as soon as somebody reports it.