2 ms·

Yes, I mentioned the same issue a few days ago -- arenas break ASAN/Valgrind, and those are the BEST tools to ensure memory safety: https://news.ycombinator.co

by chubot 3y ago

Yes, I mentioned the same issue a few days ago -- arenas break ASAN/Valgrind, and those are the BEST tools to ensure memory safety: https://news.ycombinator.com/item?id=36427264 https://news.ycombinator.com/item?id=36427264 In response to a claim about arena-based allocators and memory safety

kazinator 3y ago
Maybe Valgrind has an API now to attach a backtrace to a custom allocation out of an arena. I haven't touched that in years. I did the initial Valgrind integration in 2009 and just maintained it with changes to the gc. I see this in the current manual: VALGRIND_MALLOCLIKE_BLOCK: If your program manages its own memory instead of using the standard malloc / new / new[], tools that track information about heap blocks will not do nearly as good a job. For example, Memcheck won't detect nearly as many errors, and the error messages won't be as informative. To improve this situation, use this macro just after your custom allocator allocates some new memory. See the comments in valgrind.h for information on how to use it. I can't remember whether I saw this before; i.e. is this something existing that had not been useful, or is it something new. I will look into it. It does look like it should be attacking the same problem: that the errors aren't detected well or nearly as informative in blocks carved out of larger block by custom allocators.
- kazinator 3y ago
  Alas, I've not been able to get it working. Possibly, the issue is that in my heaps, I do not have red zones; things are tightly packed. That's okay because the heap cells do not store arrays. I'm not concerned about overruns whatsoever. When I go through the hoops of registering each heap as a memory pool, which supports malloc-like allocations doled out of it, problems happen. When cell [0] of a heap becomes garbage and is indicated as freed (reclaimed by GC), Valgrind throws an invalid free/delete/delete[] error, saying there is a conflict between that and the entire block. I think this is because there is no offset between the block and the start of the heap. Valgrind assumes there will be some header or red zone between the entire memory zone and the pool area. It seems to be saying, whoa, you can't do VALGRIND_FREELIKE_BLOCK at that address; you allocated that address from malloc. Well no shit, and I told you that block was a heap, from which I'm getting VALGRIND_MALLOCLIKE_BLOCK blocks. Secondly, after the first GC cycles, errors go haywire, like "invalid read of 2 bytes: 0 bytes after 16 byte block allocated at ... blah blah". Basically, it thinks that various accesses are overruns, possibly due to there being no red zones between the blocks. Its quite baffling. Maybe current Valgrind has fixes for some of this; I just use what distros package.

by chubot 3y ago

kazinator 3y ago

Maybe Valgrind has an API now to attach a backtrace to a custom allocation out of an arena. I haven't touched that in years. I did the initial Valgrind integration in 2009 and just maintained it with changes to the gc. I see this in the current manual: VALGRIND_MALLOCLIKE_BLOCK: If your program manages its own memory instead of using the standard malloc / new / new[], tools that track information about heap blocks will not do nearly as good a job. For example, Memcheck won't detect nearly as many errors, and the error messages won't be as informative. To improve this situation, use this macro just after your custom allocator allocates some new memory. See the comments in valgrind.h for information on how to use it. I can't remember whether I saw this before; i.e. is this something existing that had not been useful, or is it something new. I will look into it. It does look like it should be attacking the same problem: that the errors aren't detected well or nearly as informative in blocks carved out of larger block by custom allocators.

kazinator 3y ago
Alas, I've not been able to get it working. Possibly, the issue is that in my heaps, I do not have red zones; things are tightly packed. That's okay because the heap cells do not store arrays. I'm not concerned about overruns whatsoever. When I go through the hoops of registering each heap as a memory pool, which supports malloc-like allocations doled out of it, problems happen. When cell [0] of a heap becomes garbage and is indicated as freed (reclaimed by GC), Valgrind throws an invalid free/delete/delete[] error, saying there is a conflict between that and the entire block. I think this is because there is no offset between the block and the start of the heap. Valgrind assumes there will be some header or red zone between the entire memory zone and the pool area. It seems to be saying, whoa, you can't do VALGRIND_FREELIKE_BLOCK at that address; you allocated that address from malloc. Well no shit, and I told you that block was a heap, from which I'm getting VALGRIND_MALLOCLIKE_BLOCK blocks. Secondly, after the first GC cycles, errors go haywire, like "invalid read of 2 bytes: 0 bytes after 16 byte block allocated at ... blah blah". Basically, it thinks that various accesses are overruns, possibly due to there being no red zones between the blocks. Its quite baffling. Maybe current Valgrind has fixes for some of this; I just use what distros package.