4 ms·

Oh boy, welcome to the exciting world of C pointer provenance :) What you just described is what compiler people call pointer provenance, where each pointer has

Oh boy, welcome to the exciting world of C pointer provenance :) What you just described is what compiler people call pointer provenance, where each pointer has, in addition to the address, a second piece of info attached to it that describes all the places to which the pointer can point. This is an extremely simplified and probably incorrect view of https://open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1434r0.html https://open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1434r0.... This is complicated because nobody agrees on what the correct behaviour should be, and we have mountains of legacy codebases that all rely on something slightly different when pointers get converted to ints and back.

gary_0 4y ago
That's what makes C programming fun: doing what the compiler/language people say you shouldn't be doing and seeing if it works anyways. At least, it works on my machine...
- pjmlp 4y ago
  Until the compiler gets upgraded....
  - adrianN 4y ago
    Until changes in seemingly far away code change the optimization decisions the compiler makes...
    - phkamp 4y ago
      That is actually the R1000's raison d'etre: Rational built a truly semantic IDE, and the result of that is that if you change a line of source-code somewhere in a huge complicated project, it will know precisely what the consequences of that change are. Ultimately, this allows it often just recompile that single line.
verdagon 4y ago
That sounds like the "generation" tag in Vale's generational references [0] or like how memory tagging works on ARM CPUs [1]. I love the approach, it's a way to get a lot more memory safety while not giving up a program's flexibility, especially in C's case. Some languages are opting to disallow pointer arithmetic and conversion between integers and pointers. We'll see how it works out! [0] https://verdagon.dev/blog/generational-references https://verdagon.dev/blog/generational-references [1] https://developer.arm.com/-/media/Arm%20Developer%20Community/PDF/Arm_Memory_Tagging_Extension_Whitepaper.pdf https://developer.arm.com/-/media/Arm%20Developer%20Communit...
raverbashing 4y ago
So instead of giving that metadata to the programmer the C language says F U and tells me only knowing where to put it is enough, right? "hey you allocated it you should know about it right?" right yeah
- simiones 4y ago
  To be clear, that metadata only (generally) exists in principle - it is not actually materialized and stored anywhere, even at compile time (definitely not at runtime). There may be cases where the compiler actually tracks it, but most often it is only tracked "best effort" - the compiler only needs to prove that two pointers can't be aliased based on any possible value of that metadata, not actually compute that metadata and use it.
anonymoushn 4y ago
The part of this that I most enjoy is that you cannot even obtain some memory from your own allocator implemented on top of mmap or on top of malloc, store ints in it, return it to the allocator, ask the allocator for memory again and receive the same memory back, and store doubles in it.
- gpderetta 4y ago
  Yes, notoriously you can't implement malloc in portable C. In C++, I think they added enough magic in that you should now be able to do it (placement new, std::launder and numerous other hacks).
  - teddyh 4y ago
    Reportedly you can implement malloc in C, using mmap().
    - gpderetta 4y ago
      well mmap is posix, not standard portable C.
  - rkangel 4y ago
    I've implemented a pools allocator for embedded devices a couple of times. Allocating memory for callers out of a block of static memory. I thought it was portable C. Which detail of the standard did I not realise I was running into?
    - gpderetta 4y ago
      I assume the block of static memory was a large static char array. My understanding that reusing the storage would violate the aliasing rules[1] and the rules against overlapping object lifetimes. [1] while char ptrs can be used to access everything, the reverse is not allowed.

gary_0 4y ago

That's what makes C programming fun: doing what the compiler/language people say you shouldn't be doing and seeing if it works anyways. At least, it works on my machine...

pjmlp 4y ago
Until the compiler gets upgraded....
- adrianN 4y ago
  Until changes in seemingly far away code change the optimization decisions the compiler makes...
  - phkamp 4y ago
    That is actually the R1000's raison d'etre: Rational built a truly semantic IDE, and the result of that is that if you change a line of source-code somewhere in a huge complicated project, it will know precisely what the consequences of that change are. Ultimately, this allows it often just recompile that single line.

verdagon 4y ago

That sounds like the "generation" tag in Vale's generational references [0] or like how memory tagging works on ARM CPUs [1]. I love the approach, it's a way to get a lot more memory safety while not giving up a program's flexibility, especially in C's case. Some languages are opting to disallow pointer arithmetic and conversion between integers and pointers. We'll see how it works out! [0] https://verdagon.dev/blog/generational-references https://verdagon.dev/blog/generational-references [1] https://developer.arm.com/-/media/Arm%20Developer%20Community/PDF/Arm_Memory_Tagging_Extension_Whitepaper.pdf https://developer.arm.com/-/media/Arm%20Developer%20Communit...

raverbashing 4y ago

So instead of giving that metadata to the programmer the C language says F U and tells me only knowing where to put it is enough, right? "hey you allocated it you should know about it right?" right yeah

simiones 4y ago
To be clear, that metadata only (generally) exists in principle - it is not actually materialized and stored anywhere, even at compile time (definitely not at runtime). There may be cases where the compiler actually tracks it, but most often it is only tracked "best effort" - the compiler only needs to prove that two pointers can't be aliased based on any possible value of that metadata, not actually compute that metadata and use it.

anonymoushn 4y ago

The part of this that I most enjoy is that you cannot even obtain some memory from your own allocator implemented on top of mmap or on top of malloc, store ints in it, return it to the allocator, ask the allocator for memory again and receive the same memory back, and store doubles in it.

gpderetta 4y ago
Yes, notoriously you can't implement malloc in portable C. In C++, I think they added enough magic in that you should now be able to do it (placement new, std::launder and numerous other hacks).
- teddyh 4y ago
  Reportedly you can implement malloc in C, using mmap().
  - gpderetta 4y ago
    well mmap is posix, not standard portable C.
- rkangel 4y ago
  I've implemented a pools allocator for embedded devices a couple of times. Allocating memory for callers out of a block of static memory. I thought it was portable C. Which detail of the standard did I not realise I was running into?
  - gpderetta 4y ago
    I assume the block of static memory was a large static char array. My understanding that reusing the storage would violate the aliasing rules[1] and the rules against overlapping object lifetimes. [1] while char ptrs can be used to access everything, the reverse is not allowed.