3 ms·

> That's why I said you need to control your allocator. The allocator does not matter unless by "control your allocator" you mean "the OS provides an API which

> That's why I said you need to control your allocator. The allocator does not matter unless by "control your allocator" you mean "the OS provides an API which can restrict the virtual memory system", which has nothign to do with the allocator. If you're using 19 bits for tagging and the OS returns a 52 bit pointer, you don't have the space for your tagging scheme, no matter what your allocator is. > PAE doesn't meaningfully extend how much memory a 32 bit processes can have. Of course not. What it does is show that there is no correlation between the physical and virtual address spaces, so you can have a 64 bits physical address space and only provide a 48 bits virtual address space to applications. > And LAA is based on software assumptions that were never particularly tied to hardware, and it can be relevant with or without PAE. Missing the point, the example of LAA shows that the OS can provide different virtual address spaces to different processes. So the OS can provide 48 bit virtual address spaces by default, and a full 64 bits virtual address space to applications which request it. The former can then keep using tagged pointers just fine.

Dylan16807 3y ago
> The allocator does not matter The allocator gets address space. The OS doesn't just shove things into your process in general. It can't return a 52 bit pointer out of nowhere. My first draft mentioned OS explicitly but it's not really the OS that's making the decisions. > What it does is show that there is no correlation between the physical and virtual address spaces It does show that, but that decorrelation is neither necessary nor sufficient to solve this problem. > Missing the point, the example of LAA shows that the OS can provide different virtual address spaces to different processes. I agree with that. In the context of your overall post it wasn't clear to me that that was why you mentioned it.
- masklinn 3y ago
  > The allocator gets address space. The allocator gets address space from the OS. > The OS doesn't just shove things into your process in general. It can't return a 52 bit pointer out of nowhere. My first draft mentioned OS explicitly but it's not really the OS that's making the decisions. Of course it’s the OS making the decision. You ask the OS for memory and it returns whatever it wants. The allocators acts as bridge between the application and OS and can do cool stuff but if the OS returns a 52 bits pointer there isn’t a thing the allocator can do about it that would result in a working 48 bits pointer. > It does show that, but that decorrelation is neither necessary nor sufficient to solve this problem. It absolutely is necessary, since the entire subject is to not break applications requiring a smaller address space for their tagging scheme to work even as the system and other applications migrate to larger ones.
  - Dylan16807 3y ago
    The allocator can choose which part of the address space to fill. It does not have to deal with "whatever the OS wants". > It absolutely is necessary, since the entire subject is to not break applications requiring a smaller address space for their tagging scheme to work even as the system and other applications migrate to larger ones. Even if you have a 1:1 memory mapping, you can leave the applications that want small addresses at the start of memory. You don't need to decorrelate virtual and physical addresses to do that.
    - masklinn 3y ago
      > The allocator can choose which part of the address space to fill. No it can not. The allocator can request which part of the address space to fill, and the OS is completely free to ignore that request. > It does not have to deal with "whatever the OS wants". It absolutely does, the OS is ultimately who controls both the physical and virtual address spaces. > Even if you have a 1:1 memory mapping, you can leave the applications that want small addresses at the start of memory. Up until it’s all taken and then you can’t do that anymore.
      - Dylan16807 3y ago
        > the OS is completely free to ignore that request I'm assuming an OS that isn't refusing to do things for no reason. Aren't you? If we're accounting for capricious refusal of program allocation requests, then 64-bit clean programs can't safely run either. > Up until it’s all taken Which is no worse than it was before you upgraded your RAM. Not that anyone is running big programs on systems without MMUs.
        masklinn 3y ago
        > I'm assuming an OS that isn't refusing to do things for no reason. > Aren't you? You are apparently assuming the OS would have no reason to refuse. I've seen DeRaadt cook, I'd rather not assume anything that's not guaranteed. And I especially wouldn't make up new modes of operations which don't exist for no justifiable reason.

Dylan16807 3y ago

> The allocator does not matter The allocator gets address space. The OS doesn't just shove things into your process in general. It can't return a 52 bit pointer out of nowhere. My first draft mentioned OS explicitly but it's not really the OS that's making the decisions. > What it does is show that there is no correlation between the physical and virtual address spaces It does show that, but that decorrelation is neither necessary nor sufficient to solve this problem. > Missing the point, the example of LAA shows that the OS can provide different virtual address spaces to different processes. I agree with that. In the context of your overall post it wasn't clear to me that that was why you mentioned it.

masklinn 3y ago
> The allocator gets address space. The allocator gets address space from the OS. > The OS doesn't just shove things into your process in general. It can't return a 52 bit pointer out of nowhere. My first draft mentioned OS explicitly but it's not really the OS that's making the decisions. Of course it’s the OS making the decision. You ask the OS for memory and it returns whatever it wants. The allocators acts as bridge between the application and OS and can do cool stuff but if the OS returns a 52 bits pointer there isn’t a thing the allocator can do about it that would result in a working 48 bits pointer. > It does show that, but that decorrelation is neither necessary nor sufficient to solve this problem. It absolutely is necessary, since the entire subject is to not break applications requiring a smaller address space for their tagging scheme to work even as the system and other applications migrate to larger ones.
- Dylan16807 3y ago
  The allocator can choose which part of the address space to fill. It does not have to deal with "whatever the OS wants". > It absolutely is necessary, since the entire subject is to not break applications requiring a smaller address space for their tagging scheme to work even as the system and other applications migrate to larger ones. Even if you have a 1:1 memory mapping, you can leave the applications that want small addresses at the start of memory. You don't need to decorrelate virtual and physical addresses to do that.
  - masklinn 3y ago
    > The allocator can choose which part of the address space to fill. No it can not. The allocator can request which part of the address space to fill, and the OS is completely free to ignore that request. > It does not have to deal with "whatever the OS wants". It absolutely does, the OS is ultimately who controls both the physical and virtual address spaces. > Even if you have a 1:1 memory mapping, you can leave the applications that want small addresses at the start of memory. Up until it’s all taken and then you can’t do that anymore.
    - Dylan16807 3y ago
      > the OS is completely free to ignore that request I'm assuming an OS that isn't refusing to do things for no reason. Aren't you? If we're accounting for capricious refusal of program allocation requests, then 64-bit clean programs can't safely run either. > Up until it’s all taken Which is no worse than it was before you upgraded your RAM. Not that anyone is running big programs on systems without MMUs.
      - masklinn 3y ago
        > I'm assuming an OS that isn't refusing to do things for no reason. > Aren't you? You are apparently assuming the OS would have no reason to refuse. I've seen DeRaadt cook, I'd rather not assume anything that's not guaranteed. And I especially wouldn't make up new modes of operations which don't exist for no justifiable reason.