8 ms·

It's unavoidable. APIs like mprotect() operate on pages; there's no way to hide that from them.

viraptor 1y ago
Also various performance characteristics graphs will have steps around the multiples of page sizes. People would find out anyway. (Although maybe the runtime detection is not a bad idea)
- londons_explore 1y ago
  Just like cache size. But when someone releases a new CPU with a larger or smaller cache, all old software continues to work. Secret page size would offer the same benefit.
  - dataflow 1y ago
    Cache line size would be a better example.
    - Tuna-Fish 1y ago
      Yup. And like with page sizes, the big problems with cache line sizes is not when people design things against a specific line size, but the (much more numerous) cases where they design something that only works well for a fixed size but they don't even know that, because they literally never thought about it, and it worked fine because every machine they used was similar. It doesn't matter what apis you were provided or what analysis you did, you don't know your software works with different hardware until you test it on it.
londons_explore 1y ago
you can simulate arbitrary sized mprotect() by having the kernel do the closest it can using the hardware, and then any pages crossing the boundary will be handled by page faults. The performance hit should be small as long as most mprotect regions are large (which they typically are).
- dzaima 1y ago
  That'd fail horrifically for guard pages around a main region where the whole used region is reasonably expected to be used. Considering that it'd be literally completely entirely unacceptable to ever hit such an emulated range in code of any significance as it'd be a slowdown in the thousands of times, it'd make mprotect entirely pointless and effectively broken. Unless of course you add "padding" of expected page size (not applicable for guard pages though, those are just SOL), which is basically status quo except that, instead of apps doing hard-coding crashing on unexpected page size, they just become potentially 1000x slower, but of course in practice there's no difference between a crash and 1000x slower. I'd even say a crash is better - means a bug report of it can't be dismissed as just a perf issue, and it's clear that something's going wrong in the first place. The size of the mprotect region does not matter. What matters is worst-case behavior, and, regardless of the size, if a single critical byte of memory ends up in the emulated region, your program is dead to the user, causing data loss or whatever consequences from being, practically speaking, completely frozen your program has.
- dwattttt 1y ago
  Emulating it would work quite terribly. With a 4kb page size you can write 4kb of memory as RW, then an adjacent 4kb of memory as RX, and have them interact. Trying to emulate that under 16kb pages would fault every memory operation.
londons_explore 1y ago
the mprotect API could have been designed more like malloc() - ie. you don't protect a preexisting memory range - instead the API returns a new memory range with the protections you've asked for, possibly copying a load of data into it for you incase you asked for a readonly range. And that 'copy' might be zero-overhead remapping of the original pages.
- duskwuff 1y ago
  There's some important use cases for mprotect(), like toggling W^X on JIT pages, which that wouldn't work for.
  - Jyaif 1y ago
    W^X would be supported by the API that they suggest: "possibly copying a load of data into it for you incase you asked for a readonly range."
    - duskwuff 1y ago
      For JIT, you specifically need to be able to write some code to a page, then switch that page from writable to executable in place. Having the page move to a new location isn't acceptable, as the generated code will contain relative calls to other JIT generated code in other pages you've previously written to.
      - Jyaif 1y ago
        Oh right, good point!
  - PhilipRoman 1y ago
    You can probably replace some of those use cases by mapping the same physical page in two places with different permissions (W^X is per-address, not per physical page).
  - IshKebab 1y ago
    If you had a way of allocating pages of at least N bytes it would work. uint8_t* mem = malloc_pages(1024); mprotect_paged(mem, 1024);
pjmlp 1y ago
That isn't officially supported API, so if any app on NDK is making use of it, though luck. https://developer.android.com/ndk/guides/stable_apis https://developer.android.com/ndk/guides/stable_apis
- duskwuff 1y ago
  I'm not sure how you get that idea - as a Linux system call, mprotect() is part of "the standard C11 library headers", and it's used internally by the dynamic linker. Android doesn't need to take any steps to support it, and there's little they could do to remove support for it.
  - pjmlp 1y ago
    I got that idea knowing ISO C standard, POSIX and not mixing themes. By not being part of the official stable APIs, Google is free to break whatever userspace application they feel like. There are seccomp and sandboxing filters in place for what is exposed to non-system apps, aka regular PlayStore apps.

Also various performance characteristics graphs will have steps around the multiples of page sizes. People would find out anyway. (Although maybe the runtime detection is not a bad idea)

londons_explore 1y ago
Just like cache size. But when someone releases a new CPU with a larger or smaller cache, all old software continues to work. Secret page size would offer the same benefit.
- dataflow 1y ago
  Cache line size would be a better example.
  - Tuna-Fish 1y ago
    Yup. And like with page sizes, the big problems with cache line sizes is not when people design things against a specific line size, but the (much more numerous) cases where they design something that only works well for a fixed size but they don't even know that, because they literally never thought about it, and it worked fine because every machine they used was similar. It doesn't matter what apis you were provided or what analysis you did, you don't know your software works with different hardware until you test it on it.

londons_explore 1y ago

you can simulate arbitrary sized mprotect() by having the kernel do the closest it can using the hardware, and then any pages crossing the boundary will be handled by page faults. The performance hit should be small as long as most mprotect regions are large (which they typically are).

dzaima 1y ago
That'd fail horrifically for guard pages around a main region where the whole used region is reasonably expected to be used. Considering that it'd be literally completely entirely unacceptable to ever hit such an emulated range in code of any significance as it'd be a slowdown in the thousands of times, it'd make mprotect entirely pointless and effectively broken. Unless of course you add "padding" of expected page size (not applicable for guard pages though, those are just SOL), which is basically status quo except that, instead of apps doing hard-coding crashing on unexpected page size, they just become potentially 1000x slower, but of course in practice there's no difference between a crash and 1000x slower. I'd even say a crash is better - means a bug report of it can't be dismissed as just a perf issue, and it's clear that something's going wrong in the first place. The size of the mprotect region does not matter. What matters is worst-case behavior, and, regardless of the size, if a single critical byte of memory ends up in the emulated region, your program is dead to the user, causing data loss or whatever consequences from being, practically speaking, completely frozen your program has.
dwattttt 1y ago
Emulating it would work quite terribly. With a 4kb page size you can write 4kb of memory as RW, then an adjacent 4kb of memory as RX, and have them interact. Trying to emulate that under 16kb pages would fault every memory operation.

londons_explore 1y ago

the mprotect API could have been designed more like malloc() - ie. you don't protect a preexisting memory range - instead the API returns a new memory range with the protections you've asked for, possibly copying a load of data into it for you incase you asked for a readonly range. And that 'copy' might be zero-overhead remapping of the original pages.

duskwuff 1y ago
There's some important use cases for mprotect(), like toggling W^X on JIT pages, which that wouldn't work for.
- Jyaif 1y ago
  W^X would be supported by the API that they suggest: "possibly copying a load of data into it for you incase you asked for a readonly range."
  - duskwuff 1y ago
    For JIT, you specifically need to be able to write some code to a page, then switch that page from writable to executable in place. Having the page move to a new location isn't acceptable, as the generated code will contain relative calls to other JIT generated code in other pages you've previously written to.
    - Jyaif 1y ago
      Oh right, good point!
- PhilipRoman 1y ago
  You can probably replace some of those use cases by mapping the same physical page in two places with different permissions (W^X is per-address, not per physical page).
- IshKebab 1y ago
  If you had a way of allocating pages of at least N bytes it would work. uint8_t* mem = malloc_pages(1024); mprotect_paged(mem, 1024);

pjmlp 1y ago

That isn't officially supported API, so if any app on NDK is making use of it, though luck. https://developer.android.com/ndk/guides/stable_apis https://developer.android.com/ndk/guides/stable_apis

duskwuff 1y ago
I'm not sure how you get that idea - as a Linux system call, mprotect() is part of "the standard C11 library headers", and it's used internally by the dynamic linker. Android doesn't need to take any steps to support it, and there's little they could do to remove support for it.
- pjmlp 1y ago
  I got that idea knowing ISO C standard, POSIX and not mixing themes. By not being part of the official stable APIs, Google is free to break whatever userspace application they feel like. There are seccomp and sandboxing filters in place for what is exposed to non-system apps, aka regular PlayStore apps.