4 ms·

No, M1 is ARM64, so any existing ARM64 compatible packages will work. However, some packages may not be compatible with 16K pages, so you may need the 4K page v

No, M1 is ARM64, so any existing ARM64 compatible packages will work. However, some packages may not be compatible with 16K pages, so you may need the 4K page version of the kernel. Linux (unlike macOS) does not support mixed page sizes (edit: at least with M1) so it will result in reduced performance for the whole system if you need 4K pages.

eklitzke 5y ago
Linux does support mixed page sizes (that's how huge pages work) and the page size isn't even detectable by userspace user than through sysconf(3) or getpagesize(2) so I'm not sure why a program wouldn't be compatible with 16K pages; after all, regular programs work just fine with THP on Linux.
- Sirened 5y ago
  From what I've seen just bumping into some of the devs on twitter, many larger software packages (i.e. Chromium) used a hardcoded pagesize. AFAIK, Asahi doesn't actually support mixed pages—just 16k—due to some hardware quirks on the M1 platform, and so running in 4K compat mode wouldn't even help. This is obviously problematic if you're trying to enforce memory permissions on 4k boundaries since you can't simply pierce the huge page like you can in THP since there is no smaller granule to fallback to.
- 58028641 5y ago
  > Linux can’t really mix page sizes like that and likely never will be able to https://asahilinux.org/2021/10/progress-report-september-2021/ https://asahilinux.org/2021/10/progress-report-september-202...
- marcan_42 5y ago
  That's not the same thing. Huge pages are in addition to standard pages. Being in 16K mode means no 4K pages. It's a global switch. Linux can't handle that, the baseline page size is set at compile time and affects a ton of macros and constants used throughout the kernel. Userspace breaks on 16K pages when it tries to do things like call mmap() with virtual addresses that aren't aligned to 16K. Usually it's allocators doing this when they think the entire world is 4K.
  - zozbot234 5y ago
    If page size can be chosen per process (such as 4K pages for Rosetta apps) that's not unlike the existing hugepages. Are we sure that it cannot be easily adapted to support both 4k and 16k processes?
    - marcan_42 5y ago
      It is completely different from hugepages. Hugepages exist alongside normal pages.
      - zozbot234 5y ago
        > Hugepages exist alongside normal pages. Don't 16K pages "exist alongside" 4k pages too, just not within the same virtual address space or (in Linux) VMA? How else are Rosetta apps supposed to work in Mac OS?
        marcan_42 5y ago
        That's the issue: 16K pages never exist alongside 4K pages in the same address space half. The CPU has two mode flags, for kernel and userspace respectively. There is no way to mix modes within the same address space half. And changing the page size completely changes the page table structure and boundaries for different walk levels, the huge page size, etc. Rosetta runs the userspace half in 4K mode, and XNU had to be reworked a lot to support this. Linux could of course be reworked to do something similar on paper, but it's a hugely intrusive change and it'd actually be easier to just make the kernel support 4K/16K pages in a single build first. Hugepages aren't like that, they actually coexist with normal pages. In general, hugepages are just a pile of contiguous/aligned small pages that the kernel manages as a unit, and it flags them to tell the MMU "I promise these are all one big contiguous chunk so you can optimize it to one larger TLB entry". Depending on the page table structure they might be coalesced to a higher-level page table entry, skipping a page table walk level.
        zozbot234 5y ago
fulafel 5y ago
The 16k page size might be temporary in this port, the HW does support 4k pages as well. In an earlier post they said this about progress on the iommu hw limitation leading to start out with 16k page size: "Sven took on the challenge and now has a patch series that makes Linux’s IOMMU support layer play nicely with hardware that has an IOMMU page size larger than the kernel page size" (https://asahilinux.org/2021/10/progress-report-september-2021/ https://asahilinux.org/2021/10/progress-report-september-202...)
- WhyNotHugo 5y ago
  Apparently 16k pages result in a huge performance boost (20%) for many workloads, so hopefully broken software will adapt too.
  - fulafel 5y ago
    Isn't it more about not breaking binary compatibility with existing Linux arm64 userland that has been stable for many years, than source level correctness bugs in apps? Or does the userspace ABI explicitly allow for switching between 4k and 16k page size? (Also apps can always opt-in to bigger pages using the existing mechanisms) Of course it's also true that 4k is a ridiculously small page size, we've been using the same size for 30-40 years while memory sizes have grown 5-6 decimal orders of magnitude. But from that POV we should now do a bigger bump than 4k->16k - I think the next bigger commonly used page size on ARM Linux has been 64k.
    - lonjil 5y ago
      Page size is not set when you compile. At runtime, you can ask the kernel what the page size is, and use that value where appropriate. Some poorly written software hardcodes the page size on a per target basis.
      - fulafel 5y ago
        I know there are some Unix specs that say that page size is reported via runtime interfaces, but I suspect that eg on x86 if you built a Linux kernel that gave userspace a 2MB page size, and started returning that from the sysconf(_SC_PAGESIZE) interface, many (most?) apps would break and shrugging that away with "those apps are just broken" would be a nonstarter. But maybe just counts as a a de facto user expectation and is not documented anywhere architecture in Linux ABIs?

eklitzke 5y ago

Linux does support mixed page sizes (that's how huge pages work) and the page size isn't even detectable by userspace user than through sysconf(3) or getpagesize(2) so I'm not sure why a program wouldn't be compatible with 16K pages; after all, regular programs work just fine with THP on Linux.

Sirened 5y ago
From what I've seen just bumping into some of the devs on twitter, many larger software packages (i.e. Chromium) used a hardcoded pagesize. AFAIK, Asahi doesn't actually support mixed pages—just 16k—due to some hardware quirks on the M1 platform, and so running in 4K compat mode wouldn't even help. This is obviously problematic if you're trying to enforce memory permissions on 4k boundaries since you can't simply pierce the huge page like you can in THP since there is no smaller granule to fallback to.
58028641 5y ago
> Linux can’t really mix page sizes like that and likely never will be able to https://asahilinux.org/2021/10/progress-report-september-2021/ https://asahilinux.org/2021/10/progress-report-september-202...
marcan_42 5y ago
That's not the same thing. Huge pages are in addition to standard pages. Being in 16K mode means no 4K pages. It's a global switch. Linux can't handle that, the baseline page size is set at compile time and affects a ton of macros and constants used throughout the kernel. Userspace breaks on 16K pages when it tries to do things like call mmap() with virtual addresses that aren't aligned to 16K. Usually it's allocators doing this when they think the entire world is 4K.
- zozbot234 5y ago
  If page size can be chosen per process (such as 4K pages for Rosetta apps) that's not unlike the existing hugepages. Are we sure that it cannot be easily adapted to support both 4k and 16k processes?
  - marcan_42 5y ago
    It is completely different from hugepages. Hugepages exist alongside normal pages.
    - zozbot234 5y ago
      > Hugepages exist alongside normal pages. Don't 16K pages "exist alongside" 4k pages too, just not within the same virtual address space or (in Linux) VMA? How else are Rosetta apps supposed to work in Mac OS?
      - marcan_42 5y ago
        That's the issue: 16K pages never exist alongside 4K pages in the same address space half. The CPU has two mode flags, for kernel and userspace respectively. There is no way to mix modes within the same address space half. And changing the page size completely changes the page table structure and boundaries for different walk levels, the huge page size, etc. Rosetta runs the userspace half in 4K mode, and XNU had to be reworked a lot to support this. Linux could of course be reworked to do something similar on paper, but it's a hugely intrusive change and it'd actually be easier to just make the kernel support 4K/16K pages in a single build first. Hugepages aren't like that, they actually coexist with normal pages. In general, hugepages are just a pile of contiguous/aligned small pages that the kernel manages as a unit, and it flags them to tell the MMU "I promise these are all one big contiguous chunk so you can optimize it to one larger TLB entry". Depending on the page table structure they might be coalesced to a higher-level page table entry, skipping a page table walk level.
        zozbot234 5y ago

fulafel 5y ago

The 16k page size might be temporary in this port, the HW does support 4k pages as well. In an earlier post they said this about progress on the iommu hw limitation leading to start out with 16k page size: "Sven took on the challenge and now has a patch series that makes Linux’s IOMMU support layer play nicely with hardware that has an IOMMU page size larger than the kernel page size" (https://asahilinux.org/2021/10/progress-report-september-2021/ https://asahilinux.org/2021/10/progress-report-september-202...)

WhyNotHugo 5y ago
Apparently 16k pages result in a huge performance boost (20%) for many workloads, so hopefully broken software will adapt too.
- fulafel 5y ago
  Isn't it more about not breaking binary compatibility with existing Linux arm64 userland that has been stable for many years, than source level correctness bugs in apps? Or does the userspace ABI explicitly allow for switching between 4k and 16k page size? (Also apps can always opt-in to bigger pages using the existing mechanisms) Of course it's also true that 4k is a ridiculously small page size, we've been using the same size for 30-40 years while memory sizes have grown 5-6 decimal orders of magnitude. But from that POV we should now do a bigger bump than 4k->16k - I think the next bigger commonly used page size on ARM Linux has been 64k.
  - lonjil 5y ago
    Page size is not set when you compile. At runtime, you can ask the kernel what the page size is, and use that value where appropriate. Some poorly written software hardcodes the page size on a per target basis.
    - fulafel 5y ago
      I know there are some Unix specs that say that page size is reported via runtime interfaces, but I suspect that eg on x86 if you built a Linux kernel that gave userspace a 2MB page size, and started returning that from the sysconf(_SC_PAGESIZE) interface, many (most?) apps would break and shrugging that away with "those apps are just broken" would be a nonstarter. But maybe just counts as a a de facto user expectation and is not documented anywhere architecture in Linux ABIs?