4 ms·

I wonder how much help they had by asahi doing a lot of the kernel and ecosystem work anablibg 16k pages. RISC-V being fixed to 4k pages seems to be a bit of a

I wonder how much help they had by asahi doing a lot of the kernel and ecosystem work anablibg 16k pages. RISC-V being fixed to 4k pages seems to be a bit of an oversight as well.

ashkankiani 2y ago
It's pretty cool that I can read "anablibg" and know that means "enabling." The brain is pretty neat. I wonder if LLMs would get it too. They probably would.
- mrob 2y ago
  LLMs are at a great disadvantage here because they operate on tokens, not letters.
  - platelminto 2y ago
    I remember reading somewhere that LLMs are actually fantastic at reading heavily mistyped sentences! Mistyped to a level where humans actually struggle. (I will update this comment if I find a source)
    - thanatropism 2y ago
      Tihs probably refers to comon mispelllings an typo's.
      - HeatrayEnjoyer 2y ago
        It's actually not. You can scramble every letter within words and it can mostly unscramble it. Keep the first letter and it recovers almost 100%.
- mrbuttons454 2y ago
  Until I read your comment I didn't even notice...
- evilduck 2y ago
  Question I wrote: > I encountered the typo "anablibg" in the sentence "I wonder how much help they had by asahi doing a lot of the kernel and ecosystem work anablibg 16k pages." What did they actually mean? GPT-4o and Sonnet 3.5 understood it perfectly. This isn't really a problem for the large models. For local small models: * Gemma2 9b did not get it and thought it meant "analyzing". * Codestral (22b) did not it get it and thought it meant "allocating". * Phi3 Mini failed spectacularly. * Phi3 14b and Qwen2 did not get it and thought it was "annotating". * Mistral-nemo thought it was a portmanteau "anabling" as a combination of "an" and "enabling". Partial credit for being close and some creativity? * Llama3.1 got it perfectly.
  - jandrese 2y ago
    Seems like there is a bit of a roll of the dice there. The ones that got it right may have just been lucky.
    - HeatrayEnjoyer 2y ago
      Ran it a few times in new sessions, 0 failures so far.
  - slaymaker1907 2y ago
    I wonder how much of a test this is for the LLM vs whatever tokenizer/preprocessing they're doing.
  - Retr0id 2y ago
    fwiw I failed to figure it out as a human, I had to check the replies.
  - Alifatisk 2y ago
    Is there any task Gemma is better at compared to others?
    - evilduck 2y ago
      Local LLM topics are a treadmill of “what’s best and what is preferred” changing basically weekly to monthly, it’s a rapidly evolving field, but right now I actually tend to gravitate to Gemma2 9b for coding assistance for Typescript work or general question and answer stuff. Its embedded knowledge and speed on the computers that I have (32GB M2 Max, 16GB M1 Air, 4080 gaming desktop) make for a good balance while also using the computer for other stuff, bigger models limit what else I can run simultaneously and are slower than my reading speed, smaller models have less utility and the speed increase is pointless if they’re dumb.
- im3w1l 2y ago
  I asked chatgpt and it did get it. Personally, when I read the comment my brain kinda skipped over the word since it contained the part "lib" I assumed it was some obscure library that I didn't care about. It doesn't fit grammatically but I didn't give it enough thought to notice.
IshKebab 2y ago
Probably wouldn't be too hard to add a 16 kB page size extension. But I think the Svnapot extension is their solution to this problem. If you're not familiar it lets you mark a set of pages as being part of a contiguously mapped 64 kB region. No idea how the performance characteristics vary. It relieves TLB pressure, but you still have to create 16 4kB page table entries.
- monocasa 2y ago
  Svnapot is a poor solution to the problem. On one hand it means that that each page table entry takes up half a cache line for the 16KB case, and two whole cache lines in the 64KB case. This really cuts down on the page walker hardware's ability to effectively prefetch TLB entries, leading to basically the same issues as this classic discussion about why tree based page tables are generally more effective than hash based page tables (shifted forward in time to today's gate counts). https://yarchive.net/comp/linux/page_tables.html https://yarchive.net/comp/linux/page_tables.html This is why ARM shifted from a Svnapot like solution to the "translation granule queryable and partially selectable at runtime" solution. Another issue is the fact that a big reason to switch to 16KB or even 64KB pages is to allow for more address range for VIPT caches. You want to allow high performance implementations to be able to look up the cache line while performing the TLB lookup in parallel, then compare the tag with the result of the TLB lookup. This means that practically only the untranslated bits of the address can be used by the set selection portion of the cache lookup. When you have 12 bits untranslated in a address, combined with 64 byte cachelines gives you 64 sets, multiply that by 8 ways and you get the 32KB L1 caches very common in systems with 4KB page sizes (sometimes with some heroic effort to throw a ton of transistors/power at the problem to make a 64KB cache by essentially duplicating large parts of the cache lookup hardware for that extra bit of address). What you really want is for the arch to be able to disallow 4KB pages like on apple silicon which is the main piece that allows their giant 128LB and 192KB L1 caches.
  - aseipp 2y ago
    > What you really want is for the arch to be able to disallow 4KB pages like on apple silicon which is the main piece that allows their giant 128LB and 192KB L1 caches. Minor nit but they allow 4k pages. Linux doesn't support 16k and 4k pages at the same time; macOS does but is just very particular about 4k pages being used for scenarios like Rosetta processes or virtual machines e.g. Parallels uses it for Windows-on-ARM, I think. Windows will probably never support non-4k pages I'd guess. But otherwise, you're totally right. I wish RISC-V had gone with the configurable granule approach like ARM did. Major missed opportunity but maybe a fix will get ratified at some point...
saagarjha 2y ago
Probably very little, since the Android ecosystem is quite divorced from the Linux one.
- nabla9 2y ago
  Android kernel is a mainstream Linux kernel, with additional drivers, and other functionality.
  - temac 2y ago
    The linux kernel already works perfectly fine with various base page sizes.
  - saagarjha 2y ago
    I am aware. This does not change what I said.
wren6991 2y ago
RV64 has some reserved encoding space in satp.mode so there's an obvious path to expanding the number of page table formats at a later time. Just requires everyone to agree on the direction (common issue with RISC-V). For RV32 I think we are probably stuck with Sv32 4k pages forever.

I wonder how much help they had by asahi doing a lot of the kernel and ecosystem work anablibg 16k pages. RISC-V being fixed to 4k pages seems to be a bit of an oversight as well.

ashkankiani 2y ago

It's pretty cool that I can read "anablibg" and know that means "enabling." The brain is pretty neat. I wonder if LLMs would get it too. They probably would.

mrob 2y ago
LLMs are at a great disadvantage here because they operate on tokens, not letters.
- platelminto 2y ago
  I remember reading somewhere that LLMs are actually fantastic at reading heavily mistyped sentences! Mistyped to a level where humans actually struggle. (I will update this comment if I find a source)
  - thanatropism 2y ago
    Tihs probably refers to comon mispelllings an typo's.
    - HeatrayEnjoyer 2y ago
      It's actually not. You can scramble every letter within words and it can mostly unscramble it. Keep the first letter and it recovers almost 100%.
mrbuttons454 2y ago
Until I read your comment I didn't even notice...
evilduck 2y ago
Question I wrote: > I encountered the typo "anablibg" in the sentence "I wonder how much help they had by asahi doing a lot of the kernel and ecosystem work anablibg 16k pages." What did they actually mean? GPT-4o and Sonnet 3.5 understood it perfectly. This isn't really a problem for the large models. For local small models: * Gemma2 9b did not get it and thought it meant "analyzing". * Codestral (22b) did not it get it and thought it meant "allocating". * Phi3 Mini failed spectacularly. * Phi3 14b and Qwen2 did not get it and thought it was "annotating". * Mistral-nemo thought it was a portmanteau "anabling" as a combination of "an" and "enabling". Partial credit for being close and some creativity? * Llama3.1 got it perfectly.
- jandrese 2y ago
  Seems like there is a bit of a roll of the dice there. The ones that got it right may have just been lucky.
  - HeatrayEnjoyer 2y ago
    Ran it a few times in new sessions, 0 failures so far.
- slaymaker1907 2y ago
  I wonder how much of a test this is for the LLM vs whatever tokenizer/preprocessing they're doing.
- Retr0id 2y ago
  fwiw I failed to figure it out as a human, I had to check the replies.
- Alifatisk 2y ago
  Is there any task Gemma is better at compared to others?
  - evilduck 2y ago
    Local LLM topics are a treadmill of “what’s best and what is preferred” changing basically weekly to monthly, it’s a rapidly evolving field, but right now I actually tend to gravitate to Gemma2 9b for coding assistance for Typescript work or general question and answer stuff. Its embedded knowledge and speed on the computers that I have (32GB M2 Max, 16GB M1 Air, 4080 gaming desktop) make for a good balance while also using the computer for other stuff, bigger models limit what else I can run simultaneously and are slower than my reading speed, smaller models have less utility and the speed increase is pointless if they’re dumb.
im3w1l 2y ago
I asked chatgpt and it did get it. Personally, when I read the comment my brain kinda skipped over the word since it contained the part "lib" I assumed it was some obscure library that I didn't care about. It doesn't fit grammatically but I didn't give it enough thought to notice.

IshKebab 2y ago

Probably wouldn't be too hard to add a 16 kB page size extension. But I think the Svnapot extension is their solution to this problem. If you're not familiar it lets you mark a set of pages as being part of a contiguously mapped 64 kB region. No idea how the performance characteristics vary. It relieves TLB pressure, but you still have to create 16 4kB page table entries.

monocasa 2y ago
Svnapot is a poor solution to the problem. On one hand it means that that each page table entry takes up half a cache line for the 16KB case, and two whole cache lines in the 64KB case. This really cuts down on the page walker hardware's ability to effectively prefetch TLB entries, leading to basically the same issues as this classic discussion about why tree based page tables are generally more effective than hash based page tables (shifted forward in time to today's gate counts). https://yarchive.net/comp/linux/page_tables.html https://yarchive.net/comp/linux/page_tables.html This is why ARM shifted from a Svnapot like solution to the "translation granule queryable and partially selectable at runtime" solution. Another issue is the fact that a big reason to switch to 16KB or even 64KB pages is to allow for more address range for VIPT caches. You want to allow high performance implementations to be able to look up the cache line while performing the TLB lookup in parallel, then compare the tag with the result of the TLB lookup. This means that practically only the untranslated bits of the address can be used by the set selection portion of the cache lookup. When you have 12 bits untranslated in a address, combined with 64 byte cachelines gives you 64 sets, multiply that by 8 ways and you get the 32KB L1 caches very common in systems with 4KB page sizes (sometimes with some heroic effort to throw a ton of transistors/power at the problem to make a 64KB cache by essentially duplicating large parts of the cache lookup hardware for that extra bit of address). What you really want is for the arch to be able to disallow 4KB pages like on apple silicon which is the main piece that allows their giant 128LB and 192KB L1 caches.
- aseipp 2y ago
  > What you really want is for the arch to be able to disallow 4KB pages like on apple silicon which is the main piece that allows their giant 128LB and 192KB L1 caches. Minor nit but they allow 4k pages. Linux doesn't support 16k and 4k pages at the same time; macOS does but is just very particular about 4k pages being used for scenarios like Rosetta processes or virtual machines e.g. Parallels uses it for Windows-on-ARM, I think. Windows will probably never support non-4k pages I'd guess. But otherwise, you're totally right. I wish RISC-V had gone with the configurable granule approach like ARM did. Major missed opportunity but maybe a fix will get ratified at some point...

saagarjha 2y ago

Probably very little, since the Android ecosystem is quite divorced from the Linux one.

nabla9 2y ago
Android kernel is a mainstream Linux kernel, with additional drivers, and other functionality.
- temac 2y ago
  The linux kernel already works perfectly fine with various base page sizes.
- saagarjha 2y ago
  I am aware. This does not change what I said.

wren6991 2y ago

RV64 has some reserved encoding space in satp.mode so there's an obvious path to expanding the number of page table formats at a later time. Just requires everyone to agree on the direction (common issue with RISC-V). For RV32 I think we are probably stuck with Sv32 4k pages forever.