4 ms·

Deleted earlier, didn't see you post, pasting here: /* Just started testing with the gguf (with gpu offload, m5 max 128gb), q4_k_m, running seemingly well. Spe

Deleted earlier, didn't see you post, pasting here: /* Just started testing with the gguf (with gpu offload, m5 max 128gb), q4_k_m, running seemingly well. Speed is initially slightly faster than antirez/ds4 - decode tok/s in the 30s, prefill ~400-ish. Expected, given the slightly smaller size. Looks to be working fine, but too early to tell. Definitely likes to "think". */ Anyways, guessing that discord might be focusing on the nvfp4 stuff. I've noticed spelling mistakes in the thinking traces, tool calls have been fine so far.

Lwerewolf 2mo ago
Well, just ran said gguf on the GeneralsX codebase with a pretty open-ended "Explain this codebase to me, and the general game loop." prompt, and... Let me also look at the GameLogic::update() to see the rest of the update flow, especially the object update loop. Actually, I think I have enough information now. Let me also check the GameLogic::UPDATE to understand the full update flow. ...repeating forever. Since they mentioned that they're working on new quants, guess I'll wait. From earlier tests on work stuff, it's definitely capable.
- Lwerewolf 2mo ago
  Right: https://huggingface.co/poolside/Laguna-S-2.1-FP8/discussions/1#6a6123906a9d9d6d05b7872d https://huggingface.co/poolside/Laguna-S-2.1-FP8/discussions... Wait time.
  - embedding-shape 2mo ago
    More updates from the Discord (also mostly about the NVFP4): > we've got an NVFP4 checkpoint that has a KD of 0.135 relative to the BF16 checkpoint (for W4A4, on some GSM8K prompts). [...] We're still seeing some looping on W4A4 (although much less than before), whereas for W4A16 we've not been able to make it loop. Given that W4A4 is still a little broken, we won't make this an official release (we have one more idea planned to fix that). [...] The RC1 for NVFP4 is reachable as poolside/Laguna-S-2.1-NVFP4 @ RC1 Seems to be available already indeed: https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/tree/RC1 https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/tree/RC1
    - Lwerewolf 2mo ago
      Tool calling in pi completely broke with the updated q4 gguf (spinquant-less). Guessing it'll take some time.
      - Lwerewolf 2mo ago
        ...they'd messed something up in that quant, apparently fixed promptly. ds4 now supports it as well and it's... well, context-limited (<=250k on a 128gb machine) but it positively flies on an m5 max - the 60tok/s decode / ~500tok/s prefill is real.

Lwerewolf 2mo ago

Well, just ran said gguf on the GeneralsX codebase with a pretty open-ended "Explain this codebase to me, and the general game loop." prompt, and... Let me also look at the GameLogic::update() to see the rest of the update flow, especially the object update loop. Actually, I think I have enough information now. Let me also check the GameLogic::UPDATE to understand the full update flow. ...repeating forever. Since they mentioned that they're working on new quants, guess I'll wait. From earlier tests on work stuff, it's definitely capable.

Lwerewolf 2mo ago
Right: https://huggingface.co/poolside/Laguna-S-2.1-FP8/discussions/1#6a6123906a9d9d6d05b7872d https://huggingface.co/poolside/Laguna-S-2.1-FP8/discussions... Wait time.
- embedding-shape 2mo ago
  More updates from the Discord (also mostly about the NVFP4): > we've got an NVFP4 checkpoint that has a KD of 0.135 relative to the BF16 checkpoint (for W4A4, on some GSM8K prompts). [...] We're still seeing some looping on W4A4 (although much less than before), whereas for W4A16 we've not been able to make it loop. Given that W4A4 is still a little broken, we won't make this an official release (we have one more idea planned to fix that). [...] The RC1 for NVFP4 is reachable as poolside/Laguna-S-2.1-NVFP4 @ RC1 Seems to be available already indeed: https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/tree/RC1 https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/tree/RC1
  - Lwerewolf 2mo ago
    Tool calling in pi completely broke with the updated q4 gguf (spinquant-less). Guessing it'll take some time.
    - Lwerewolf 2mo ago
      ...they'd messed something up in that quant, apparently fixed promptly. ds4 now supports it as well and it's... well, context-limited (<=250k on a 128gb machine) but it positively flies on an m5 max - the 60tok/s decode / ~500tok/s prefill is real.