2 ms·

Right: https://huggingface.co/poolside/Laguna-S-2.1-FP8/discussions/1#6a6123906a9d9d6d05b7872d https://huggingface.co/poolside/Laguna-S-2.1-FP8/discussions...

• embedding-shape 2mo ago
  More updates from the Discord (also mostly about the NVFP4): > we've got an NVFP4 checkpoint that has a KD of 0.135 relative to the BF16 checkpoint (for W4A4, on some GSM8K prompts). [...] We're still seeing some looping on W4A4 (although much less than before), whereas for W4A16 we've not been able to make it loop. Given that W4A4 is still a little broken, we won't make this an official release (we have one more idea planned to fix that). [...] The RC1 for NVFP4 is reachable as poolside/Laguna-S-2.1-NVFP4 @ RC1 Seems to be available already indeed: https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/tree/RC1 https://huggingface.co/poolside/Laguna-S-2.1-NVFP4/tree/RC1
  • Lwerewolf 2mo ago
    Tool calling in pi completely broke with the updated q4 gguf (spinquant-less). Guessing it'll take some time.
    • Lwerewolf 2mo ago
      ...they'd messed something up in that quant, apparently fixed promptly. ds4 now supports it as well and it's... well, context-limited (<=250k on a 128gb machine) but it positively flies on an m5 max - the 60tok/s decode / ~500tok/s prefill is real.