4 ms·

I’ve been running it via llama-server with no issues. Running the latest Bartowski 6-bit quant

• brightball 7mo ago
  Bartowski? Like Chuck Bartowski from the TV show?
  • BoredomIsFun 7mo ago
    Different one. Bartowski is a minor celebrity in the local LLM world, together with Unsloth.
    • Balinares 7mo ago
      What's the selling point of these quants vs the Unsloth ones?
      • BoredomIsFun 7mo ago
        Sometimes unsloth has broken ones for a particular model, sometimes no quants at all, and there is subtle difference in behavior.
• abhikul0 7mo ago
  Thanks, i'll check his quants.