3 ms·We’re using 6bit quants since we have 32GB cards. Gemma QAT is an honourable mention.by trollbridge 2mo agoWe’re using 6bit quants since we have 32GB cards. Gemma QAT is an honourable mention.