4 ms·
Would be interesting to see if we can make this model smaller with REAP + unsloth dynamic quant. It might become 4x cheaper to run for similar quality output
by ttoinou 4mo ago
Would be interesting to see if we can make this model smaller with REAP + unsloth dynamic quant. It might become 4x cheaper to run for similar quality output