3 ms·
It's not a tradeoff in this case, this is an optimized megakernel for the same model for better throughput. And no, in most cases accuracy can be sacrificed in
by orbital-decay 4mo ago
It's not a tradeoff in this case, this is an optimized megakernel for the same model for better throughput. And no, in most cases accuracy can be sacrificed in favor of throughput or latency (assessing it automatically is the harder part).