4 ms·
Wrong. The data (and compute) matters infinitely more than the training regime/architecture. This is the Bitter lesson. I could rerun Kimi K3 with full transfo
by Der_Einzige 2mo ago
Wrong. The data (and compute) matters infinitely more than the training regime/architecture. This is the Bitter lesson.
I could rerun Kimi K3 with full transformers instead of the hybrid mamba architecture and it would only get better.