3 ms·
minGPT is actually quite performant too, the min refers to breadth of supported functionality (eg the absence of support for various additional conditioning, ex
by karpathy 6y ago
minGPT is actually quite performant too, the min refers to breadth of supported functionality (eg the absence of support for various additional conditioning, exotic masking, masked LMs, finetuning, pruning, etc).
- activatedgeek 6y agoSure thing! I only meant to imply the relative ordering of considerations.
- minimaxir 6y agoGPT training performance on the CPU is funny. The vocab size and context window size have a massive effect on both speed and accuracy.