Someone "grafted" llama 33B onto llama v2 13B to make "llama 22B" https://huggingface.co/chargoddard/llama2-22b https://huggingface.co/chargoddard/llama2-22b

Someone "grafted" llama 33B onto llama v2 13B to make "llama 22B" https://huggingface.co/chargoddard/llama2-22b https://huggingface.co/chargoddard/llama2-22b Theoretically this is an even better size, as it would fit on a 20GB-24GB GPU with more relaxed quantization and much longer context. Metrics are slightly below 13B, but the theory is that the higher parameter count is more amenable to finetuning. If you search for 22B on huggingface, you can see that frankenllama experiments are ongoing: https://huggingface.co/models?sort=modified&search=22b https://huggingface.co/models?sort=modified&search=22b