2 ms·
Doesnt that say they based it on llama?, sooooo not really a bottoms up training - since the cost of llama is 100% surely not part of their quote.
by bythreads 2y ago
Doesnt that say they based it on llama?, sooooo not really a bottoms up training - since the cost of llama is 100% surely not part of their quote.
- diggan 2y agoI did a quick search for "llama" and didn't find anywhere they outright state they just fine-tuned some llama weights. Is it possible that they based their model architecture on the llama model architecture? Rather than just fine-tuned already training llama weights? In that case, they'd still have to do "bottoms up" training.