3 ms·
This is true (and sort of the point of PaLM - it was an experiment on truly huge model sizes). But also Llama 65B is very efficient for its size: it generally o
by nl 4y ago
This is true (and sort of the point of PaLM - it was an experiment on truly huge model sizes). But also Llama 65B is very efficient for its size: it generally outperforms models twice as large on most benchmarks.
- VHRanger 4y agoYes, because it turns out if you have a more reasonable number of parameters, but train for longer, the outcome is a more efficient model