3 ms·
That last bit sounds impressive…
by bjornsing 3y ago
That last bit sounds impressive…
- verdverm 3y agoIt does, they are largely using a full sized LLM to guide a smaller one, but it does mean we can pool resources for the large LLM and then build a company smaller LLM for wide distribution.
- dpflan 3y agoYes, these huge models are the expert systems for training specialized systems that will run often and need to run cheaply.
- VHRanger 3y agoPaLM is ridiculously inefficient in the number of parameters. It's about as good on quantitative benchmarks as LLaMa 65B
- nl 3y agoThis is true (and sort of the point of PaLM - it was an experiment on truly huge model sizes). But also Llama 65B is very efficient for its size: it generally outperforms models twice as large on most benchmarks.
- VHRanger 3y agoYes, because it turns out if you have a more reasonable number of parameters, but train for longer, the outcome is a more efficient model
- deleted 3y ago[deleted]