4 ms·
Could we bank on the Lottery Ticket Hypothesis, distillation, or other model compression algorithms to make these models smaller?
by whymauri 6y ago
Could we bank on the Lottery Ticket Hypothesis, distillation, or other model compression algorithms to make these models smaller?
- aquajet 6y agoI would guess so, but compressing it by 1/3rd it's size (ie. distilgpt) would still be quite large. To be fair, I don't know if distillation scales like that.