3 ms·
Seems odd to claim 25x reduction in size when the algo involves looking into a database of a trillion chunks of text.
by davefol 5y ago
Seems odd to claim 25x reduction in size when the algo involves looking into a database of a trillion chunks of text.
- visarga 5y agoThe "algo" here refers to the neural net itself. The text index is considered an easy problem as you can do lookups in logarithmic time.
- ehsankia 5y agoThe word "Algo" here is definitely awkward. The point is though that what matters most here is the number of parameters, as those correlate quite closely with training and inference time. Storage space is pretty trivial, but TPU cycles are less so.
- davefol 5y agoThanks for this. Rereading + your comment and I think I have a better understanding of why this is progress.