4 ms·
When you say "inference", do you mean "interface", or is "inference" an ML term I'm not familiar with?
by buffington 5y ago
When you say "inference", do you mean "interface", or is "inference" an ML term I'm not familiar with?
- sanity31415 5y agoFirst you train a model then you use it, "inference" is a fancy word for using the model.
- igorkraw 5y agoIt's a ML term, inference basically means using the probability model you learned to draw "inferences" about a piece of data. In this context, it means giving the language model some context and using some method (either arg max sampling or something more sophisticated like beam search) to do what amounts to statistical auto implemention on it. As you might imagine, doing this with 530 GB of data at speed is quite energy intensive, even though there are things you can do to compress the model (distillation, pruning, compression/discretization) and specialised inference hardware. Technically there is some very specific meaning to inference vs. prediction, but it's been heavily overloaded with meaning by now