2 ms·
I believe you have confused training and inference. Training usually uses large amounts of data to get your system to recognize a pattern. It generally uses h
by m463 6y ago
I believe you have confused training and inference.
Training usually uses large amounts of data to get your system to recognize a pattern. It generally uses huge memory and compute and generates a model.
Inference will use that generated model to recognize the pattern in new data. It uses significantly fewer resources and can run either very fast or on a smaller system.
I believe the speed of inference may be affected by the resources available during training, where more speed/memory for training can produce better models.
- tantalor 6y agoI think it's parent commenter who is confused, complains how hard it is to "squeeze state of the art models into my limited memory", which sounds like inference.