3 ms·
Technically the llms require a pass for each token, but the passes are cheaper and benefit more from batching.
by brucethemoose2 3y ago
Technically the llms require a pass for each token, but the passes are cheaper and benefit more from batching.