3 ms·

Here's a simple calculator for LLM inference requirements: https://rahulschand.github.io/gpu_poor/ https://rahulschand.github.io/gpu_poor/