2 ms·
Depends on what you're doing. I'm pretty sure the bandwidth for inference isn't much.
by Dylan16807 1y ago
Depends on what you're doing. I'm pretty sure the bandwidth for inference isn't much.
- eurekin 1y agoDepends, if it's tensor parallel or pipeline parallel. Only PP doesn't pass too much. TP does