3 ms·
If you're hitting a memory wall it means you're not scaling. This stuff really doesn't apply to scaled up inference but rather local small batch execution
by brrrrrm 2y ago
If you're hitting a memory wall it means you're not scaling. This stuff really doesn't apply to scaled up inference but rather local small batch execution