5 ms·
I think the distinction with ML inference workloads is that you often have very little control flow, so this type of architecture lets you match layers to adjac
by adgjlsfhk1 3y ago
I think the distinction with ML inference workloads is that you often have very little control flow, so this type of architecture lets you match layers to adjacent cores so that each operation gets it's data directly from the step before rather than from RAM.
- weebull 3y agoRight, so latency is less of an issue (because control flow is predictable) and performance becomes about bandwidth.