4 ms·

This calculation doesn't account for batches, it makes no sense.

• BriggyDwiggs42 1y ago
  On average how much does batching bring costs down?
  • GaggiX 1y ago
    It balances the computing and memory bandwidth bottleneck so by a lot, with continuous batching you can easily see a x10, x20 or more.
    • BriggyDwiggs42 1y ago
      Wow! Thanks.