4 ms·
Is there anything that affects inference compute time besides the number of parameters? Assuming same hardware, etc.
by dwaltrip 4y ago
Is there anything that affects inference compute time besides the number of parameters? Assuming same hardware, etc.
- typon 4y agoYes - for example adding memory to the attention mechanism (similar to RETRO or Memorizing Transformers paper)