21 ms·

Currently software is king when it comes to inference performance though.

• Almondsetat 2mo ago
  Having the entire model in the vram without having to transfer anything is even more king, though