4 ms·
TensorRT-LLM being open source is a lie, all the important kernels are loaded from cubins.
by andersa 1y ago
TensorRT-LLM being open source is a lie, all the important kernels are loaded from cubins.
- nabakin 1y agoYeah you're right (although, they started to open source some of that recently iirc). I meant SOTA for inference engines we can actually download and use ourselves.