3 ms·
> Additionally, TensorRT-LLM includes fully optimized, ready-to-run versions of many LLMs widely used in production today. This includes Meta Llama 2, OpenAI GP
by tikkun 3y ago
> Additionally, TensorRT-LLM includes fully optimized, ready-to-run versions of many LLMs widely used in production today. This includes Meta Llama 2, OpenAI GPT-2 and GPT-3, Falcon, Mosaic MPT, BLOOM, and a dozen others, all of which can be implemented with the simple-to-use TensorRT-LLM Python API.
Is this a typo? They mention it includes an optimized version of GPT-3, but GPT-3 is closed source.
- visarga 3y agoThe code is known, nothing special. Just the weights are missing.
- spott 3y agoThe weights of gpt3 weren’t released, but the architecture is pretty close to gpt2 and explained in the paper: https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf I think the “ready to run” might be more of a byow (“bring your own weights”) kinda thing.