If you want high performance inference us this: https://vllm.ai/ https://vllm.ai/ For local batch stuff: https://huggingface.co/docs/transformers/main_classes/

If you want high performance inference us this: https://vllm.ai/ https://vllm.ai/ For local batch stuff: https://huggingface.co/docs/transformers/main_classes/pipelines https://huggingface.co/docs/transformers/main_classes/pipeli...