Hey I did, and sorry for the self promo, Please check out https://github.com/geniusrise https://github.com/geniusrise - tool for running llms and other stuff,

Hey I did, and sorry for the self promo, Please check out https://github.com/geniusrise https://github.com/geniusrise - tool for running llms and other stuff, behaves like docker compose, works with whatever is supported by underlying engines: Huggingface - MPS, cuda VLLM - cuda, ROCm llama.cpp, whisper.cpp - cuda, mps, rocm Also coming up integration with spark (TorchDistributor), kafka and airflow.