4 ms·
Show HN: Local fine tuning for Mistral and SDXL, GPU mem/latency optimization
100% bootstrapped new startup. It lets you fine tune Mistral-7B and SDXL. In particular, for the LLM fine tuning we implemented a dataprep pipeline that turns websites/pdfs/doc files into question-answer pairs for training the small LLM using an big LLM.
It includes a GPU scheduler that can do finegrained GPU memory scheduling (Kubernetes can only do whole-GPU, we do it per-GB of GPU memory to pack both inference and fine tuning jobs into the same fleet) to fit model instances into GPU memory to optimally trade off user facing latency with GPU memory utilization
It's a pretty simple stack of control plane and a fat container that runs anywhere you can get hold of a GPU (e.g. runpod).
Architecture: https://docs.helix.ml/docs/architecture https://docs.helix.ml/docs/architecture
Demo walkthrough showing runner dashboard:
https://docs.helix.ml/docs/overview https://docs.helix.ml/docs/overview
Run it yourself: https://docs.helix.ml/docs/controlplane https://docs.helix.ml/docs/controlplane
Discord: https://discord.gg/VJftd844GE https://discord.gg/VJftd844GE
Please roast me!
- lewq 3y agoSome resources: Demo: https://www.youtube.com/watch?v=Ym4nPSzfer0 https://www.youtube.com/watch?v=Ym4nPSzfer0 About Helix Helix is a generative AI platform that you can run on our cloud or deploy in your own data center or cloud account. It provides an easy-to-use interface to using open source AI that's accessible to everyone. Under the hood, it uses the best open source models and includes a GPU scheduler that can fit model instances into GPU memory to optimally trade off user facing latency with GPU memory utilization. If you think this is cool, please vote for us on https://www.producthunt.com/posts/helix-5 https://www.producthunt.com/posts/helix-5 today. Docs: https://docs.helix.ml/docs/overview https://docs.helix.ml/docs/overview Architecture: https://docs.helix.ml/docs/architecture https://docs.helix.ml/docs/architecture Things to try with LLM fine-tuning using Helix: - https://docs.helix.ml/docs/papers https://docs.helix.ml/docs/papers - https://docs.helix.ml/docs/engaging-content https://docs.helix.ml/docs/engaging-content - https://docs.helix.ml/docs/insights-data https://docs.helix.ml/docs/insights-data - https://docs.helix.ml/docs/website-content https://docs.helix.ml/docs/website-content Sample sessions for SDXL: - https://app.tryhelix.ai/session/e1b50789-a209-46c8-aa60-4d097af1aa8b https://app.tryhelix.ai/session/e1b50789-a209-46c8-aa60-4d09... - https://app.tryhelix.ai/session/cc6004cd-111b-48ae-9a8c-d651f3ed45c8 https://app.tryhelix.ai/session/cc6004cd-111b-48ae-9a8c-d651... - https://app.tryhelix.ai/session/d50db369-4ffa-4a49-88dd-1cff05fee947 https://app.tryhelix.ai/session/d50db369-4ffa-4a49-88dd-1cff...
- leblancfg 3y agoThis is huge. Luke I think you have a winner here, this is great. Can't wait to try it over the holidays. If I can be cheeky, be sure to repost over the coming days at different hours – you're likely to spawn more traffic that way =)
- lewq 3y agoThank you! <3