3 ms·
Show HN: Gradient – a web API for fine-tuning and deploying Llama2
We’ve just launched Gradient — an API that helps you build private LLMs that you own. We simplify inference and fine-tuning on open-source LLMs such as llama2, and you only pay by the token.
Our API platform makes it possible for you to create private models with a single API call. Run inference on your fine tuned model instantly with no cold boot (and no need to pay for compute costs).
The product is truly on demand - when you run fine tuning and inference on our platform, there's nearly 0 startup latency for these API calls. And you're not paying for the compute, you just pay for the tokens you're consuming.
This makes it possible for developers to build and serve nearly unlimited fine tuned models without incurring ridiculous infrastructure fees.
We currently support Llama2 7B and Nous Hermes2 (Llama2 13B unlocked variant), and are releasing LlamaCoder and Llama2 70b in a few weeks.
Gradient is also SOC 2 and HIPAA compliant.
When you sign up, we give you $10 in free credits to start experimenting. We'd love to get your feedback, and let us know what you're building on our Discord!
Sign up (https://gradient.ai/ https://gradient.ai/)
Twitter (https://twitter.com/Gradient_AI_ https://twitter.com/Gradient_AI_)
Discord (https://discord.gg/yvgVKEgkmd https://discord.gg/yvgVKEgkmd)
- azirbel 3y agoThis looks great. I found the docs helpful for getting a clear idea of what it does: https://docs.gradient.ai/docs/introduction https://docs.gradient.ai/docs/introduction
- chrchang510 3y agoThanks - let us know what you build on our platform!
- rchin57 3y agoGreat job!
- yangbo 3y agoI am a developer from Gradient. I think the most interesting use case of our product is interactive few-shot fine-tuning in a Jupyter Notebook. When fine-tuning a model with less than 10 samples and around 3 epoch, it can learn knowledge in a couple of seconds, then you can immediately get the fine-tuned model. The experience is like to be your model's private tutor. It just continuously learns new things you told it, like a life. Be cautious about your learning rate, or it will go mad.
- ernest3 3y agoAs someone who has spent way too much time fine-tuning model parameters, I am grateful for this service!
- unicornsprinkle 3y agoAmazing solution for API latency, love that the product comes with regulation compliance!
- Jp6hBY06 3y agoGreat stuff. Looking forward to learning more!
- xiaoqiao10 3y agoLooks awesome! Congrats on the launch