aarnphm

4 ms·

1. 1.▲
   Show HN: Building compiler from scratch without the help of LLMs [video](youtube.com)

   2 points by aarnphm 6mo ago|0 comments
2. 2.▲
   Moving to Cloudflare(aarnphm.xyz)

   4 points by aarnphm 1y ago|0 comments
3. 3.▲
   by aarnphm 1y ago

   Hi srameshc, the core of BentoML is still considered MLOps. A lot of our customers are pretty much MLOps users. However, LLMOps seem like a natural progression of the product, given that a lot of our users now want to experiment/build
4. 4.▲
   by aarnphm 1y ago

   Thanks for the recommendation, I'm actually working on something similar for this part of the docs (I'm also working at BentoML).
5. 5.▲
   Structured decoding in LLMs, a guide for the impatient(aarnphm.xyz)

   2 points by aarnphm 2y ago|0 comments
6. 6.▲
   by aarnphm 3y ago

   You can pretty much make the same argument about Docker. Docker abstracts away runc, runc abstract away cgroup. I don't think calling it abstraction is correct. OneDiffusion is designed to be opinionated and help users to run diffusion
7. 7.▲
   OneDiffusion(github.com)

   39 points by aarnphm 3y ago|7 comments
8. 8.▲
   by aarnphm 3y ago

   Currently on main, 8bit and 4bit quant is supported One can simply do ```openllm start falcon --model-id tiiuae/falcon-40b-instruct --quantize int4``` Beware that there is no free lunch, meaning the quality of inference will degrade by
9. 9.▲
   by aarnphm 3y ago

   Hi there, 8bit and 4bit is currently supported on main. GPTQ is working in progress, as well as GGML
10. 10.▲
    by aarnphm 3y ago

    Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our
11. 11.▲
    by aarnphm 4y ago

    Again?? I definitely have to try out BentoML then to see what the hype is about!!
12. 12.▲
    by aarnphm 4y ago

    BentoML is an amazing tool that allows you to quickly and easily deploy your machine learning models as APIs. It is extremely user-friendly and easy to use, and the results are amazing. I highly recommend BentoML for anyone who wants to dep