danielhanchen

27 ms·

91.
by danielhanchen 7mo ago
You would be surprised! Nearly every Fortune 500 company has utilized either our RL fine-tuning package or used our quants and models - the UI was primarily a culmination of pain points folks had when doing either training or inference! We&
92.
by danielhanchen 7mo ago
Yep UV is fantastic - should have just that default!
93.
by danielhanchen 7mo ago
Thank you! Great idea - that is actually one of the follow up goals! We know some folks want to do large scale training runs or inference with large models so the next most natural thing is to provide an API for folks and also provide infra
94.
by danielhanchen 7mo ago
Thank you for your continued support - we have much more planned for it!
95.
by danielhanchen 7mo ago
Thank you! We're still iterating on it so any suggestions are welcome!
96.
by danielhanchen 7mo ago
You would be surprised - we're the 4th largest independent distributor of LLMs in the world - and nearly every Fortune 500 company has utilized either our RL fine-tuning package or used our quants and models - we for example collab dir
97.
by danielhanchen 7mo ago
Yep - uv is a better fit - and you get parallel downloads as well
98.
by danielhanchen 7mo ago
Yep uv pip install unsloth works as well - we probably should have just made that the default - in fact Unsloth makes its own venv using UV if you have it dynamically
99.
by danielhanchen 7mo ago
Hey we're still working on making installation much better - appreciate the feedback! We come from Python land mainly so packaging and distribution is all very new to us - homebrew will definitely be next!
100.
Show HN: Unsloth Studio - Local Fine-tuning, Chat UI(github.com)
8 points by danielhanchen 7mo ago|2 comments
101.
by danielhanchen 7mo ago
Thanks! Oh we do split if over 50GB - do you mean also split on 50GB shards? HuggingFace XET has an interesting feature where each file is divided into blocks, so it'll do a SHA256 on each block, and only update blocks
102.
by danielhanchen 7mo ago
Oh https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks might be helpful - it provides benchmarks for Q4_K_XL vs Q4_K_M etc for disk space vs KL Divergence (proxy for how close to the original full precisio
103.
by danielhanchen 7mo ago
Function calling and also finetuning with FC is a big use-case across any companies - we constantly see large orgs have internal APIs with some schema, and JSON guided output is good, but finetuning with FC is just much more powerful since
104.
by danielhanchen 7mo ago
These are fair points considering LLMs are getting smarter and better every week - but to be fair the biggest benefits of finetuning / RL are still not yet realized: 1. If we have robots at home, they need some sort of efficient contin
105.
by danielhanchen 7mo ago
Oh I wrote up a post on X on this exact question! https://x.com/danielhanchen/status/1979389893165060345?s=20 1. Cursor used online RL to get +28% approval rate: https://cursor.com/blog/tab-rl
106.
by danielhanchen 7mo ago
Oh I think on X a few ago: https://x.com/poezhao0605/status/2029151951167078454 - Alibaba's CEO and CTO are having an emergency all hands now! Hope it all goes well!
107.
by danielhanchen 7mo ago
Excited for it!
108.
by danielhanchen 8mo ago
You forgot to check the disk sapce - _M and _XL are not the same across quants: Unsloth Q4_K_M 18.49GB 0.5478 KLD 99.9% 0.0192 mean Unsloth Q4_K_XL 19.17GB 0.4097 KLD 99.9% 0.0137 mean bartowski Q4_K_M 19.77GB 0.5771 KLD 99.9% 0.0182
109.
by danielhanchen 8mo ago
No this is false - unsure if you saw our new blog - https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks which shows SOTA on nearly all bits, and we shared all our research as well
110.
by danielhanchen 8mo ago
Yes the new blog post https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks has some benchmarks from community people on our quants vs others on LiveCodeBench for eg!
111.
by danielhanchen 8mo ago
Didn't expect this to be on HN haha - but sometimes HN does have older posts come up sometimes. No your conclusion is false - only the old Q4_K_XL had slightly higher perplexity, all other quants are fine. We uploaded 9TB of research a
112.
by danielhanchen 8mo ago
No our Qwen3.5 new ones show the opposite see https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks
113.
by danielhanchen 8mo ago
Didn't expect this as well haha on HN again - probably related to Qwen3.5
114.
by danielhanchen 8mo ago
Thank you!
115.
by danielhanchen 8mo ago
Oh I didn't expect this to be on HN haha - but yes for our new benchmarks for Qwen3.5, we devised a slightly different approach for quantization which we plan to roll out to all new models from now on!
116.
by danielhanchen 8mo ago
Oh it's more time that's the issue - each benchmark takes 1-3 hours ish to run on 8 GPUs, so running on all quants per model release can be quite painful. Assume AWS spot say $20/hr B200 for 8 GPUs, then $20 ish per quant, so
117.
by danielhanchen 8mo ago
Appreciate it immensely haha :) Never tired - always excited and pumped for this year!
118.
by danielhanchen 8mo ago
Working on it! :)
119.
by danielhanchen 8mo ago
Yes sadly very expensive :( Maybe a select few quants could happen - we're still figuring out what is the most economical and most efficient way to benchmark!
120.
by danielhanchen 8mo ago
Thanks! Yes we actually did think about that - it can get quite expensive sadly - perplexity benchmarks over short context lengths with small datasets are doable, but it's not an accurate measure sadly. We're actually investigatin

More ›