4 ms·

This gave me lots of confidence in Unsloth when I first read it. I'll admit I was a little skeptical of Unsloth, since anything that boasts free perf improveme

This gave me lots of confidence in Unsloth when I first read it. I'll admit I was a little skeptical of Unsloth, since anything that boasts free perf improvement, just by dropping in some middleware, makes me suspicious. Especially from such a small team. I assumed it was just introducing some hacks that create an inexact implementation of attention or some faster-but-inaccurate cuda kernels or something. But now I believe this small team really knows their stuff :)

GaggiX 3y ago
The article mentioned in the comment: https://unsloth.ai/blog/gemma-bugs https://unsloth.ai/blog/gemma-bugs
- danielhanchen 3y ago
  Whoops I think I might have forgotten to add it to the Colab!!
theaussiestew 3y ago
The founder I know personally, he interned at Nvidia and contributed many performance improvements, he's the real deal - just really enthuasitic so it may come off as boastfulness ;)
- danielhanchen 3y ago
  Apologies on the enthusiasm!! :) And hi!!
  - unrahul 3y ago
    Daniel is of the best engineers I have ever worked with. Engineer in the true sense of wanting to know how something works and figuring out ways to improve it !
    - danielhanchen 3y ago
      Oh thanks!! Super high praise! :)
    - deleted 3y ago
      [deleted]
  - theaussiestew 3y ago
    No need to apologise, it's a great trait to have :)
    - danielhanchen 3y ago
      :)
bugglebeetle 3y ago
They’ve had their work applauded by Karpathy and Jeremy P. Howard as well, which are about the best credentials you could ever get for open source AI stuff: https://twitter.com/karpathy/status/1765473722985771335 https://twitter.com/karpathy/status/1765473722985771335 I’ve been using the library since it started out and it works really well. Daniel is also super helpful and responsive in their Discord, assisting everyone from the most basic users to breaking down complex ML math stuff.
- lamroger 3y ago
  real recognizing real
  - danielhanchen 3y ago
    :)
- danielhanchen 3y ago
  Thanks to Andrej and Jeremy as well :) And also thanks to community members like you! It makes me super happy to keep making Unsloth better so appreciate it a lot!
danielhanchen 3y ago
Oh thanks! I get that a lot :) But ye there's no approximations at all! Just special maths hacks with no degradations, rewriting everything and creating a custom backprop engine, sprinkling Triton / CUDA everywhere and more :) But thanks you believe in me + my bro more :) Appreciate it a lot!

GaggiX 3y ago

The article mentioned in the comment: https://unsloth.ai/blog/gemma-bugs https://unsloth.ai/blog/gemma-bugs

danielhanchen 3y ago
Whoops I think I might have forgotten to add it to the Colab!!

theaussiestew 3y ago

The founder I know personally, he interned at Nvidia and contributed many performance improvements, he's the real deal - just really enthuasitic so it may come off as boastfulness ;)

danielhanchen 3y ago
Apologies on the enthusiasm!! :) And hi!!
- unrahul 3y ago
  Daniel is of the best engineers I have ever worked with. Engineer in the true sense of wanting to know how something works and figuring out ways to improve it !
  - danielhanchen 3y ago
    Oh thanks!! Super high praise! :)
  - deleted 3y ago
    [deleted]
- theaussiestew 3y ago
  No need to apologise, it's a great trait to have :)
  - danielhanchen 3y ago
    :)

bugglebeetle 3y ago

They’ve had their work applauded by Karpathy and Jeremy P. Howard as well, which are about the best credentials you could ever get for open source AI stuff: https://twitter.com/karpathy/status/1765473722985771335 https://twitter.com/karpathy/status/1765473722985771335 I’ve been using the library since it started out and it works really well. Daniel is also super helpful and responsive in their Discord, assisting everyone from the most basic users to breaking down complex ML math stuff.

lamroger 3y ago
real recognizing real
- danielhanchen 3y ago
  :)
danielhanchen 3y ago
Thanks to Andrej and Jeremy as well :) And also thanks to community members like you! It makes me super happy to keep making Unsloth better so appreciate it a lot!

danielhanchen 3y ago

Oh thanks! I get that a lot :) But ye there's no approximations at all! Just special maths hacks with no degradations, rewriting everything and creating a custom backprop engine, sprinkling Triton / CUDA everywhere and more :) But thanks you believe in me + my bro more :) Appreciate it a lot!