danielhanchen

29 ms·

631.
by danielhanchen 3y ago
Interesting! I did have some chats from interested people who have say 2 or 4 or 8 GPUs who are willing to somehow buy the Pro plan - the issue is now how do we price it, do we make a training platform? etc
632.
by danielhanchen 3y ago
Sorry about 1080 Ti - sadly Triton and Xformers support Cuda 7.0, so unless if OpenAI and Meta makes it support Cuda 6.0, we sadly don't support it. The main reason is because from Turing, Tensor Cores got provided, and so the matrix m
633.
by danielhanchen 3y ago
I agree fully - what do you suggest then? OSS the entire code base and using AGPL3? I tried that with https://github.com/danielhanchen/hyperlearn to no avail - we couldn't even monetize it at all, so I just OSSed
634.
by danielhanchen 3y ago
Sorry about the confusion - the 30X is comparing QLora with QLora - all benchmarking is QLoRa bsz=2, ga=4
635.
by danielhanchen 3y ago
Sorry about that - I'm super new to pricing and stuff so it might seem off since I'm literally making the plans with my bro as we go along. If you don't believe the timings, I was the author of Hyperlearn https://g
636.
by danielhanchen 3y ago
Good point - we thought about this - we're still figuring out the pricing as we go along - so all suggestions are welcome! I'm all very new to this, so I'm literally making stuff as I go along!! Apologies!
637.
by danielhanchen 3y ago
Thanks a bunch!! Super appreciate it!
638.
by danielhanchen 3y ago
Llama is generally OSS, except for some gating to large companies - but as a first try I made it work for Llama since the architecture is replicated in other models like Mistral or Yi.
639.
by danielhanchen 3y ago
Interesting!! I'll talk with my brother about this - sounds very cool!
640.
by danielhanchen 3y ago
Do you mean http://www.unsloth.ai/ doesn't work? My brother is working on it as we speak! Edit: it seems like http://unsloth.ai works - we're talking with our web hosting people to fix it
641.
by danielhanchen 3y ago
There's more as well!! I agree hypothetically in a few years people will catch up - we're well aware of that! But we made this into a startup, and so this was just one product we were releasing! We're going to be making many
642.
by danielhanchen 3y ago
P100 - oh my so Xformers for Flash Attention I think does support it, but Triton supports Compute Capability 7.0+, whilst a P100 is 6.0 :( So technically the code can run, but I'll have to edit it to remove the Triton changes.
643.
by danielhanchen 3y ago
Ohhh interesting! So like ML Perf or Faster CPython https://github.com/faster-cpython/benchmarking-public ?
644.
by danielhanchen 3y ago
Hey! Oh noo we don't do any XOR chain optimizations or Cauchy Reed Solomon computations - I had to wikipedia it and learn about them - mroe reading to do! It's mainly trying to reduce data movement, maximize FLOP utilization via m
645.
by danielhanchen 3y ago
M1 support is probably coming in the future if there is enough support - could you make an Issue at https://github.com/unslothai/unsloth - that would be much appreciated! I think there are a few Redditors from /r&
646.
by danielhanchen 3y ago
Thanks!! My brother came up with the name!! "Un-sloth" where sloth is the slow cute animal, and it also means lazy and slow, and "un" reverses it!
647.
by danielhanchen 3y ago
Hey! Great question! That's what I'm confused about as well! So in GPUs the goal is to saturate the GPU with matrix multiplies instead of data movement. I'll write a more detailed blog but approximately: 1. Flash Attention v2
648.
by danielhanchen 3y ago
Hey there! Yes that's exactly what I thought! I was writing up a blog post at https://colab.research.google.com/drive/1AOuhMVILE06mD-Go7-R... which step by step shows every change I did plus gave timings / me
649.
Show HN: 80% faster, 50% less memory, 0% loss of accuracy Llama finetuning(github.com)
385 points by danielhanchen 3y ago|119 comments
650.
Finetune language models 30x faster(unsloth.ai)
2 points by danielhanchen 3y ago|0 comments