5 ms·
> Two years ago when I first tried LLaMA I never dreamed that the same laptop I was using then would one day be able to run models with capabilities as strong a
by NitpickLawyer 1y ago
> Two years ago when I first tried LLaMA I never dreamed that the same laptop I was using then would one day be able to run models with capabilities as strong as what I’m seeing from GLM 4.5 Air—and Mistral 3.2 Small, and Gemma 3, and Qwen 3, and a host of other high quality models that have emerged over the past six months.
Yes, the open-models have surpassed my expectations in both quality and speed of release. For a bit of context, when chatgpt launched in Dec22, the "best" open models were GPT-J(~6-7B) and GPT-neoX (~22B?). I actually had an app running live, with users, using gpt-j for ~1 month. It was a pain. The quality was abysmal, there was no instruction following (you had to start your prompt like a story, or come up with a bunch of examples and hope the model will follow along) and so on.
And then something happened, LLama models got "leaked" (I still think it was a on purpose leak - don't sue us, we never meant to release, etc), and the rest is history. With L1 we got lots of optimisations like quantised models, fine-tuning and so on, L2 really saw fine-tuning go off (most of the fine-tunes were better than what meta released), we got alpaca showing off LoRA, and then a bunch of really strong models came out (mistrals, mixtrals, L3, gemmas, qwens, deepseeks, glms, granites, etc.)
By some estimations the open models are ~6mo behind what SotA labs have released. (note that doesn't mean the labs are releasing their best models, it's likely they keep those in house to use on next runs data curation, synthetic datasets, for distilling, etc). Being 6mo behind is NUTS! I never in my wildest dreams believed we'll be here. In fact I thought it would take ~2years to reach gpt3.5 levels. It's really something insane that we get to play with these models "locally", fine-tune them and so on.
- tonyhart7 1y agois GLM 4.5 better than Qwen3 coder??
- diggan 1y agoFor what? It's really hard to say what model is "generally" better then another, as they're all better/worse at specific things. My own benchmarks has a bunch of different tasks I use various local models for, and I run it when I wanna see if a new model is better than the existing ones I use. The output is basically a markdown table with a description of which model is best for what task. They're being sold as general purpose things that are better/worse at everything but reality doesn't reflect this, they all have very specific tasks they're worse/better at, and the only way to find that out is by having a private benchmark you run yourself.
- kelvinjps10 1y agocoding? they are coding models? what specific tasks is one performing better than the other?
- diggan 1y agoThey may be, but there are lots of languages, lots of approaches, lots of methodologies and just a ton of different ways to "code", coding isn't one homogeneous activity that one model beats all the other models at. > what specific tasks is one performing better than the other? That's exactly why you create your own benchmark, so you can figure that out by just having a list of models, instead of testing each individually and basing it on "feels better".
- reverius42 1y ago> coding isn't one homogeneous activity that one model beats all the other models at If you can't even replace one coding model with another, it's hard to imagine you can replace human coders with coding models.
- diggan 1y agoWhat you mean "can't even replace"? You can, nothing in my comment says you cannot?
- Philpax 1y agoYou probably can't replace a seasoned COBOL programmer with a seasoned Haskell programmer. Does that mean that either person is bad at programming as a whole?
- reverius42 1y agoThis was my point -- if programmers are not fungible, how can companies claim to be replacing them by the thousands with AI?
- NitpickLawyer 1y agoI haven't tried them (released yesterday I think?). The benchmarks look good (similar I'd say) but that's not saying much these days. The best test you can do is have a couple of cases that match your needs, and run them yourself w/ the cradle that you are using (aider, cline, roo, any of the CLI tools, etc). Openrouter usually has them up soon after launch, and you can run a quick test really cheap (and only deal with one provider for billing & stuff).
- genewitch 1y agoI'll bite. How do i train/make and/or use LoRA, or, separately, how do i fine-tune? I've been asking this for months, and no one has a decent answer. websearch on my end is seo/geo-spam, with no real instructions. I know how to make an SD LoRA, and use it. I've known how to do that for 2 years. So what's the big secret about LLM LoRA?
- minimaxir 1y agoIf you're using Hugging Face transformers, the library you want to use is peft: https://huggingface.co/docs/peft/en/quicktour https://huggingface.co/docs/peft/en/quicktour There are Colab Notebook tutorials around training models with it as well.
- notpublic 1y agohttps://github.com/unslothai/unsloth https://github.com/unslothai/unsloth I'm not sure if it contains exactly what you're looking for, but it includes several resources and notebooks related to fine-tuning LLMs (including LoRA) that I found useful.
- techwizrd 1y agoWe have been fine-tuning models using Axolotl and Unsloth, with a slight preference for Axolotl. Check out the docs [0] and fine-tune or quantize your first model. There is a lot to be learned in this space, but it's exciting. 0: https://axolotl.ai/ https://axolotl.ai/ and https://docs.axolotl.ai/ https://docs.axolotl.ai/
- syntaxing 1y agoWhat hardware do you train on using axolotl? I use unsloth with Google colab pro
- arkmm 1y agoWhen do you think fine tuning is worth it over prompt engineering a base model? I imagine with the finetunes you have to worry about self-hosting, model utilization, and then also retraining the model as new base models come out. I'm curious under what circumstances you've found that the benefits outweigh the downsides.
- Nesco 1y agoZuck wouldn’t have leaked it on 4chan of all the places
- tough 1y agoprob just told an employee to get it done no?
- vaenaes 1y agoWhy not?
- eckelhesten 1y agoIt got leaked as a PR with an url to a magnet (torrent) afaik.
- throwzasdf 1y ago[dead]