8 ms·
Set up your own benchmark test. Run several models through meaningful workflows. I run local models every day and the harness is key for squeezing performance o
by spottedmarley 3mo ago
Set up your own benchmark test. Run several models through meaningful workflows. I run local models every day and the harness is key for squeezing performance out of smaller models. I'd estimate that, depending on the work you're doing of course, you could get Qwen3.5:27B to within 80% code quality if you spent some some time fine tuning your harness specifically to that model and your workflow. It would be a lot of trial and error testing, and I've done a ton of it, but the results are pretty nice. I use mostly Qwen3.5:122b-a10b but its the same family and the harness I've developed for it has made a huge difference as it has evolved over time.
- llmdreamer 3mo agoHow do you fine-tune, and at what rank? I tried fine-tuning Gemma4 and realized it broke the model, which then started returning gibberish.
- spottedmarley 3mo agoThe only fine tune I've attempted is a Qwen3.5:0.8b trained on RL data generated using Qwen3.5:122b playing 50-thousand hands of Hearts (the card game) because I wanted a tiny model that was, idealy, an expert Hearts player so I could hopefully get a super-fast local model that I could play hearts against without any per-turn lag. I wanted more than 50k hands but even just 50k took almost 3 days of RL running on a RunPod RTX 4090. Used the RL data to build a LoRa adapter for Qwen3.5:0.8b -- ultimately a failure but mostly because the data we generated wasn't of sufficient quality. Turns out hearts is a very complex game as played by sentient humans lol.
- llmdreamer 3mo agoThat sounds fun. I am surprised LLM just didnt memorize the hands.