4 ms·

I've been using the 35B-A3B today for some web scraping work, and it has been on par with Qwen3.8 27B at a much higher speed and at a higher quant (q4 vs q8). I

I've been using the 35B-A3B today for some web scraping work, and it has been on par with Qwen3.8 27B at a much higher speed and at a higher quant (q4 vs q8). I'm impressed.

jadbox 1mo ago
I need someone to run actual benchmarks between the two.
- swatcoder 1mo ago
  Benchmarks are the BMI of model evaluation. They may have utility in trying to look at the whole landscape of models, but are very misleading when it comes to making 1:1 comparisons or in developing confidence at to how a given model will deliver on your workflow.
  - dofm 1mo ago
    > Benchmarks are the BMI of model evaluation. That is such an elegant way to put it.
- NitpickLawyer 1mo ago
  Only relevant benchmarks are those you make yourself, targeted specifically for your workflows. Anything else is just number go up on a pretty graph, and every model out there is probably benchmaxxed to hell on the public ones anyway. Keep yours private.
- gertlabs 1mo ago
  These models have gotten a fair amount of attention -- we're hoping it's enough to get them added to some reliable inference providers and OpenRouter, at which point we'll run them on our full benchmark suite.
  - RomulusHill 1mo ago
    Hi Gertlabs, my inference company has actually started offering Ornith1.5 today for 9B and 35B A3B. If you are still interested, send me a message on X and I'll help you get started! https://x.com/romulushill https://x.com/romulushill https://scalattice.com/models/ornith-1.5-9b/ https://scalattice.com/models/ornith-1.5-9b/ https://scalattice.com/models/ornith-1.5-35b-a3b/ https://scalattice.com/models/ornith-1.5-35b-a3b/ https://scalattice.com/developers/ https://scalattice.com/developers/
jakswa 1mo ago
I had to go down to UD-Q3_K_XL for Qwen 3.8 27B to get it to fit in VRAM and be usable, but I worry I'm gutting its intelligence somewhat. I too am interested in faster + more-usable alternative that can exchange blows with the Q3-dumbed 27B.
mark_l_watson 1mo ago
What you say is in agreement with someone I follow on X. My family has been traveling for two weeks and I have been looking forward to trying qwen3.8-37b when I get home. Now I might try Ornith-1.5-35B first even though I don’t see an official MLX version.

I've been using the 35B-A3B today for some web scraping work, and it has been on par with Qwen3.8 27B at a much higher speed and at a higher quant (q4 vs q8). I'm impressed.

jadbox 1mo ago

I need someone to run actual benchmarks between the two.

swatcoder 1mo ago
Benchmarks are the BMI of model evaluation. They may have utility in trying to look at the whole landscape of models, but are very misleading when it comes to making 1:1 comparisons or in developing confidence at to how a given model will deliver on your workflow.
- dofm 1mo ago
  > Benchmarks are the BMI of model evaluation. That is such an elegant way to put it.
NitpickLawyer 1mo ago
Only relevant benchmarks are those you make yourself, targeted specifically for your workflows. Anything else is just number go up on a pretty graph, and every model out there is probably benchmaxxed to hell on the public ones anyway. Keep yours private.
gertlabs 1mo ago
These models have gotten a fair amount of attention -- we're hoping it's enough to get them added to some reliable inference providers and OpenRouter, at which point we'll run them on our full benchmark suite.
- RomulusHill 1mo ago
  Hi Gertlabs, my inference company has actually started offering Ornith1.5 today for 9B and 35B A3B. If you are still interested, send me a message on X and I'll help you get started! https://x.com/romulushill https://x.com/romulushill https://scalattice.com/models/ornith-1.5-9b/ https://scalattice.com/models/ornith-1.5-9b/ https://scalattice.com/models/ornith-1.5-35b-a3b/ https://scalattice.com/models/ornith-1.5-35b-a3b/ https://scalattice.com/developers/ https://scalattice.com/developers/

jakswa 1mo ago

I had to go down to UD-Q3_K_XL for Qwen 3.8 27B to get it to fit in VRAM and be usable, but I worry I'm gutting its intelligence somewhat. I too am interested in faster + more-usable alternative that can exchange blows with the Q3-dumbed 27B.

mark_l_watson 1mo ago

What you say is in agreement with someone I follow on X. My family has been traveling for two weeks and I have been looking forward to trying qwen3.8-37b when I get home. Now I might try Ornith-1.5-35B first even though I don’t see an official MLX version.