3 ms·
It is much faster though. On my m1 max, describing a picture (quick way to get a pretty large context): Qwen 3.6 35b a3b: 34 tok/sec Qwen 3.5 27b: 10 tok/sec
by spwa4 7mo ago
It is much faster though. On my m1 max, describing a picture (quick way to get a pretty large context):
Qwen 3.6 35b a3b: 34 tok/sec
Qwen 3.5 27b: 10 tok/sec
Qwen 3.5 35b a3b: doesn't support image input
- upboundspiral 7mo agoI've been using Qwen 3.5 35B-A3B with images as input so I suspect you perhaps didn't include the vision part of the model during testing (I use llama.cpp and I learned I needed to include the separate mmproj part).
- m-emre 7mo agoWhat is the quantization level of your Owen 3.6 3b model?