5 ms·
Show HN: Cactus Needle 3: 8-29MB automation models can match DeepSeek V4 Flash
Hey HN, Henry from Cactus here.
We submitted Needle 2 here a few weeks ago, and the feedback in the discussion thread was incredibly valuable, thanks! Thanks to all that feedback, we’ve been able to move quickly to release Needle 3 and I'd love to hear what you think again.
The key features:
1) Automation (tool calls & structured JSON output): Needle still doesn't chat by design, its quite challenging to pack general capacity into such small models, so we focus on tool calls and structured JSON. If no tool you declared fits the request, you get an empty list back (note for when playing with the demo).
2) Intelligence Laddering: Every layer (2 to 20) is a deployable subnetwork, so one set of weights, 25 to 121 million parameters at 2-bit, shipping as 8-29MB binaries. On a Raspberry Pi 5 it decodes at up to 4k tokens/sec and prefills at up to 10k.
3) Monarch Hadamard MLP: replaces the dense FFN with three learnable Walsh-Hadamard-initialized Kronecker (Monarch) factor pairs interleaved with per-channel diagonal scales, fixed permutations, a SiLU nonlinearity, and a rank-8 input-conditioned gate, so each token gets a fully mixed nonlinear transform of its d_model channels at O(d√d) parameters and compute instead of the O(d²) a dense 4x-expansion MLP would cost.
4) Performance: On Mobile Actions (phone commands, scored on the exact call) the 20-layer model gets 86.0 through the shipped 2-bit binary; LFM2.5 1.2B is at 82.4, Qwen3.5 0.8B at 76.0, Apple's on-device model at 57.6, all at f16. More results on the link, we do not win everywhere ofc.
5) Multilingual: Needle 3 now supports English, French, Spanish, German, Dutch, Italian, Polish, with more languages coming.
6) Finetuning: You can achieve DeepSeek v4 Flash grade performance on a narrow task with just 4L, stress on "narrow task", we found that production users often prefer tuning before production.
7) Triggers: Grounding is a common challenge for tool call, at least for Needle 2, so we added support case-insensitive regular expressions matched against each request to gate false negatives.
8) Confidence: Every response also carries a calibrated confidence score, the minimum of a judgement on the finished call and its decode probability. Act above your threshold, show the call and ask below it, or escalate to a bigger model.
9) Supported Platforms: macOS, Linux on x86-64, ARM64, ARMv7, RISC-V and MIPS32, Windows x64 and ARM, Android, iOS, watchOS, tvOS, the browser as WebAssembly, and a WASI component.
Thanks for reading and as always, thoughts appreciated!
- mihau 1mo agoNone of the queries I asked worked: - "more light" - "less light" - "both doors should be locked" - "if blinds are open, open back door"
- HenryNdubuaku 1mo agothanks for this actually, so the demo is a preset, you can edit tools descriptions and add available tools the way you want, else Needle heavily guards against false negatives, users asked for this. For this, we will update presets on our end.
- HenryNdubuaku 1mo agopresets updated for you now :)
- hirako2000 1mo agoMy thought, the growing number of dubious claims that a tiny model beats LLMs will make any useful innovation be overlooked. What's more important than the resource requirements is to highlight what the model simply cannot even attempt to do that general LLMs do decently well. In other words, tell me the anti use case clearly so that I don't have to find out myself.
- HenryNdubuaku 1mo agoStrong point! Needle is a task-specific model and bullet 6 stressed that it is only trained to be good on a set of narrow tasks, but I guess it could be clearer?
- owebmaster 1mo agoIt would be clearer if you didn't use AI to reply.
- janalsncm 1mo agoAn LLM is a Swiss Army knife. This is a corkscrew. All of the other tasks a general-purpose LLM can do (write me a poem about pizza, rewrite this code in rust, tell me about the causes of the war of the roses) are unsupported. The only use case this supports is converting unstructured text into structured json calls, and doing that quickly in a low memory environment.