HenryNdubuaku

4 ms·

1. 31.▲
   by HenryNdubuaku 1mo ago

   True! We finished Needle 3 before Jev launched. Also, we are merely chasing one DeepSeek v4 Flash capacity with a small model, DeepSeek models are really good.
2. 32.▲
   by HenryNdubuaku 1mo ago

   thanks for these haha, you can actually edit the tools and/or their descriptions, the demo is just a "get started" preset. But still we do have room for reasoning improvement!
3. 33.▲
   by HenryNdubuaku 1mo ago

   Fair, we gotta do a better job at explaining this properly! So an 8-27B on a LAN box wins for generic tasks on hardware that can hold it. Needle is for hardware that can't, like plain ARMv7, MIPS32 (the Ingenic chips in cheap IP camera
4. 34.▲
   by HenryNdubuaku 1mo ago

   Strong point! Needle is a task-specific model and bullet 6 stressed that it is only trained to be good on a set of narrow tasks, but I guess it could be clearer?
5. 35.▲
   by HenryNdubuaku 1mo ago

   thanks for this actually, so the demo is a preset, you can edit tools descriptions and add available tools the way you want, else Needle heavily guards against false negatives, users asked for this. For this, we will update presets on our e
6. 36.▲
   Show HN: Cactus Needle 3: 8-29MB automation models can match DeepSeek V4 Flash(cactuscompute.com)

   5 points by HenryNdubuaku 1mo ago|0 comments
7. 37.▲
   by HenryNdubuaku 2mo ago

   Ok we'd be adopting Little Language Model officially haha
8. 38.▲
   by HenryNdubuaku 2mo ago

   Now you can try running the python package directly on your Mac to see proper speedup. WebAssembly has some limitations, running on pure hardware, especially ARM-based to see thousands (which 99% of consumer devices are ARM).
9. 39.▲
   by HenryNdubuaku 2mo ago

   Please give it a try and let us know!
10. 40.▲
    by HenryNdubuaku 2mo ago

    Haha, my rent is cheap lol
11. 41.▲
    by HenryNdubuaku 2mo ago

    yes, that's what we had in mind while building
12. 42.▲
    by HenryNdubuaku 2mo ago

    Ok, this is genuinely funny, we will fix these as we iterate, thanks lol.
13. 43.▲
    by HenryNdubuaku 2mo ago

    Insane!
14. 44.▲
    by HenryNdubuaku 2mo ago

    Your thinking is correct haha
15. 45.▲
    by HenryNdubuaku 2mo ago

    The problem is the target device, an LLM can't run on an average TV well.
16. 46.▲
    by HenryNdubuaku 2mo ago

    Thanks! Let us know how it goes :)
17. 47.▲
    by HenryNdubuaku 2mo ago

    SO when confused, it gives you a low confidence score as a signal, in which case its recommended to finetune using th python package, train on your mac/PC
18. 48.▲
    by HenryNdubuaku 2mo ago

    You explained it better, we've done a bad job at communicating its nice :(
19. 49.▲
    by HenryNdubuaku 2mo ago

    Yep
20. 50.▲
    by HenryNdubuaku 2mo ago

    Exactly the use cases we designed Needle for!
21. 51.▲
    by HenryNdubuaku 2mo ago

    Thanks!
22. 52.▲
    by HenryNdubuaku 2mo ago

    7 for now, English, German, Polish, French, Dutch, Latin, Italin.
23. 53.▲
    by HenryNdubuaku 2mo ago

    A lot of Needle 1 users deployed on microcontrollers, we couldn't just throw them under the bus only t chase benchmarks, so its a price we are willing to pay :(
24. 54.▲
    by HenryNdubuaku 2mo ago

    Haha, its not a generalist model, cheap devices cannot run LLM in production, so we brig on-device intelligence to them in the size they can. Needle is niche, not a Gemma or GPT replacement.
25. 55.▲
    by HenryNdubuaku 2mo ago

    So TFLite is an inference engine while Needle is a model, which you can port on TFLite btw.
26. 56.▲
    by HenryNdubuaku 2mo ago

    thanks, give the playground a go and let us know how to improve!
27. 57.▲
    by HenryNdubuaku 2mo ago

    Yes, though for better results in production, after creating your tool json, use the provided data synthesis and fine-tuning pipeline. It tunes on on your mac.
28. 58.▲
    by HenryNdubuaku 2mo ago

    Ah, another failure point on our end! So a simple "5 + 7" and "add 5 and 7" works. But to handle ambiguity, the python package ships pipelines to synthesize augmentations and fine-tune on your samples for robustness. Jus
29. 59.▲
    by HenryNdubuaku 2mo ago

    That demo was Needle 1 indeed and we are creating the guide for ESP32 now as we speak.
30. 60.▲
    by HenryNdubuaku 2mo ago

    Thanks, we shall improve this for the next release.