HenryNdubuaku

7 ms·

1. 151.▲
   by HenryNdubuaku 1y ago

   Thanks for pointing this out, another factor for us to figure out. We waive the license for such cases, wanna get in touch? I don’t think your consumers have to worry about the license.
2. 152.▲
   by HenryNdubuaku 1y ago

   We are focused on phones and we did add some benchmarks and will add more. However, anyone can see performance for themselves with the repo directly.
3. 153.▲
   by HenryNdubuaku 1y ago

   Although GPT OSS 20B has 1.7B activated parameters which will be fast, 20B weights is a lot for developers to bundle or consumers to download. That’s the actual problem.
4. 154.▲
   by HenryNdubuaku 1y ago

   Real-time video and audio inference.
5. 155.▲
   by HenryNdubuaku 1y ago

   The license change doesn’t affect you based on your explanation actually, the licence has been updated with clearer words. We really appreciate you as a user, please share any more feedback you have, thanks.
6. 156.▲
   by HenryNdubuaku 1y ago

   Done, thanks, let us know anything else.
7. 157.▲
   by HenryNdubuaku 1y ago

   This was one of the issues we set out to solve, so not as much as you’d expect.
8. 158.▲
   by HenryNdubuaku 1y ago

   It’s absolutely fine to share your thoughts, that’s the point of this post, we want to understand where people’s heads are at, it’s what determines our next decisions. What do you really think? I’m genuinely asking so I don’t think mods wil
9. 159.▲
   by HenryNdubuaku 1y ago

   Thanks for sharing your thoughts. Honestly, I’d be annoyed too and it might sound like an excuse, but our circumstance was quite unique, it was a difficult decision at that time being an open-source contributor myself. It’s still free for t
10. 160.▲
    by HenryNdubuaku 1y ago

    Understandable, though to explain, Cactus is still free for personal & small projects if you fall into that category. We’re early and would definitely consider your concerns on license in our next steps, thanks.
11. 161.▲
    by HenryNdubuaku 1y ago

    It can incorporate any tool you want at all. This company’s app use exactly that feature, you can download and get a sense of it before digging in. https://anythingllm.com/mobile
12. 162.▲
    by HenryNdubuaku 1y ago

    Thanks for noticing! The app is just a demo for the framework, so devs can compare the open-source models against frontier Cloud models and make a decision. We removed the comparison now so those screenshots indeed has to be updated.
13. 163.▲
    by HenryNdubuaku 1y ago

    Thanks for the kind words, we’ve improved performance now actually, follow the instructions on the core repo. Same model should run 3x faster on the same phone. These improvements are still being pushed to the SDKs though.
14. 164.▲
    by HenryNdubuaku 1y ago

    Cactus is free for hobbyists and personal projects, but we charge a tiny fee for commercial use which comes with more features that are relevant for enterprises.
15. 165.▲
    by HenryNdubuaku 1y ago

    400mb if you ship the model as an asset. However, you can also build the app to download the model post-install, Cactus SDKs support this, as well as agentic workflows you’d need.
16. 166.▲
    Launch HN: Cactus (YC S25) – AI inference on smartphones(github.com)

    123 points by HenryNdubuaku 1y ago|63 comments
17. 167.▲
    by HenryNdubuaku 1y ago

    Thanks!
18. 168.▲
    by HenryNdubuaku 1y ago

    [flagged]
19. 169.▲
    by HenryNdubuaku 1y ago

    We are writing our own backend, but tflite (now called LiteRT) was not faster than GGML when we tested and GGML is already well supported. But we are moving away completely anyway.
20. 170.▲
    by HenryNdubuaku 1y ago

    We are following Ollama's design, but not verbatim due to apps being sandboxed. Phones are resource-constrained, we saw significant battery overhead with in-process HTTP listeners so we stuck with simple stateful isolates in Flutter an
21. 171.▲
    by HenryNdubuaku 1y ago

    Thanks for the comment, but: 1) The commit history goes back to April. 2) LlaMa.cpp licence is included in the Repo where necessary like Ollama, until it is deprecated. 3) Flutter isolates behave like servers, and Cactus codes use that.
22. 172.▲
    by HenryNdubuaku 1y ago

    Please feel free to join our Discord: https://discord.com/invite/bNurx3AXTJ
23. 173.▲
    Show HN: Cactus – Ollama for Smartphones(github.com)

    231 points by HenryNdubuaku 1y ago|82 comments