3 ms·
Very nice, good work. I think you should add the chat app links on the readme, so that visitors get a good idea of what the framework is capable of. The perfor
by politelemon 1y ago
Very nice, good work. I think you should add the chat app links on the readme, so that visitors get a good idea of what the framework is capable of.
The performance is quite good, even on CPU.
However I'm now trying it on a pixel, and it's not using GPU if I enable it.
I do like this idea as I've been running models in termux until now.
Is the plan to make this app something similar to lmstudio for phones?
- rshemet 1y agoappreciate the feedback! Made the demo links more prominent on the README. Some Android models won't support GPU hardware; we'll be addressing that as we move to our own kernels. The app itself is just a demonstration of Cactus performance. The underlying framework gives you the tools to build any local mobile AI experience you'd like.