2 ms·
You may well be able to do this on-device right now. The latest local models are all capable of tool calls and summarization, which is what this demo needs. A
by simonw 1y ago
You may well be able to do this on-device right now. The latest local models are all capable of tool calls and summarization, which is what this demo needs.
A decent sized Qwen 3 or Gemma 3 might well do the job. Mistral 3.1 Small is great too.
(Personally I'd be happy to pipe my email through an LLM from a respectable vendor that promises not to train on my inputs - Anthropic, OpenAI and Gemini all promise that for paid API usage.)
- colechristensen 1y agoToday I put together a demo of gemma3 27b parameter running locally looking through my photo library for event fliers, it extracts the information satisfactorily well. With some enhancement I expect it will be quite useful.
- paffdragon 1y agoI think, I need to buy new HW maybe. My 12 core 32GRAM laptop is running these local models so slowly, it's unusable (I do have an Nvidia card in it as well, but I ended up disabling due to issues under Wayland/wlroots and didn't have time to fix that yet). And most of my phone's advanced AI features won't work when only on-device processing is allowed.