4 ms·
My hope was to find a system which does ASR, then LLM processing with MCP use and finally TTS: "Put X on my todo list" / "Mark X as done" -> LLM thinks, reads t
by qwertox 1y ago
My hope was to find a system which does ASR, then LLM processing with MCP use and finally TTS: "Put X on my todo list" / "Mark X as done" -> LLM thinks, reads the todo list, edits the todo list, and tells me "I added X to your todo list", ... "Turn all the lights off" -> llm thinks and uses MCP to turn off the lights -> "Lights have been turned off". "Send me an email at 8pm reminding me to do" .... "Email has been scheduled for 8pm"
That's all I want. It does not have to be fast, but it must be capable of doing all of that.
Oh, and it should be energy efficient. Very important for a 24/7 machine.
- bayindirh 1y agoEnergy efficient LLM inference (for now) is as realistic as existence of perpetual motion.
- mindcrash 1y agoYou can already do that on most desktop GPU's (even going as far as prev gen Nv 1050/1060/1070 for example). You'll need a model able to work with tools, like llama 3.2 (https://huggingface.co/meta-llama https://huggingface.co/meta-llama), serve it, hook up MCPs, include a STT interface, and you're cooking.
- bayindirh 1y agoEven a bottom of the barrel N95 has audio acceleration features helping with speech to text, but the LLM inference part still will be far from being efficient. Plus, you need to keep the card at "ready" state, you can't idle/standby it completely.
- nl 1y agoYou probably can do this now. Non-generative LLMs don't need to be as big so something like Gemma 4B on the CPU will work. You may have better results with semi-templated responses though.