3 ms·
IMHO we are at the point now where ambient computing is fully possible using locally self hosted AI and LLMs. I explore the ideas more in a post at https://mea
by com2kid 1y ago
IMHO we are at the point now where ambient computing is fully possible using locally self hosted AI and LLMs.
I explore the ideas more in a post at https://meanderingthoughts.hashnode.dev/lets-do-some-actual-innovative-sht https://meanderingthoughts.hashnode.dev/lets-do-some-actual-... but the tl;dr is ~24-32GB of VRAM in a server shoved in a basement can do a lot. Imagine a machine that is fully owned by the user with no corporate spying. It can listen 24/7 to everything in the house, using technology like wifi location sensing it knows what room people are in (it is a thing now!) and can relay messages for a person to the closest speaker.
Even 8B parameter LLMs are great at ambiguous inputs, and new TTS models are weighting in under 1B parameters.
Connecting the LLM up to everything a person wants to do is the real issue. Homekit integrations exist, but Homekit isn't exactly a mass consumer technology. What RabbitOS aims to do is IMHO the proper path: Drive an android phone to accomplish tasks.