4 ms·

I'm not against idea of having LLM be smart assistant for home, but I do have problem with sending any of that to cloud. It's one thing to use it as convenient

by ilyt 3y ago

I'm not against idea of having LLM be smart assistant for home, but I do have problem with sending any of that to cloud. It's one thing to use it as convenient way to change anything remotely but linking home automation not only into constant internet connectivity but also "that one particular cloud thing that might disappear at any moment" (hello google) just seems like setting yourself for problems later. At least in this case he's still left with HASS if cloudy part goes away. But I wish there was more push into hybrid model - like have your router or small ARM box be server that runs queue and some, for lack of better word, "lambda-like" code that handles most of the programmed events (lights, heating etc.), say via WASM and some APIs around it, then just have HASS-like software (both cloud and local) deploy control rules/code onto it. Big nice UI for control goes down ? Doesn't matter, the controller is just running simple code, not rest of the visualisation and controls. Want to replace it ? Configuration as code makes that a breeze, hell, have cloud backup of whole setup. Don't want controller ? It's essentially just a queue and some code runners, that cloud providers can host for you. Want pretty graphs in cloud or some aggregation ? Just make your local node filter and send relevant MQ events there.

nine_k 3y ago
"Internet of Things will not happen until LAN of Things arrives", to paraphrase someone. This already happens in the industry, like heavy industry. At home, it's held back by absence of an adequate server. Often a router box or a NAS would be fine to host something like OwnCloud. But to run ML middle, different and much more expensive hardware is required, which would sit idle 99% of time.
- kingo55 3y ago
  LAN of things - this is a fascinating idea. Can you share where you are paraphrasing this from or link to more info on this concept? Sounds familiar with the self hosted applications people now run on their NAS/routers these days.
  - c_o_n_v_e_x 3y ago
    The phrase is used quite a bit in hackernews comments, particularly on any IoT or home automation threads.
  - ilyt 3y ago
    The concept is nothing new, it's just not something manufacturers picked on. Hobbyist are doing it all the time, usually in form of most devices either communicating directly or being bridged into queue. So technically via that every device can call every other device directly so for simple things you don't even need programming The idea is just to keep that within your own LAN. IoT world mostly wants to create closed gardens so you buy other products around that garden. Easiest way to make it easy for consumer is to connect your devices to cloud and so most decide to go that way, no need for end user to buy a hub. And even if you need to (say devices use zigbee and you need a bridge), putting that in cloud means user's phone can always access it, regardless of whether they are connected to the home's wifi, without much fuss (as home internet will most likely be behind NAT/firewall so it can be hard to access directly
- noobface 3y ago
  It's getting cheaper: https://coral.ai/products/ https://coral.ai/products/ $20 for the SMD TPU isn't bad, but it's definitely at the top end of the BOM for custom PCB projects. Ideally we'll see some competition with on-package options.
  - jsjohnst 3y ago
    > $20 for the SMD TPU isn't bad, but it's definitely at the top end of the BOM for custom PCB projects. Sounds great in theory, but show me a place where you can buy Coral TPUs at anywhere near msrp. Unless something has changed recently, finding a real live unicorn would be easier.
    - zeagle 3y ago
      If you see a usb one for sale somewhere in Canada near msrp please let me know!
  - moffkalast 3y ago
    Any idea if anyone will spin a board with a bunch of these and 12G of VRAM for loading LLMs?
- ilyt 3y ago
  Still can be solution, if your system degrades to "have to press a button on phone" when Internet/cloud is down that is still IMO acceptable. Until we get something local powerful enough "cloudsourcing" LLM while keeping everything else locally is not a terrible compromise.
deleted 3y ago
[deleted]
Joeri 3y ago
A mac mini running home assistant with some kind of llama plugin running one if the 7B models should do well enough and would run entirely locally. It is just a matter of time before someone builds it. I think one of the issues for a lot of people is the beefy hardware required for running even a 7B model locally. A RPi4 just won’t cut it for interactive use, and that’s what most people would run their home assistant setup on.
joshvm 3y ago
This is broadly the ethos behind Home Assistant. It runs on a Pi or similar box and it acts as an abstraction layer on top of a bunch of drivers for smart sensors and actuators. It's totally offline, though if you're really against being connected that also means you can't access it remotely (unless you SSH into a local machine). The dashboarding is competent, though there are often little things that are frustrating to modify (but it's open source, so the opportunity is there). The whole system is a sort of IFTTT and quite powerful. Since every entity is exposed as a generic device with some capabilities, you can easily set up little automation routines or you can interact with the local API. At this point pretty much all the "big" names are supported - I have a mix of Hue, Aqara, iRobot, IKEA and other devices and they mostly just connect. There's also support for unusual things like Octoprint and router status. And now there's also ESPHome so you can make your own sensor nodes. The biggest challenge with offline voice control is latency between speech to text, inference and text to speech. Running offline is less of a problem and open source models will eventually get good enough, but the really impressive aspect of Alexa, Siri, etc is how fast they can respond to you. https://www.home-assistant.io/ https://www.home-assistant.io/

by ilyt 3y ago

nine_k 3y ago

"Internet of Things will not happen until LAN of Things arrives", to paraphrase someone. This already happens in the industry, like heavy industry. At home, it's held back by absence of an adequate server. Often a router box or a NAS would be fine to host something like OwnCloud. But to run ML middle, different and much more expensive hardware is required, which would sit idle 99% of time.

kingo55 3y ago
LAN of things - this is a fascinating idea. Can you share where you are paraphrasing this from or link to more info on this concept? Sounds familiar with the self hosted applications people now run on their NAS/routers these days.
- c_o_n_v_e_x 3y ago
  The phrase is used quite a bit in hackernews comments, particularly on any IoT or home automation threads.
- ilyt 3y ago
  The concept is nothing new, it's just not something manufacturers picked on. Hobbyist are doing it all the time, usually in form of most devices either communicating directly or being bridged into queue. So technically via that every device can call every other device directly so for simple things you don't even need programming The idea is just to keep that within your own LAN. IoT world mostly wants to create closed gardens so you buy other products around that garden. Easiest way to make it easy for consumer is to connect your devices to cloud and so most decide to go that way, no need for end user to buy a hub. And even if you need to (say devices use zigbee and you need a bridge), putting that in cloud means user's phone can always access it, regardless of whether they are connected to the home's wifi, without much fuss (as home internet will most likely be behind NAT/firewall so it can be hard to access directly
noobface 3y ago
It's getting cheaper: https://coral.ai/products/ https://coral.ai/products/ $20 for the SMD TPU isn't bad, but it's definitely at the top end of the BOM for custom PCB projects. Ideally we'll see some competition with on-package options.
- jsjohnst 3y ago
  > $20 for the SMD TPU isn't bad, but it's definitely at the top end of the BOM for custom PCB projects. Sounds great in theory, but show me a place where you can buy Coral TPUs at anywhere near msrp. Unless something has changed recently, finding a real live unicorn would be easier.
  - zeagle 3y ago
    If you see a usb one for sale somewhere in Canada near msrp please let me know!
- moffkalast 3y ago
  Any idea if anyone will spin a board with a bunch of these and 12G of VRAM for loading LLMs?
ilyt 3y ago
Still can be solution, if your system degrades to "have to press a button on phone" when Internet/cloud is down that is still IMO acceptable. Until we get something local powerful enough "cloudsourcing" LLM while keeping everything else locally is not a terrible compromise.

deleted 3y ago

[deleted]

Joeri 3y ago

A mac mini running home assistant with some kind of llama plugin running one if the 7B models should do well enough and would run entirely locally. It is just a matter of time before someone builds it. I think one of the issues for a lot of people is the beefy hardware required for running even a 7B model locally. A RPi4 just won’t cut it for interactive use, and that’s what most people would run their home assistant setup on.

joshvm 3y ago

This is broadly the ethos behind Home Assistant. It runs on a Pi or similar box and it acts as an abstraction layer on top of a bunch of drivers for smart sensors and actuators. It's totally offline, though if you're really against being connected that also means you can't access it remotely (unless you SSH into a local machine). The dashboarding is competent, though there are often little things that are frustrating to modify (but it's open source, so the opportunity is there). The whole system is a sort of IFTTT and quite powerful. Since every entity is exposed as a generic device with some capabilities, you can easily set up little automation routines or you can interact with the local API. At this point pretty much all the "big" names are supported - I have a mix of Hue, Aqara, iRobot, IKEA and other devices and they mostly just connect. There's also support for unusual things like Octoprint and router status. And now there's also ESPHome so you can make your own sensor nodes. The biggest challenge with offline voice control is latency between speech to text, inference and text to speech. Running offline is less of a problem and open source models will eventually get good enough, but the really impressive aspect of Alexa, Siri, etc is how fast they can respond to you. https://www.home-assistant.io/ https://www.home-assistant.io/