9 ms·
Using ChatGPT for home automation
- syntaxing 3y agoIs there something that works with home assistant? Siri is just terrible and have been trying to replace it with anything. The ESP box posted last weeks looks pretty promising but wonder if there’s something that could work directly off of the rpi I host HA with.
- tikkun 3y agoFor anyone curious, the ESP box mentioned is likely: https://news.ycombinator.com/item?id=35948462 https://news.ycombinator.com/item?id=35948462
- kingo55 3y agoYes, see my example above. I use Node Red to get chat GPT and Home Assistant talking to each other. No code required (unless you count the prompt template).
- syntaxing 3y agoHmm do you have a node red template? Never used it so confused how to set this up. Is it voice activated?
- kingo55 3y agoIt activates upon certain activities or at every hour - I haven't thought about voice activation though. Here's an export and screenshot of the flow (these links will expire in 2 weeks): https://pastes.io/5dfmorqw1d https://pastes.io/5dfmorqw1d https://ibb.co/gF55h46 https://ibb.co/gF55h46 I use a few custom "nodes" to make it easy to query and parse the response though: * node-red-contrib-string * node-red-contrib-custom-chatgpt * node-red-contrib-home-assistant-websocket On the Home Assistant side, you'll need to create some "Helpers" or "input_number" entities and an "input_text" entity for the reason. Hopefully that helps you get up and running.
- syntaxing 3y agoThanks for the detailed explanation! Gonna give it a go this weekend.
- iamflimflam1 3y agoThere’s a good video showing it working here: https://youtu.be/BeJVv0pL5kY https://youtu.be/BeJVv0pL5kY
- charbull 3y agovery cool !
- evmaki 3y agoSuper exciting to see the work happening in this area! I can especially appreciate the use of ChatGPT to orchestrate the necessary API calls, rather than relying on some kind of middleware to do it. I have been working in this area (LLMs for ubiquitous computing, more generally) for my PhD dissertation and have discovered some interesting quality issues when you dig deeper [0]. If you only have lights in your house, for instance, the GPT models will always use them in response to just about any command you give, then post-rationalize the answer. If I say "it's too chilly in here" in a house with only lights, it will turn them on as a way of "warming things up". Kind of like a smart home form of hallucination. I think these sorts of quality issues will be the big hurdle to product integration. [0] https://arxiv.org/abs/2305.09802 https://arxiv.org/abs/2305.09802
- iamflimflam1 3y agoVery interesting paper! It's something that I've been wondering about with ChatGPT plugins - they've kind of left it up to the user to enable/disable plugins. But there's definitely going to come a point where plugins conflict and the LLM is going to have to choose the most appropriate plugin to use. I have been very impressed at how good it is at turning random commands into concrete API calls. You are right though, pretty much any command can be interpreted as an instruction to use a plugin.
- evmaki 3y agoThanks! That is part of the challenge as this idea scales imo - once you've increased the number of plugins or "levers" available to the model, you start to increase the likelihood that it will pull some of them indiscriminately. To your point about turning random commands into API calls: if you give it the raw JSON from a Philips Hue bridge and ask it to manipulate it in response to commands, it can even do oddly specific things like triggering Hue-specific lighting effects [0] without any description in the plugin yaml. I'm assuming some part of the corpus contains info about the Hue API. [0] https://evanking.io/posts/homegpt/ https://evanking.io/posts/homegpt/
- ftxbro 3y ago> If I say "it's too chilly in here" in a house with only lights, it will turn them on as a way of "warming things up". is it bad that I'm on GPT's side on this, i mean if they are incandescent lights they are small electric heaters and what else do you expect it to do
- graiz 3y agoNice - exploring the use of home assistant for something similar. It already has all the integrations and API's for control, just lacks the Chat GPT plugin.
- iamflimflam1 3y agoThey are very easy to make - it's just an API. The main difficulty at the moment is just getting access to them. There's a long waiting list for develop access.
- chpatrick 3y agoAs an AI language model, I can't let you do that, Dave.
- deleted 3y ago[deleted]
- jacooper 3y agoSelf hosted ai ftw.
- ilyt 3y agoI'm not against idea of having LLM be smart assistant for home, but I do have problem with sending any of that to cloud. It's one thing to use it as convenient way to change anything remotely but linking home automation not only into constant internet connectivity but also "that one particular cloud thing that might disappear at any moment" (hello google) just seems like setting yourself for problems later. At least in this case he's still left with HASS if cloudy part goes away. But I wish there was more push into hybrid model - like have your router or small ARM box be server that runs queue and some, for lack of better word, "lambda-like" code that handles most of the programmed events (lights, heating etc.), say via WASM and some APIs around it, then just have HASS-like software (both cloud and local) deploy control rules/code onto it. Big nice UI for control goes down ? Doesn't matter, the controller is just running simple code, not rest of the visualisation and controls. Want to replace it ? Configuration as code makes that a breeze, hell, have cloud backup of whole setup. Don't want controller ? It's essentially just a queue and some code runners, that cloud providers can host for you. Want pretty graphs in cloud or some aggregation ? Just make your local node filter and send relevant MQ events there.
- nine_k 3y ago"Internet of Things will not happen until LAN of Things arrives", to paraphrase someone. This already happens in the industry, like heavy industry. At home, it's held back by absence of an adequate server. Often a router box or a NAS would be fine to host something like OwnCloud. But to run ML middle, different and much more expensive hardware is required, which would sit idle 99% of time.
- kingo55 3y agoLAN of things - this is a fascinating idea. Can you share where you are paraphrasing this from or link to more info on this concept? Sounds familiar with the self hosted applications people now run on their NAS/routers these days.
- c_o_n_v_e_x 3y agoThe phrase is used quite a bit in hackernews comments, particularly on any IoT or home automation threads.
- kingo55 3y agoI have been using chat GPT to control my light colours for about a month now. It's too tedious to properly set the colours and temperatures of our lights manually and too complex to consider all factors like activity, weather, music, time of day and season. Chat GPT is now our personal lighting DJ, giving us dynamic and interesting light combinations that respect our circadian rhythm. Here's my prompt - the output of which feeds Home Assistant: Set the hue for my home's lights using the HSL/HSB scale from 0-360 by providing a primary and complementary colour which considers the current situation. The HSL color spectrum ranges from 0 (red), 120 (green), to 240 (blue) and back to 360 (red). Lower values (0-60) represent warmer colors, while higher values (180-240) represent cooler colors. Middle values (60-180) are neutral. Consider these factors in setting the primary hue (in order of importance): 1. Preferences throughout the day: - When about to wake: Reds, oranges or hot pinks - Approaching bedtime: Hot pinks or reds - During worktime: Blues, greens or yellows - Other times: Greens, yellows or oranges 2. Current activity: Bedtime 3. Sleep schedule: Bedtime 23:00, Wake-up time 07:00 4. Date & time: Sunday May 21, 05:40 5. Current primary hue: 10 6. Current complementary hue: 190 7. Weather: 13°C, wind speed 9 km/h, autumn Respond in this format and provide a reason in <250 characters: {"primary_hue": PRIMARY_HUE, "complementary_hue: COMPLEMENTARY_HUE, "reason": REASON } The output looks like this: {primary_hue: 10 complementary_hue: 190 reason: "Approaching bedtime and early hours of morning, so a warm and calming hue is needed. Complementary hue adjusted slightly to 195 to maintain balance."}
- iamflimflam1 3y agoThat's really cool. Are you using the API to run this locally?
- kingo55 3y agoYes, I access the API through Node Red which fills out the prompt template and returns Chat GPT's output to Home Assistant. Costs about $3/year in API quota.
- avidphantasm 3y ago[flagged]
- odiroot 3y agoYou can also use ChatGPT to help you write Home Assistant YAML code for sensors and automations. I even got it to write ESPHome rules for me.
- jonplackett 3y agoI had been thinking about doing this. I’m getting really tired of talking to Alexa now I’m used to a machine actually understanding me. Lights is just the start. Things like getting the weather, playing a playlist (or coming up with a new one for me) would all be SO much better with an LLM instead of a dumb bot. Edit: if you’re going to downvote. Leave a comment at least. Are there just a lot of Alexa fans in the house?
- awwaiid 3y agoMaybe you need to let your LLM manage Alexa
- majormajor 3y agoComing up with a playlist is a fun one, but I've had much more luck with stuff like "generate a description of a playlist" - I have a little side project that talks to the spotify API and dumps song metadata into sqlite, and plugging GPT in as a SQL query generator was super useful vs writing all those queries by hand. Of course that requires a lot of up-front work other than GPT, but not a ton more than is necessary to talk to the API in the first place, but GPT itself has not impressed me in the ability to rank things, like "select the fifty other songs I've liked that are most like this song" even when given a lot of quantitative metadata.
- gerdesj 3y agoIt's in its infancy but this does work - https://www.home-assistant.io/blog/2023/05/03/release-20235 https://www.home-assistant.io/blog/2023/05/03/release-20235 I have been able to turn a device on and off by speaking to the app on my phone. Nothing leaves my house. There is a bit of a lag but it is early days. You simply add the Piper and Whisper addons, then add the integration in the GUI. Then you press the assistant button in the app or your browser and then press the mic button and talk.
- Corrado 3y agoAgreed. In order to get Alexa to do some "higher order" stuff you have to be very explicit and program each step into a routine. This routine never changes or grows and barely adapts to the environment. For example, I have a routine called "breakfast" that turns on the TV and the light in the living room. Alexa doesn't really know anything about breakfast it just blindly turns on the light and TV. I think it would be handy to have Alexa know that I've been watching a particular show during "breakfast" and turn the TV to the next episode. It would also be super handy for Alexa to notice that it's my usual breakfast time, I've just gotten milk and cereal out of the cabinet, and I'm heading to the TV room. Instead, I have to utter the words "Alexa, breakfast" when I want it to get ready for me to start my day.
- quickthrower2 3y agoBut don’t forget AI safety you fine pioneers
- b112 3y agoOh I can just see it. A fire. A desperate resident. A plea for help. "Fucking help me!!!" screamed out. Rather than activate the sprinklers, ChatGPT admonishing the on fire, burning person, for language, and a lack of respect. This is the AI is see currently. A hazard. A menace. A joke.
- sukilot 3y ago[dead]
- micromacrofoot 3y agoversus what happens now, you scream “fucking help me” and just die anyway
- b112 3y agoNo, currently sprinkler control is handled by dumb, automated equipment, along with sensors for the alarm company, along with a human calling. Additionally, it was a comment on the current state of AI, which is far less reliable than people seem to believe.
- micromacrofoot 3y agoWe're a far ways off from replacing sprinkler logic with AI, you're kind of mashing together building code with home automation. Most homes don't have sprinkler systems unless they're in large buildings that mandate it, and in those cases they're never controlled by individual occupants but a centralized detection system. Building code lags behind consumer technology quite far, with good reason.
- b112 3y agoUnless you are a startup, and unless sprinklers aren't required in the building code, and are optional.. such as in my area. But you see, it was a joke, which illustrates my frustration with all this chatgpt blather. And as with many jokes, it has a point. That is, that this tech is wildly unreliable for any use requiring decisions, yet people are working with it, as if it is. And "bad things" will be the result.
- XCSme 3y agoIs this reading specific outputs (in text form) from ChatGPT and then forwarding them to an API? Or how does "ChatGPT" actually make the call based on the OpenAPI description?
- gumballindie 3y agoHere is what a large budget buys: massive marketing campaigns. ChatGPT spam is not showing signs of slowing down anytime soon. Are people aware there are other ais out there they can use? Some free?
- sukilot 3y ago[dead]
- danols 3y agoFeel free to educate us and give a few examples with comments. Bonus points if they are available in EU...
- gumballindie 3y agoWhat kind of ai stuff would you like to do? There are indeed few with as many integrations but individually there are open source variants out there and specialised products.
- geysersam 3y agoWhat example isn't available in the EU?
- danols 3y agoBard
- aussieguy1234 3y agoIf it can turn on a light, it can open the pod bay doors.
- wkat4242 3y agoThis is so cool. Only drawback: it won't feel complete unless you build pod bay doors on your house.
- paulddraper 3y ago1990s: What if we had to fight Skynet? 2020s: Let's make Skynet.
- SergeAx 3y agoFrom the beginning I was ridiculing AGI dommsdayers, but just one month later I am waitng for an article about "Isn't it a great idea to connect LLM to NORAD system?"
- bobbyi 3y agoThe June issue of the Atlantic has an article about giving control of nuclear weapons to AI https://www.theatlantic.com/magazine/archive/2023/06/ai-warfare-nuclear-weapons-strike/673780/ https://www.theatlantic.com/magazine/archive/2023/06/ai-warf...
- aylmao 3y agoThis sounds very exciting, but the more I think about this the more I think the right interface UI for light control isn't language. I recently bought smart switches for my smart lights and its fantastic to be back to clicking. Having to exclaim "I'm going to bed" every time you're going to bed, or "turn on bathroom lights" every time you're going to the bathroom is cartoonish at best, and gets annoying. Switching "turn on bathroom lights" for "I'm going to use the bathroom" doesn't make it much better. The ideal interface IMO is presence sensors or in their absence, well-placed light switches, given how quickly and subconsciously one can flick them. I think there's room for augmentation and improvement using LLMs and language. A system that automatically adjusts the color of the lights based on the time of day, or one you can occasionally ask exact or complex queries (ie, "make the living room red", or "turn the lights off in 50 minutes please"). But for the day-to-day, having to "think" about your lights gets annoying.
- floitsch 3y agoI have connected my esp32s to chat (telegram) and openai. With the correct prompt and an interpreter they can now execute requests in human language: https://medium.com/p/3242af6f2988 https://medium.com/p/3242af6f2988
- KETpXDDzR 3y agoI wonder when we'll have ChatGPT for sth like Amazon Alexa. I bet Amazon is already working on that and open source versions are totally doable.
- winddude 3y agoI'm sorry as a large language model I cannot turn on "kinky sex bang list 47".