6 ms·
I've been testing various LLMs (that can run locally - sans cloud) and (for example) the llava-v1.5-7b-q4 does a decent job for home automation. Example: I giv
by canada_dry 3y ago
I've been testing various LLMs (that can run locally - sans cloud) and (for example) the llava-v1.5-7b-q4 does a decent job for home automation.
Example: I give the LLM a range of 'verbal' instructions related to home automation to see how well they can identify the action, timing, and subject:
User: in the sentence "in 15 minutes turn off the living room light" output the subject, action, time, and location as json
Llama: {
"subject": "light",
"action": "turned off",
"time": "15 minutes from now",
"location": "living room"
}
Several of the latest models are on par to the results from Gpt4 in my tests.
- jasonjmcghee 3y agoOut of curiosity what are you using the vision aspect for? Fwiw bakllava is a much more recent model, using mistral instead of llama. Same size and capabilities
- canada_dry 3y ago> vision aspect It checks a webcam feed to tell me the current weather outside (e.g. sunny, snowing) though the language parsing is a more important feature. > more recent model Yes... models are coming out quicker every week - it's hard to keep up! But I put this one in place a few months ago and its been working fine for my purposes (basic voice controller home automation).
- ilaksh 3y agoDoes anyone know if there is something like bakllava but with commercial use permitted?
- polishdude20 3y agoWhat about like, if I said "switch off the lamp at 3:45" How would you translate the Json you'd get out of that to get the same output? The subject would be "lamp" . Your app code would need to know that lamp is also light.
- jorvi 3y agoLLM just are waayyy too dangerous for something like home automation, until it becomes a lot more certain you can guarantee an output for an input. A very dumb innocuous example would be you ordering a single pizza for the two of you, then telling the assistant “actually we’ll treat ourselves, make that two”. Assistant corrects the order to two. Then the next time you order a pizza “because I had a bad day at work”, assistant just assumes you ‘deserve’ two even if your verbal command is to order one. A much scarier example is asking the assistant to “preheat the oven when I move downstairs” a few times. Then finally one day you go on vacation and tell the assistant “I’m moving downstairs” to let it know it can turn everything off upstairs. You pick up your luggage in the hallway none the wiser, leave and.. yeah. Bye oven or bye home. Edit: enjoy your unlocked doors, burned down homes, emptied powerwalls, rained in rooms! :)
- coder543 3y agoNo. LLMs do not have memory like that (yet). Your 'scary' examples are very hypothetical and would require intentional design to achieve today; they would not happen by accident.
- jorvi 3y agoI love how burning your house down is something that deserves air quotes according to you. All I can tell you is this: LLM’s frequently misinterpret, hallucinate and “lie”. Good luck.
- amluto 3y agoPreventing burning your house down belongs on the output handling side, not the instruction processing side. If there is any output from an LLM at all that will burn your house down, you already messed up.
- flemhans 3y agoI'd go as far as saying it should be handled on the "physics" level. Any electric apparatus in your home should be able to be left on for weeks without causing fatal consequences.
- dr_dshiv 3y ago> Several of the latest models are on par to the results from Gpt4 in my tests. Wow! So almost as good as alexa?
- AdrienBrault 3y agoProbably much better than alexa. Gpt 3.5 is miles ahead alexa
- dr_dshiv 3y agoSorry that was a bad joke
- shortrounddev2 3y agoBut why use an llm for that? This kind of intent recognition has existed for a while now and we already have it in the form of smart speakers. It seems like an overkill tool for the job