3 ms·

Thanks! How "bad" are these LLMs though, especially for smart home-esque basic tasks? I'd imagine them to be alright-ish?

• rcarmo 3y ago
  Well, the ones I tried that would fit into 4GB RAM have trouble following directions and inconsistent output. I can't really tell you (yet) which would fit into, say, 16GB RAM and work consistently (and fast enough to, say, turn a light off faster than it would take you to get up and reach for the switch), but I'll eventually get to it...
  • user_7832 3y ago
    Thank you, that makes sense. I think I'll stick with my google hub for now haha
  • FergusArgyll 3y ago
    Dolphin-phi (uncensored tweaked version of msft phi) has been pretty good, I've been testing for only a couple days. It's 2.7B params, so depending on how much RAM the os is using you might be able to run that on 4GB. I run it on 8GB windows using wsl2/ollama and the OS takes up around 5GB (I think), so maybe....