5 ms·

Anyone have any experience with the offline OSS offerings? Eager to replace my cheap Echo devices with more expensive privacy friendly options. Another questi

by RajT88 2y ago

Anyone have any experience with the offline OSS offerings? Eager to replace my cheap Echo devices with more expensive privacy friendly options. Another question: anyone aware of community custom firmware efforts? I know early gen devices had some exploits but it never resulted to much last I checked.

sofixa 2y ago
I've used Home Assistant Voice (Preview Edition) and ESP32-S3-BOX-3 alongside Home Assistant 's Voice capabilities - by default there are phrases included, and you can add your own, but you have to be precise - "turn off the lights" and "turn off the lamp" are two separate phrases (or a variable one, which can be expanded by a locally (or not) hosted LLM. Voice detection is pretty good, especially on the Voice, even from across a room. Not 100% (especially with higher pitched voices), but definitely high and usable. The main downside is the fact that you either need to host a local LLM, which isn't cheap, or predefine all phrases you want.
- stavros 2y ago
  I'd really love something that enabled me to use the hardware without Home Assistant. I know it's OSS (basically ESPhome), but I'd like a server I could point my Voice thing to and be able to receive a webhook with the text I just spoke, and have it read the text that my server returns out loud. I'd implement the actual smarts myself, I just want the STT/TTS interface sorted.
  - zrail 2y ago
    Home Assistant uses the Wyoming protocol to talk to Whisper, Piper, and the various other components of the speech pipeline. Might be something to check out? https://github.com/rhasspy/wyoming https://github.com/rhasspy/wyoming This subproject looks interesting too https://github.com/rhasspy/wyoming-handle-external https://github.com/rhasspy/wyoming-handle-external
    - stavros 2y ago
      Yep, I had seen that but I didn't go deeper than that. I'll have another look, thanks!
- joshstrange 2y ago
  What has your experience been with Home Assistant voice? I also have one but still use my Alexa’s. I love HA and it runs my house but my initial tests were not great. I tried both OpenAI and HA cloud assistants but I could have spent more time tweaking things and I bet I could have done a lot by writing my own prompt.
  - sofixa 2y ago
    It's OK in terms of voice recognition, as in, it almost always work. But integrating with an external LLM for more flexibility in voice commands has been a meh experience and doesn't work perfectly (to be fair, I'm using an external OpenAI compatible API, so it might be something to do with that).
    - joshstrange 2y ago
      Yeah, I've tried local LLMs, Open AI, and Anthropic. They all sort-of-work and can feel magical but I feel I need to a lot more tweaking/configuring/prompting to improve the experience.
nickthegreek 2y ago
Home assistant is your gateway into this. You can even run it all locally with decent but not outrageous specs or plug into a cloud api. There is even an option to process home control locally and send other queries to the cloud.
genewitch 2y ago
It is literally impossible to transcribe voice, especially if you whisper. There's no way to model the language, it's too large - amanzon has many computers. Your computer is like a tortoise, it can't do text-to-speech. There's no way you can get any of this to use the Web, Dav... er, Cal. Trying to do this would be like trying to torch a python with your bare metal hands. so in conclusion: nope.
- zrail 2y ago
  Eh, Whisper is pretty good, especially if you have some GPU to throw at it.
  - genewitch 2y ago
    huh, i've never heard of openai-whisper. I certainly didn't mention it in my first sentence.
    - porridgeraisin 2y ago
      You said "especially if you whisper" :-) for the skimming reader that parses as whisper the model. I have to admit I did it at first too.
      - genewitch 2y ago
        here, i'll give the key to decode my message ;-) It is literally impossible to transcribe voice, especially if you whisper. There's no way to model the language, it's too large - amanzon has many computers. Your computer is like a tortoise, it can't do text-to-speech. There's no way you can get any of this to use the Web, Dav... er, Cal. Trying to do this would be like trying to torch a python with your bare metal hands. that is: baremetal pytorch https://en.wikipedia.org/wiki/PyTorch WebDAV/CalDAV https://en.wikipedia.org/wiki/CalDAV tortoise-tts https://github.com/neonbjb/tortoise-tts/issues LLM/Large Language Model https://lmstudio.ai/ whisper https://github.com/openai/whisper/ in reverse order
    - zrail 2y ago

by RajT88 2y ago

sofixa 2y ago

I've used Home Assistant Voice (Preview Edition) and ESP32-S3-BOX-3 alongside Home Assistant 's Voice capabilities - by default there are phrases included, and you can add your own, but you have to be precise - "turn off the lights" and "turn off the lamp" are two separate phrases (or a variable one, which can be expanded by a locally (or not) hosted LLM. Voice detection is pretty good, especially on the Voice, even from across a room. Not 100% (especially with higher pitched voices), but definitely high and usable. The main downside is the fact that you either need to host a local LLM, which isn't cheap, or predefine all phrases you want.

stavros 2y ago
I'd really love something that enabled me to use the hardware without Home Assistant. I know it's OSS (basically ESPhome), but I'd like a server I could point my Voice thing to and be able to receive a webhook with the text I just spoke, and have it read the text that my server returns out loud. I'd implement the actual smarts myself, I just want the STT/TTS interface sorted.
- zrail 2y ago
  Home Assistant uses the Wyoming protocol to talk to Whisper, Piper, and the various other components of the speech pipeline. Might be something to check out? https://github.com/rhasspy/wyoming https://github.com/rhasspy/wyoming This subproject looks interesting too https://github.com/rhasspy/wyoming-handle-external https://github.com/rhasspy/wyoming-handle-external
  - stavros 2y ago
    Yep, I had seen that but I didn't go deeper than that. I'll have another look, thanks!
joshstrange 2y ago
What has your experience been with Home Assistant voice? I also have one but still use my Alexa’s. I love HA and it runs my house but my initial tests were not great. I tried both OpenAI and HA cloud assistants but I could have spent more time tweaking things and I bet I could have done a lot by writing my own prompt.
- sofixa 2y ago
  It's OK in terms of voice recognition, as in, it almost always work. But integrating with an external LLM for more flexibility in voice commands has been a meh experience and doesn't work perfectly (to be fair, I'm using an external OpenAI compatible API, so it might be something to do with that).
  - joshstrange 2y ago
    Yeah, I've tried local LLMs, Open AI, and Anthropic. They all sort-of-work and can feel magical but I feel I need to a lot more tweaking/configuring/prompting to improve the experience.

nickthegreek 2y ago

Home assistant is your gateway into this. You can even run it all locally with decent but not outrageous specs or plug into a cloud api. There is even an option to process home control locally and send other queries to the cloud.

genewitch 2y ago

It is literally impossible to transcribe voice, especially if you whisper. There's no way to model the language, it's too large - amanzon has many computers. Your computer is like a tortoise, it can't do text-to-speech. There's no way you can get any of this to use the Web, Dav... er, Cal. Trying to do this would be like trying to torch a python with your bare metal hands. so in conclusion: nope.

zrail 2y ago
Eh, Whisper is pretty good, especially if you have some GPU to throw at it.
- genewitch 2y ago
  huh, i've never heard of openai-whisper. I certainly didn't mention it in my first sentence.
  - porridgeraisin 2y ago
    You said "especially if you whisper" :-) for the skimming reader that parses as whisper the model. I have to admit I did it at first too.
    - genewitch 2y ago
      here, i'll give the key to decode my message ;-) It is literally impossible to transcribe voice, especially if you whisper. There's no way to model the language, it's too large - amanzon has many computers. Your computer is like a tortoise, it can't do text-to-speech. There's no way you can get any of this to use the Web, Dav... er, Cal. Trying to do this would be like trying to torch a python with your bare metal hands. that is: baremetal pytorch https://en.wikipedia.org/wiki/PyTorch WebDAV/CalDAV https://en.wikipedia.org/wiki/CalDAV tortoise-tts https://github.com/neonbjb/tortoise-tts/issues LLM/Large Language Model https://lmstudio.ai/ whisper https://github.com/openai/whisper/ in reverse order
  - zrail 2y ago