6 ms·

There is a huge market segment waiting here. At least I think there is. Well, at least people like me want this. Ok, tens of dollars can be made at least. It is

by jmward01 10mo ago

There is a huge market segment waiting here. At least I think there is. Well, at least people like me want this. Ok, tens of dollars can be made at least. It is just missing a critical tipping point. Basically, I want an alexa like device for the home backed by local inference and storage with some standardized components identified: - the interactive devices - all the alexa/google/apple devices out there are this interface, also, probably some TV input that stays local and I can voice control. That kind of thing. It should have a good speaker and voice control. It probably should also do other things like act as a wifi range extender or be the router. That would actually be good. I would buy one for each room so no need for crazy antennas if they are close and can create true mesh network for me. But I digress. - the home 'cloud' server that is storage and control. This is a cheap CPU, a little ram and potentially a lot of storage. It should hold the 'apps' for my home and be the one place I can back-up everything about my network (including the network config!) - the inference engines. That is where this kind of repo/device combo comes in. I buy it and it knows how to advertise in a standard way its services and the controlling node connects it to the home devices. It would be great to just plug it in and go. Of course all of these could be combined but conceptually I want to be able to swap and mix and match at these levels so options here and interoperability is what really matters. I know a lot of (all of) these pieces exist, but they don't work well together. There isn't a simple standard 'buy this turn it on and pair with your local network' kind of plug and play environment. My core requirements are really privacy and that it starts taking over the unitaskers/plays well together with other things. There is a reason I am buying all this local stuff. If you phone home/require me to set up an account with you I probably don't want to buy your product. I want to be able to say 'Freddy, set timer for 10 mins' or 'Freddy, what is the number one tourist attraction in South Dakota' (wall drugs if you were wondering)

protocolture 10mo ago
Keen for this also. Been having issues getting a smooth voice experience from HA to ChatGPT. I dont like the whole wakeword concept for the receiver either. I think theres work to be done on the whole stack.
- 6510 10mo ago
  It should participate in all conversations, take initiative and experiment.
  - sdenton4 10mo ago
    "Hey, hey, are you still asleep? Using spare cycles, I have designed an optimal recipe for mashed potatoes, as you mentioned ten days ago. I need you to go get some potatoes."
    - FeepingCreature 10mo ago
      I unironically want this.
      - 6510 10mo ago
        I forget who but someone onhere a while back said he made a contraption that listens in and tries to determine the winner of each conversation.
      - estimator7292 10mo ago
        Agreed, this is hilarious.
    - terribleperson 10mo ago
      A local AI system that hears your conversations, identifies problems, and then uses spare cycles to devise solutions for them is actually an incredible idea. I'm never going to give a cloud system the kind of access it would need to do a really good job, but a local one I control? Absolutely. "Hey, are you still having trouble with[succinct summary of a problem it identified]?" "Yes" "I have a solution that meets your requirements as I understand them, and fits in your budget."
      - darkwater 10mo ago
        "Did you find how to make peace with $FRIEND_OF_SPOUSE after they came here last week and they were pretty mad at you because you should tell something to $SPOUSE ? I thought about it in my spare cycles and all psychologists agree that truth and trust are paramount in a healthy relationship"
Normal_gaussian 10mo ago
No, there isn't a plug and play one yet, but I've have great success with Home Assistant and the Home Assistant Voice Preview edition and its goal is pretty much to get rid of Alexa. I'd imagine you'd have a bunch of cheap ones in the house that are all WiFi + Mic + Speakers, streaming back to your actual voice processing box (which would cost a wee bit more, but also have local access to all the data it needs). You can see quite quickly that this becomes just another program running on a host, so if you use a slightly beefier machine and chuck a WiFi card in as well you've got your WiFi extenders.
- mcny 10mo ago
  And if it is plugged in to the wall, I'd be tempted to add a touch screen display and a camera just in case. But really my use case is as simple as 1. Wake word, what time is it in ____ 2. Wake word, how is the weather in ____ 3. Wake word, will it rain/snow/?? in _____ today / tomorrow / ?? 4. Wake word, what is ______ 5. Wake word, when is the next new moon / full moon? 6. Wake word, when is sunrise / sunset? And something similar like that
  - sallveburrpi 10mo ago
    So you need a clock maybe? Plus something like wttr.in
    - mcny 10mo ago
      Problem is it should be accessible by voice for like a ninety year old person.
- joshstrange 10mo ago
  > but I've have great success with Home Assistant and the Home Assistant Voice Preview edition As compared to Alexa? I bought their preview hardware (and had a home-rolled ESP32 version before that even) and things are getting closer, I can see the future where this works but we aren't there today IMHO. HA Voice (the current hardware) does not do well enough in the mic or speaker [0] department when compared to the Echos. My Echo can hear me over just about anything and I can hear it back, the HA Voice hardware is too quiet and the mic does not pick my up from the same distances or noise pollution levels as the Echo. I _love_ my HA setup and run everything through it. I'd like nothing more than to trash all my Echos, I cam close to ordering multiple of the preview devices but convinced myself to get just 1 to test (glad I did). Bottom line: I think HA Voice is the future (for me) but it's not ready yet, it doesn't compare to the Echos. I wish so much that my Sonos speakers could integrate with HA Voice since I already have those everywhere and I know they sound good. [0] I use Sonos for all my music/audio listening in my house so I only care about the speaker for hearing it talk back to me, I don't need high-end audiophile speakers.
throwaway7783 10mo ago
And toys
fuzzer371 10mo ago
And there never will be. You know why? Because the giant corporations can't suck up all your data and tailor advertisements to you. Why sell a good thing once, when you can sell crappy shovelware ridden with ads and a subscription service every month?
- jmward01 10mo ago
  Open source is amazing for this. Honestly, I suspect this is much simpler than the jellyfin ecosystem and other open source projects out there. Really, we are so close to this now it is just missing a few things like a good 'how to' that ties it all together and turns into the opensource repo that bundles things.
colechristensen 10mo ago
I've been working on this on and off for a couple of years now, the loop is definitely closing, I think it's possible at this point but not yet easy.
matusp 10mo ago
The sota chatbots are getting more and more functionality that is not just LLM inference. They can search the web, process files, integrate with other apps. I think that's why most people will consider local LLMs to be insufficient very soon.
- woooooo 10mo ago
  But that's just software that also runs fine locally. A few tools with a local LLM can do it.
  - matusp 10mo ago
    Well I don't see people running their web search locally, so I don't think they will run their own search+LLM.
    - woooooo 10mo ago
      No but you can call out to Google or DDG APIs.
- BoxOfRain 10mo ago
  Nah I disagree, tool calling isn't that difficult. I've got my own Cats Effect based model orchestration project I'm working on, and while it's not 100% yet I can do web browse, web search, memory search (this one is cool), and others on my own hardware.
ragebol 10mo ago
A bit like HomeAssistant Voice? https://www.home-assistant.io/voice-pe/ https://www.home-assistant.io/voice-pe/
zwnow 10mo ago
> Well, at least people like me want this. Yeah because dynamic digital price signs in shops based on what data vendors have about you and AI can extract from it are such fun! Total surveillance. More than what's already happening. Such fun!
mkul 10mo ago
I've just started using it but I'd recommend https://github.com/steipete/clawdis https://github.com/steipete/clawdis, you need to set it up a bit but it's really cool to just be able to do things on the go by just texting an assistant. You can see all the different ways people are using it @clawdbot on twitter.
- mr_mitm 10mo ago
  Can you give us some highlights on how this is helpful in your day-to-day life for those of us who aren't on twitter?
- BizarroLand 10mo ago
  Why does it require an online AI service? Why can it not work with ollama or some other locally hosted setup?
sofixa 10mo ago
It sounds like you want Home Asisstant. You have all of the different components: * you can use a number of things for the interactive devices (any touchscreen device, buttons, voice, etc) * have it HA do the basic parsing (word for word matching), with optionally plugging into something more complex (cloud service like ChatGPT, or self-hosted Ollama or whatever) for more advanced parsing (logical parsing) Every part of the ecosystem is interchangeable and very open. You can use a bunch of different devices, a bunch of different LLMs to do the advanced parsing if you want it. HA can control pretty much everything with an API, and can itself be controlled by pretty much anything that can talk an API.
PunchyHamster 10mo ago
There is but that market doesn't sell subscriptions and that is what tech giants wants to sell - renewable flow of money that will keep flowing even if product stagnates because effort to move to competition is big.
- Haaargio 10mo ago
  We are in a free market with china still playing the open source game. The market is not ready for building this due to costs etc. not because the big companies block them or anything. And nvidia is not selling subscriptions at all.
estimator7292 10mo ago
Just last week I hacked my Echo Show to install a custom OS and hook it into HomeAssistant. Even gave it a custom wake word, she's Janet now. HA is pretty clunky and there's a lot of manual setup. But I have a voice assistant contained entirely within my local infrastructure. I'm even planning to wire it up to my local ollama server for actual AI inference behind it. So far it's exactly as crappy as Alexa, but only because I haven't waded deep enough into configuration. I'm okay with tools being crap when it's my fault instead of the tool being crap because it doesn't make Amazon enough money.
- password4321 10mo ago
  > hacked my Echo Show Wowsers I did not know this was a thing; TIL, thanks!

protocolture 10mo ago

Keen for this also. Been having issues getting a smooth voice experience from HA to ChatGPT. I dont like the whole wakeword concept for the receiver either. I think theres work to be done on the whole stack.

6510 10mo ago
It should participate in all conversations, take initiative and experiment.
- sdenton4 10mo ago
  "Hey, hey, are you still asleep? Using spare cycles, I have designed an optimal recipe for mashed potatoes, as you mentioned ten days ago. I need you to go get some potatoes."
  - FeepingCreature 10mo ago
    I unironically want this.
    - 6510 10mo ago
      I forget who but someone onhere a while back said he made a contraption that listens in and tries to determine the winner of each conversation.
    - estimator7292 10mo ago
      Agreed, this is hilarious.
  - terribleperson 10mo ago
    A local AI system that hears your conversations, identifies problems, and then uses spare cycles to devise solutions for them is actually an incredible idea. I'm never going to give a cloud system the kind of access it would need to do a really good job, but a local one I control? Absolutely. "Hey, are you still having trouble with[succinct summary of a problem it identified]?" "Yes" "I have a solution that meets your requirements as I understand them, and fits in your budget."
    - darkwater 10mo ago
      "Did you find how to make peace with $FRIEND_OF_SPOUSE after they came here last week and they were pretty mad at you because you should tell something to $SPOUSE ? I thought about it in my spare cycles and all psychologists agree that truth and trust are paramount in a healthy relationship"

Normal_gaussian 10mo ago

No, there isn't a plug and play one yet, but I've have great success with Home Assistant and the Home Assistant Voice Preview edition and its goal is pretty much to get rid of Alexa. I'd imagine you'd have a bunch of cheap ones in the house that are all WiFi + Mic + Speakers, streaming back to your actual voice processing box (which would cost a wee bit more, but also have local access to all the data it needs). You can see quite quickly that this becomes just another program running on a host, so if you use a slightly beefier machine and chuck a WiFi card in as well you've got your WiFi extenders.

mcny 10mo ago
And if it is plugged in to the wall, I'd be tempted to add a touch screen display and a camera just in case. But really my use case is as simple as 1. Wake word, what time is it in ____ 2. Wake word, how is the weather in ____ 3. Wake word, will it rain/snow/?? in _____ today / tomorrow / ?? 4. Wake word, what is ______ 5. Wake word, when is the next new moon / full moon? 6. Wake word, when is sunrise / sunset? And something similar like that
- sallveburrpi 10mo ago
  So you need a clock maybe? Plus something like wttr.in
  - mcny 10mo ago
    Problem is it should be accessible by voice for like a ninety year old person.
joshstrange 10mo ago
> but I've have great success with Home Assistant and the Home Assistant Voice Preview edition As compared to Alexa? I bought their preview hardware (and had a home-rolled ESP32 version before that even) and things are getting closer, I can see the future where this works but we aren't there today IMHO. HA Voice (the current hardware) does not do well enough in the mic or speaker [0] department when compared to the Echos. My Echo can hear me over just about anything and I can hear it back, the HA Voice hardware is too quiet and the mic does not pick my up from the same distances or noise pollution levels as the Echo. I _love_ my HA setup and run everything through it. I'd like nothing more than to trash all my Echos, I cam close to ordering multiple of the preview devices but convinced myself to get just 1 to test (glad I did). Bottom line: I think HA Voice is the future (for me) but it's not ready yet, it doesn't compare to the Echos. I wish so much that my Sonos speakers could integrate with HA Voice since I already have those everywhere and I know they sound good. [0] I use Sonos for all my music/audio listening in my house so I only care about the speaker for hearing it talk back to me, I don't need high-end audiophile speakers.

throwaway7783 10mo ago

And toys

fuzzer371 10mo ago

And there never will be. You know why? Because the giant corporations can't suck up all your data and tailor advertisements to you. Why sell a good thing once, when you can sell crappy shovelware ridden with ads and a subscription service every month?

jmward01 10mo ago
Open source is amazing for this. Honestly, I suspect this is much simpler than the jellyfin ecosystem and other open source projects out there. Really, we are so close to this now it is just missing a few things like a good 'how to' that ties it all together and turns into the opensource repo that bundles things.

colechristensen 10mo ago

I've been working on this on and off for a couple of years now, the loop is definitely closing, I think it's possible at this point but not yet easy.

matusp 10mo ago

The sota chatbots are getting more and more functionality that is not just LLM inference. They can search the web, process files, integrate with other apps. I think that's why most people will consider local LLMs to be insufficient very soon.

woooooo 10mo ago
But that's just software that also runs fine locally. A few tools with a local LLM can do it.
- matusp 10mo ago
  Well I don't see people running their web search locally, so I don't think they will run their own search+LLM.
  - woooooo 10mo ago
    No but you can call out to Google or DDG APIs.
BoxOfRain 10mo ago
Nah I disagree, tool calling isn't that difficult. I've got my own Cats Effect based model orchestration project I'm working on, and while it's not 100% yet I can do web browse, web search, memory search (this one is cool), and others on my own hardware.

ragebol 10mo ago

A bit like HomeAssistant Voice? https://www.home-assistant.io/voice-pe/ https://www.home-assistant.io/voice-pe/

zwnow 10mo ago

> Well, at least people like me want this. Yeah because dynamic digital price signs in shops based on what data vendors have about you and AI can extract from it are such fun! Total surveillance. More than what's already happening. Such fun!

mkul 10mo ago

I've just started using it but I'd recommend https://github.com/steipete/clawdis https://github.com/steipete/clawdis, you need to set it up a bit but it's really cool to just be able to do things on the go by just texting an assistant. You can see all the different ways people are using it @clawdbot on twitter.

mr_mitm 10mo ago
Can you give us some highlights on how this is helpful in your day-to-day life for those of us who aren't on twitter?
BizarroLand 10mo ago
Why does it require an online AI service? Why can it not work with ollama or some other locally hosted setup?

sofixa 10mo ago

It sounds like you want Home Asisstant. You have all of the different components: * you can use a number of things for the interactive devices (any touchscreen device, buttons, voice, etc) * have it HA do the basic parsing (word for word matching), with optionally plugging into something more complex (cloud service like ChatGPT, or self-hosted Ollama or whatever) for more advanced parsing (logical parsing) Every part of the ecosystem is interchangeable and very open. You can use a bunch of different devices, a bunch of different LLMs to do the advanced parsing if you want it. HA can control pretty much everything with an API, and can itself be controlled by pretty much anything that can talk an API.

PunchyHamster 10mo ago

There is but that market doesn't sell subscriptions and that is what tech giants wants to sell - renewable flow of money that will keep flowing even if product stagnates because effort to move to competition is big.

Haaargio 10mo ago
We are in a free market with china still playing the open source game. The market is not ready for building this due to costs etc. not because the big companies block them or anything. And nvidia is not selling subscriptions at all.

estimator7292 10mo ago

Just last week I hacked my Echo Show to install a custom OS and hook it into HomeAssistant. Even gave it a custom wake word, she's Janet now. HA is pretty clunky and there's a lot of manual setup. But I have a voice assistant contained entirely within my local infrastructure. I'm even planning to wire it up to my local ollama server for actual AI inference behind it. So far it's exactly as crappy as Alexa, but only because I haven't waded deep enough into configuration. I'm okay with tools being crap when it's my fault instead of the tool being crap because it doesn't make Amazon enough money.

password4321 10mo ago
> hacked my Echo Show Wowsers I did not know this was a thing; TIL, thanks!