7 ms·

I’m getting more and more tempted to start a startup to sell a small box for a purely local LLM for HomeKit and google assistant control (and home assistant of

I’m getting more and more tempted to start a startup to sell a small box for a purely local LLM for HomeKit and google assistant control (and home assistant of course). I wonder if the market is strong enough for this. Sure, it won’t be GPT 4 level but it’ll definitely be better than Siri. Maybe YC 2026.

dariusj18 2y ago
I've been wanting a local LLM appliance.
- brookst 2y ago
  Tech is evolving too quickly; every year the hardware will be much more powerful at the same price (as LLM optimizations reach hardware), so you’d end up replacing the device frequently.
  - deleted 2y ago
    [deleted]
  - readthenotes1 2y ago
    Like phones?
  - nextts 2y ago
    Not convinced. Are CPUs and GPUs killing it %/$ wise each year like it's 1996? Models are killing it but that is just an "ollama run" command away.
    - brookst 2y ago
      GPUs and NPUs are gaining optimizations for the transformer architecture. It’s not “GPU is 3x faster this year”, it’s “GPU has gates specifically designed to accelerate your LLM workload” See for instance [0], which is just starting to appear in commercial parts. This is continuing; pretty much every low cost SoC maker is racing to build and extend ML optimizations. 0. https://www.synopsys.com/blogs/chip-design/best-edge-ai-processor-vision-systems.html https://www.synopsys.com/blogs/chip-design/best-edge-ai-proc...
paulcole 2y ago
> I wonder if the market is strong enough for this It is not. You could not pay the average person to run a local LLM instead of just relying on Siri/Alexa/Google Assistant/whatever is built into the apps and hardware they use every day. You could sell these to HN commenters who will play with it for a week and then put it in the cupboard to gather dust next to their Raspberry Pis.
- genewitch 2y ago
  i have an rpi zero w (i think) that came as part of a voice kit - a cardboard box, an rgb led arcade button, a speaker, and a really COOL mems multi-microphone array pi zero hat. Out of the box it had raspbian or whatever on it, and a config file you edited while following instructions that came in the kit - you have to set up a firebase account, authorize the "app" on the pi, and so on. It basically gave you access to the google ecosystem in a tiny cardboard box, like a cube a baseball might fit in. It didn't have a wakeword, you'd hit the arcade button (which had a dim color when inactive), it'd pulse white or blue, and then the assistant voice would ask whatever google assistants ask, and the light would change red, and you could ask your question, the light would pulse, and a second or two later, the assistant would talk, replying to your request or whtaever. now, i am working from memory, so it may have been amazon services, but i am unsure if they allow third party access, and it may have immediately waited for your question when you pushed the button. I don't remember. If i can find one of my boxes, in a closet gathering dust next to the rest of my rpi, i'll at least try to get the model/name of the microphone array hat, because i think that's the part that will make a DIY voice assistant work rather than be a curiosity. mine stopped working after a few weeks, and i couldn't ever figure out why. I think maybe google or whatever wanted money for API usage, or they changed the rules of how firebase worked... but the hardware still works, it just doesn't "wake up" when you push the button. the logs show the button push, etc. anyhow, i got the last two at target like 5(?) years ago, for $10 each. i think it was called "AIY Voice Kit"
  - paulcole 2y ago
    I’m not sure if you’re telling me that I’m right or that I’m wrong.
bhhaskin 2y ago
Like Home Assistant voice? https://www.home-assistant.io/voice-pe/ https://www.home-assistant.io/voice-pe/
- syntaxing 2y ago
  I have one, its only wake word detection, microphone, and HA integration. The STT, LLM, and TTS is completely independent of the Voice box.
  - joshstrange 2y ago
    Independent and optionally local, which is the important part. I can use my HA Voice with a Local LLM, OpenAI, or Home Assistant cloud. Similar I can swap out TTS and STT with local versions if I want.
raphman 2y ago
The guys at Jolla (developer of Sailfish OS smartphone OS) are currently trying something like this¹: https://www.jollamind2.com/ https://www.jollamind2.com/ ¹) however, their focus is less on home automation, more on personal information management, AFAIK.
nextts 2y ago
Sell me the flash to push into a rpi and an STL to print please.

dariusj18 2y ago

I've been wanting a local LLM appliance.

brookst 2y ago
Tech is evolving too quickly; every year the hardware will be much more powerful at the same price (as LLM optimizations reach hardware), so you’d end up replacing the device frequently.
- deleted 2y ago
  [deleted]
- readthenotes1 2y ago
  Like phones?
- nextts 2y ago
  Not convinced. Are CPUs and GPUs killing it %/$ wise each year like it's 1996? Models are killing it but that is just an "ollama run" command away.
  - brookst 2y ago
    GPUs and NPUs are gaining optimizations for the transformer architecture. It’s not “GPU is 3x faster this year”, it’s “GPU has gates specifically designed to accelerate your LLM workload” See for instance [0], which is just starting to appear in commercial parts. This is continuing; pretty much every low cost SoC maker is racing to build and extend ML optimizations. 0. https://www.synopsys.com/blogs/chip-design/best-edge-ai-processor-vision-systems.html https://www.synopsys.com/blogs/chip-design/best-edge-ai-proc...

paulcole 2y ago

> I wonder if the market is strong enough for this It is not. You could not pay the average person to run a local LLM instead of just relying on Siri/Alexa/Google Assistant/whatever is built into the apps and hardware they use every day. You could sell these to HN commenters who will play with it for a week and then put it in the cupboard to gather dust next to their Raspberry Pis.

genewitch 2y ago
i have an rpi zero w (i think) that came as part of a voice kit - a cardboard box, an rgb led arcade button, a speaker, and a really COOL mems multi-microphone array pi zero hat. Out of the box it had raspbian or whatever on it, and a config file you edited while following instructions that came in the kit - you have to set up a firebase account, authorize the "app" on the pi, and so on. It basically gave you access to the google ecosystem in a tiny cardboard box, like a cube a baseball might fit in. It didn't have a wakeword, you'd hit the arcade button (which had a dim color when inactive), it'd pulse white or blue, and then the assistant voice would ask whatever google assistants ask, and the light would change red, and you could ask your question, the light would pulse, and a second or two later, the assistant would talk, replying to your request or whtaever. now, i am working from memory, so it may have been amazon services, but i am unsure if they allow third party access, and it may have immediately waited for your question when you pushed the button. I don't remember. If i can find one of my boxes, in a closet gathering dust next to the rest of my rpi, i'll at least try to get the model/name of the microphone array hat, because i think that's the part that will make a DIY voice assistant work rather than be a curiosity. mine stopped working after a few weeks, and i couldn't ever figure out why. I think maybe google or whatever wanted money for API usage, or they changed the rules of how firebase worked... but the hardware still works, it just doesn't "wake up" when you push the button. the logs show the button push, etc. anyhow, i got the last two at target like 5(?) years ago, for $10 each. i think it was called "AIY Voice Kit"
- paulcole 2y ago
  I’m not sure if you’re telling me that I’m right or that I’m wrong.

bhhaskin 2y ago

Like Home Assistant voice? https://www.home-assistant.io/voice-pe/ https://www.home-assistant.io/voice-pe/

syntaxing 2y ago
I have one, its only wake word detection, microphone, and HA integration. The STT, LLM, and TTS is completely independent of the Voice box.
- joshstrange 2y ago
  Independent and optionally local, which is the important part. I can use my HA Voice with a Local LLM, OpenAI, or Home Assistant cloud. Similar I can swap out TTS and STT with local versions if I want.

raphman 2y ago

The guys at Jolla (developer of Sailfish OS smartphone OS) are currently trying something like this¹: https://www.jollamind2.com/ https://www.jollamind2.com/ ¹) however, their focus is less on home automation, more on personal information management, AFAIK.

nextts 2y ago

Sell me the flash to push into a rpi and an STL to print please.